BIBLIOMETRICS AND NEW TRENDS IN SOCIAL SCIENCES, ART&HUMANITIES RESEARCH ASSESSMENT:
 PERSPECTIVE FROM VIETNAM

Phạm Thị Ly
Centre for Human Resource Development, VNU-HCM

Acknowledgment

I would like to acknowledge and thank Prof Tuan V.Nguyen for his outstanding insights and suggestions that were provided, especially thank Prof. Dennis Berg for helpful review and editing English version. I sincerely thank my colleagues Prof. Bui Manh Hung and Binh Nguyen for their continued support and friendship. Of course, any mistakes or wrong-headed thinking that might remain in this paper, I accept responsibility for it.

Astract

Based on bibliometrics data, this essay summarizes the disparity in the Social Sciences and Art/Humanities research outputs in the world. The data illustrate the limitations of scientometrics in research assessment due to language and regional bias.  The author also points out a new trend in the improvement of bibliometrics methodologies, which adds qualitative criteria into the research assessment process. In the discussion, the author proposes recommendations for improving the quality of Social Sciences and Arts&humanities research in Vietnam.

Keywords: bibliometrics, social sciences, arts, humanities, research assessment

Introduction

Bibliometrics, a crucial subarea  of scientometrics, can be considered a rather newly developed discipline even though the very first analysis of citations  was reported  in 1961[1]. While remaining a useful tool and making an undeniable positive contribution to the assessment of research output, the bibliometrics system has significant limitations when applied to the fields of social sciences, art and humanities. Such limitations hinder a broad and accurate perspective on the whole picture of research performances in social sciences, art & humanities in the world. More importantly, these limitations have a negative impact on research assessment and lead to wrong interpretations of research activities and funding. As such, they fail to provide the encouragement of true excellence in research in the social sciences, art & humanities. This essay addresses the limitations in the application of bibliometrics in the social sciences and arts & humanities as well as discussing the new trends in attempts to solve these problems and to strengthen social science research. From a Vietnamese perspective, the author proposes some recommendations, which if implemented, could improve research capacities in the social science (sciences), arts and humanities, as well as increase publications in these fields in international journals.

Data and Methodologies

Most quantitative data used in this paper were abstracted and synthesized from ISI’s online citation index, the Web of Science (WoS). To retrieve the number of publications, the SCI-Expanded, SSCI, A&HCI databases within the ISI system and Elsviere’s Scopus system were used. Other data are from secondary sources. Using meta-analysis methodology, the author compares and analyzes various sources of data; tries to abstract and interpret emerging phenomenon in bibliometrics in the world and their impacts on social sciences, art and humanities research activities.

Results

Brief of Recent Situation and the Need for Social Sciences, Art & Humanities in the World

In the foreword of the World Social Science Report 2010, Irina Bokova, Director-General of UNESCO, pointed out that Social Sciences are invaluable in promoting the agreement between nations and -in responding to the challenges of today and tomorrow and improving  the quality of life. The ideas and information they generate – how human population interact with one another, and by extension, with the environment – can therefore make a precious contribution to the formulation of effective policies to shape our world for the greater good.

However, the century we live in is also witnessing a huge disparity between the rich and the poor, developed countries and the developing countries. This is also true in creating knowledge. The disparity is especially clear between pure sciences and social sciences, and unfortunately, social scientific knowledge is often the least developed and at risk  in some parts of the world where it is most needed.

There are many reasons of the current situation.  Language bias for English in the international arena causes the absence of social sciences, art and humanities publications from developing countries in international peer reviewed journals. This is harmful (in) in two ways: first, language barriers hinder local social sciences academia from connections with state-of-the-art research in the world and leaves them in isolated situations among the international academic community. Second, international scholars/researchers might have a limited understanding about regional perspectives or other social realities that are embedded into particular languages and traditions.

Figure 1. Geographic distribution of journals indexed to Scopus Social Sciences, 2009  (Source: Michael Kahn, 2010)

The figure 1 shows the dominance of English speaking countries in the geographic distribution of the social sciences journals indexed in Elsviere’s Scopus. The United States of America alone accounts for almost half of all indexed social sciences journals in the world (43.1%); the United Kingdom accounts for 26.2%  and the rest of the world only accounts for 30.6% of the indexed journals. If we do not include Canada, France, Germany and the Netherlands, then the remaining 186 countries account for only 544 indexed journals, just 13.7% of the total[2].

This is both the cause and the effect of the huge disparity in social sciences publications in the world. Looking at the number of citations, the data contained in Table 1, the disparity is once again obvious.  In the Thomson Reuters database, the United States of America alone produced 40.15% of the total citations in social sciences and 41.6% in art and humanities over the world. Note that in science,  the U.S.A. accounts for only 22.8% of the total citations. The percent of citations from China are 1.94% (social sciences), 1.11% (art and humanities), and 6.89% (science). If we compare the number of citations to the corresponding population figures, the disparity between these countries is even much larger.

SCI-E: Science Citation Index-Expanded; SSCI: Social Science Citation Index;A&HCI: Art and Humanities Citation Index

Table 1: Number of citations of select countries and regions (2007) abstracted from the Thompson Reuters database

Elsviere’s Scopus provided a result that is quite consistent with the Thomson Reuters database for all countries with the exception of China. According to Scopus, North America and Western Europe account for 75.86%  of the citations in social sciences and 79.74% in  art and humanities. East Asia, South Asia and Pacific account for 11.71% in the social sciences and 6.55% in the arts and humanities. The rest of the world accounts for only 0.85% in the social sciences and 0.99% in the arts and humanities. Again, the disparity of language and in geographic distribution of the social scientific knowledge is clearly revealed in the current bibliometrics data.

The language barrier is a major reason but not the only one for the disparity. India is an English speaking country but its achievement in (the) social sciences and the art/humanities is even worse than what is found in China: the number of citations for India accounts for only 0.61% in the social sciences and 0.28% in  the  art/humanities.  The lack of productivity in the area of social sciences publications can be explained by examining  reasons found in traditions and regimes.

In the case of India, it is assumed that the colonial inheritance of the educational system focused on “retailing knowledge”- training people for using their skills as a machine, rather than on developing creativity which is essential for scientific performance. A liberal arts education is especially needed for training people for research development.  Research on applied science and technology might have financial motivation, but in social sciences research, it seems that the major motivation is to meet the need of knowledge curiosity and solve the problems for the public good. The benefit (that) social sciences research brings into a society is invaluable but when directly measured by tangible measurements like financial outcomes it  does not compare to scientific endeavors.

Another observation is that, both China and Japan have very low numbers of citations in the social sciences: China accounts for 0.61% (social sciences) and 2.28 % (art and humanities); while Japan has 1.46% and 0.61% respectively, though they have completely different political systems.  Again, note that the overall picture of social sciences publication is much more complicated and not easy to explain.

The large gap and the disparity in social sciences research publications among countries showed in the above data, however, suggest that measurement methodologies and strategies for improving research capacity in developing countries should be reconsidered.

Bibliometrics and research assessment

The academia  continues to evaluate the research excellence of individual scholars or of a country based on the number of publications and citations. In fact, the factor measured by the H index is also based on these data. The two well-known databases of such information are:

Thomson Reuter Web of Science, also known as ISI[3], which includes 13,455 journals and the combined Science Citation Index Expanded (SCI-E) back to 1900; as well as the Social Science Citation Index (SSCI), and the Arts and Humanities Citation Index (A&HCI) dating back to 1956.

The second database, Elsevier’s Scopus[4], is recently established and becoming more and more popular.  Regarding social sciences, Thomson Reuters indexed 2,759 titles of journals and 1,516 journals on art & humanities (by 2009) while Scopus indexed approximately 4,000 and 1,100 respectively. However, Scopus only dates from 1996. For an objective comparison one usually takes data from both of these  sources.

As mentioned above, most of the social science journals are in English, the reputed journals usually reject almost 90% of the submissions, sometimes for poor English. It is said that in non-English countries, English is more challenging to the social science researchers than to the scientists. Therefore, there might be valuable research work that has no chance to be published in international peer reviewed journals. There is no perfect measurement of research output Thomson Reuters and Scopus are no exceptions. The most obvious weakness is the language and regional bias of these databases. This seriously affects the evaluation of research achievements in the social sciences, arts & humanities in non-English speaking countries. More importantly, it hinders transnational collaboration(s) opportunities between social sciences researchers in these countries.

Due to the above limitations, some countries have been establishing their own national bibliometrics, such as China, Russia and Spain. From the late 1990s, Chinese Science Citations Indices and Social Sciences Citation Indices have applied a quantitative assessment of research performance, a significant factor to consider in research funding allocation (Wei Lili, 2010). The Russian Federation and Spain have also created national databases, with the same bibliometrics methodologies as the WoS (Pipiya, 2010). Due to the limited number of journals indexed in these systems from these countries, such national databases are very helpful for providing an overall observation of the research activities including social sciences, art & humanities.

New Trends in Improving Social, Art and Humanities Research Assessment

Assessment of research output is a necessary requirement. The information is needed for research and development policy strategies and to form a foundation for research funding allocation decisions.  However, the dominant bibliographical databases Thomson WoS and Scopus have strong linguistics and geographical bias. They do not cover the whole picture in that they do not fully represents research performance throughout the world.

In 2007, UNESCO created a guide for improving research assessment in developing countries. These guidelines provide a comprehensive evaluation, to assess research outcomes in connection with levels of investment such as the number of researchers and (the) size of research funding.

Regarding the qualitative versus the quantitative evaluation of research, it is impossible to avoid metrics-based methods. Compared with the natural  sciences, the bibliometrics methods used for social sciences and humanities cause more problems and concerns. First, Thomson Reuters’ WoS only counts journal articles while books and serials, and or other forms of scholarly communication such as reports are not accounted for, in spite of their  significant importance (Archambault  E. and Larivière, 2010). On the other side, many research works in the social sciences, especially action research, aimed to solve local problems rather than add to the collection of published literature. Such research(s) might not attract the attention of the international academics and thus have little chance of being published in indexed high impact journals.  In these cases, qualitative assessments are better (Weingar and Schwechheimer, 2010). “By measuring impact in terms of papers cited by academic peers, citations and bibliometrics indices can ignore research that affects policy, legislation or regulatory regimes, technological or social intervention, business creation and employment” (Ellen Hazelkorn, 2010, p. 256) . These indicators could be improved so they are more appropriate and better for social sciences and humanities. Archambault et al, 2009 pointed out that papers in medical, natural science and arts & humanities fields are (rapidly) cited rapidly after publication, but the citation frequency drops fairly quickly. Papers in the social sciences are less readily cited upon publication and only reach their peak citation some ten years after publication. Archambault, et al suggested that we should allow for a longer citation window when examining the impact of research in the social sciences rather than counting the number of citations only (Archambault and Larivière, 2010).

Moreover, social sciences research output in the non-Anglo-Saxon world are known and cited in their original language rather than in English version (Gingras, 1984; Line, 1999) and therefore, Thomson Reuters WoS and Scopus databases, mostly based on English, are incapable of revealing the full impact of social sciences research even when they are published in international peer review journals.

The UK’s Research Assessment Exercises (RAE) could be seen as multi-faceted assessment, combining qualitative and quantitative indicators. Conducted by a state agency every 4 years since 1986, RAE panelists and experts evaluate information on inputs and outputs provided by research institutions such as overall head count, full time equivalent, expenditure on R&D, and other indicators that reveal the research environment of the institutions. Panelists who review these reports include scientists and users of research outputs in certain areas. However, RAE was also accused of making research more ‘short-term’ due to the pressures to publish, and of encouraging  bad practices (split papers, duplicate publications, mushrooming of new journals and so on). (Alis Oancea, 2010).

Therefore, the value of a research work in the social sciences should not be assessed only by bibliometrics data which are based on publications in indexed international peer review(ed) journals. Though we would not ignore this quantitative approach, such data should be given great consideration and be supplemented by a qualitative perspective; i.e., the value of  social sciences research might also be measured by its contributions to making positive changes in policies, the economy and society.

As to qualitative assessment of the quality of social sciences and humanities research, Weignart and Schwechheimer (2010, p. 250) suggested two sets of criteria: organizational and intellectual.

Organizational criteria are about both condition for research and expressions of research culture. A healthy-social sciences and humanities culture should have sufficient size to allow for a plurality of approaches and methods. Crucial questions are whether the social sciences and humanities have normal department status, and whether the social sciences and humanities are represented in national scholarly   associations and professional societies.

Intellectual criteria are at the core of any assessment of the health and quality of a discipline or research field. Social sciences and humanities do not have to be integrated into an international scholarly discourse to  the same degree as the natural sciences in order to be qualitatively of a high standard. Those research  activities that are more narrowly focused on national and culturally specific subject matters and topics must be judged on their own merits. They must, above all, exhibit originality in their theories and methodologies. Indications of this are lively intellectual debates among the relevant scholarly communities, a recognizable progress of research over time, and in the ideal case, an impact on public debates (Weignart and Schwechheimer. 2010, p. 250).

Discussions (on) Implications of the Bibliometrics and Unexpected Results

Scientific research activities, like other fields, contain fierce competiveness. Bibliometrics is built to provide an objective approach to assess research outputs of an individual scholar, a research agency, a university or a nation.  Based on such outcomes, research funding is allocated, the reputation or credentials of a scholar or an institution are assessed. Like other measurements, bibliometrics reflect only a part of research activity production due to its complexity. Bibliometric assessment has been continuously criticized and improved over time for multi-dimensional measurement, reflecting more accurately the reality of research activities.

It is obvious that bibliometrics contributes greatly to the research activity by putting a pressure that makes scientists, scholars and faculty creates more research production. Through publications, the scholarly exchanges are facilitated and researchers become connected for solving common issues.

However, on the other side, as expressed in following well known saying, “Publish or perish,” the culture in some institutions has been creating unhealthy competitiveness in academic activities. Due to the pressure for publications, especially in emerging countries, some scientists plagiarize or fake data, split articles into many pieces, etc. (Mara Hvistendahl. 2006; Harvey, L. 2008; Liu, N. C., and Cheng, Y. 2005). Having publications become the major motivation of research performance is not worthy encouragement, especially in the social sciences and humanities. The nature of research that is search for knowledge and truth now is lost in the pressure for increased bibliometrics.

Recommendations for Increasing International Integration of Social Sciences and Humanities Research in Vietnam

Even though the limitations and problems of the bibliometrics approach in social sciences assessment has been pointed out, one cannot deny the importance of increasing Vietnamese social scientific work publications in indexed international peer review(ed) journals. It will help close the gaps between Vietnam and other countries and promote understanding between nations, especially when many global issues such as environmental protection, climate change, education and brain drain, need to be addressed by transnational collaborations.

The contributions of Vietnamese researchers in the social sciences are still modest. Only 5% of the total Vietnamese scientific publications are published from 1996-2005 in the international academic journals and of the 3456 peer reviewed articles there are only 1.6% in the social sciences. That means, about 5 articles per year are appearing in  international social sciences journals.

Figure 2. Publications from Vietnam in Thomson Reuters WoS database, 1996-2005 (Total = 3456).

Source: Nguyễn Văn Tuấn, Garvan Institute of Medical Research, Australia.

To improve the quality of research in social sciences and humanities in Vietnam, there are several possibilities that could be (and should be) considered:

- Strengthen research skills by training workshops in quantitative methodologies and academic writing skills according to international norms and standards.

- Creating bilingual journals:

• First, to publish only articles of the international academics in certain fields and topics that are relevant to Vietnam and in the Vietnamese academic community’s interest. Design a bilingual language format: English and Vietnamese, aimed to help Vietnamese social scientists and researchers becoming familiar with international approaches, expression, methodologies and updated knowledge.
• Second, also to publish a bilingual journal, produced by leading research institutes or universities in certain fields. It follows an international peer review principle, and publishes the research work of Vietnamese academics in social sciences and humanities. These publications would be limited to only those which are internationally reviewed by their peers around the world.  This journal is crucial to bringing Vietnamese academic research forward into international norms and standards and integrating them into the global academic community.

- Promoting transnational collaboration research in social sciences and humanities especially in education to provide data for evidence-based policy making. Cooperation in research is the best way to build up research capacity for research staff.

- The government should have polices that encourage academic institutions to provide better intellectual resources for faculty. For example, institutional libraries should obtain subscriptions to resources like JSTOR, so that Vietnamese faculty can easily access international peer reviewed journals. In the list of JSTOR subscribers, there are 7000 institutions from 159 countries including numerous institutions in the U.S.A., many Asian coutries, a few Africa countries, even the very poor ones. However, there is no institution from Vietnam in this list, not even the two Vietnam National Universities.

- Establishing a national bibliometrics database for Vietnam, which indexes publications and citation nationally and internationally of Vietnamese scientists in the social sciences and arts & humanities. Data from citation indexes can be analyzed to determine the popularity and impact of specific articles, authors, and publications and help policy making, as well as to assess the excellence in research performance.

Conclusion

When a butterfly flaps his wings in Beijing it might cause hurricane patterns in New York; more than ever before the social sciences and humanities academics need to be connected to respond more effectively to  global issues. Vietnam should have a long term vision to develop social sciences and humanities where it is most keenly needed. After decades of significant progress in science and technology, people have stepped forward to creating prosperity and wealth, as well as to generate a huge amount of knowledge. However at the same time people also create numerous problems that might lead to human annihilation: exhausted exploitation of nature resources and destruction of the environment; the disparities and larger and larger gaps between the rich and the poor that might lead to economic crisis and unstable society; the inability of education to prepare people for a productive life and a world full of harmony. All of these problems need to be examined seriously by social sciences to provide an evidence based foundation for policy makers.

Bibliometrics need to be improved to reflect more accurately research outcomes. It is just a tool, not a purpose itself of the research activities; therefore it will be best if we interpret the bibliometrics results with great caution because of the limitations pointed out above. Having bibliometrics results become our goal will lead us in the wrong direction in strategic planning for research and development in the social sciences. And top the contrary, if we underestimate the importance of international bibliometrics databases, we will stay outside of the international academic arena forever instead of having common words and purpose(s) with international colleagues. International integration is not only for high ranked universities but  serves to benefit society from the  inheritance of knowledge that is accumulated around the world. International integration will contribute to improving capacity of research staff , contribute the best knowledge and practices for solving national problems and promote understanding between nations.

REFERENCES

Alis Oancea. 2010. Research Assessment in United Kingdom. World Social Science Report, UNESCO. Chapter 7, pp. 259-260.

Archambault, É., Campbell, D., Gingras, Y. and Larivière, V. 2009. “Comparing Bibliometric Statistics Obtained From The Web Of Science And Scopus”. Journal of the American Society for Information Science and Technology, Vol. 60, No. 7.

Archambault Eric and Vincent Larivière , 2010. The Limits of Bibliometrics For Analysis of The Social Sciences And Humanities Literature.  World Social Science Report, UNESCO. Chapter 7, pp. 251-254.

Archambault, É., Vignola-Gagné, É., Côté, G., Larivière, V. and Gingras, Y. 2006. Benchmarking Scientific Output In The Social Sciences And Humanities: The Limits of Existing Databases. Scientometrics, Vol. 68, No. 3, pp. 329–42.

Australian Government. 2006. Research Quality Framework: Assessing the Quality and Impact of Research in Australia. Commonwealth of Australia. http://www.unisa.edu.au/rqie/rqfhistory/docs/

Australian Research Council. 2009. The Excellence in Research for Australia (ERA) Initiative (Canberra). http://www.arc.gov.au/era/default.htm (Truy cập 13/10/2011).

Dadds, M. and Kynch, C. 2003. The Impact Of The RAE 3b Rating On Educational Research In Teacher Education Departments. Research Intelligence, No. 84.

Deem, R., Mok, K. H. and Lucas, L. 2008. Transforming Higher Education In Whose Image? Exploring The Concept of The ‘World-Class’ University In Europe And Asia. Higher Education Policy, Vol. 21, pp. 83–97.

Elsevier. Scopus Overview: What is it? http://www.info.scopus.com/overview/what(Truy cập 1 -10- 2011.)

Elton, L. 2000. The UK Research Assessment Exercise: Unintended Consequences. Higher Education Quarterly,Vol. 54, No. 3, pp. 274–83.

Peter Weingart and Holger Schwechheimer. 2010. Conceptualizing And Measuring Excellence In Social Sciences And Humanities. World Social Science Report, UNESCO.

European Commission. 2006. RICARDIS: Encourage Corporate Measuring and Reporting on Research and OtherForms of Intellectual Capital (Brussels). http://ec.europa.eu/invest-in-research/pdf/download_en/2006-2977_web1.pdf

Glänzel, W. 1996. A Bibliometric Approach To Social Sciences. National Research Performance In 6 Selected Social Science Areas, 1990–1992. Scientometrics, Vol. 35, No. 3, pp. 291–307.

Glänzel, W. and Schoepflin, U. 1999. A Bibliometric Study Of Reference Literature In The Sciences And Social Sciences. Information Processing and Management, Vol. 35, No. 1, pp. 31–44.

Gingras, Y. 1984. The Value Of A Language for A Field of Research. Recherches Sociographiques, Vol. 25, No. 2, pp. 285–96.

Hackett, E. J. 1990. Science As A Vocation In The 1990s: The Changing Organizational Culture Of Academic Science.Journal of Higher Education, Vol. 61, No. 3, pp. 241–79.

Hare, P. G. 2003. The United Kingdom’s RAE: Impact On Institutions, Departments, Individuals. Higher EducationManagement and Policy, Vol. 15, No. 2, pp. 43–61.

Harvey, L. 2008. Rankings Of Higher Education Institutions: A Critical Review. Quality in Higher Education, Vol. 14,No. 3, pp. 187–207.

Hazelkorn, E. 2008. Learning To Live With League Tables And Ranking: The Experience Of Institutional Leaders. HigherEducation Policy, Vol. 21, No. 2, pp. 193–215.

Hicks, D. M. 1999. The Difficulty of Achieving Full Coverage of International Social Science Literature And The Bibliometric Consequences. Scientometrics, Vol. 44, No. 2, pp. 193–215.

Higher Education Funding Council of England (HEFCE). 2008. Research Assessment Exercise. http://www.rae.ac.uk

Huang Changzhu and Huang Yufu. 2008. Social Science and Humanities Abroad: Policy and Management. Beijing, Social Sciences Academic Press, p. 313.

Huang Ping. 2010. The Status of the Social Sciences in China. World Social Science Report, UNESCO. Chapter 2, pp. 73-76.

IHEP. 2009. Impact of College Rankings on Institutional Decision Making: Four Country Case Studies (Washington, DC.)

http://www.ihep.org/assets/files/publications/g-l/ImpactofCollegeRankings.pdf

Ji Liang. 2005. The Development Of Social Science Bibliometrics. Management and Review of Social Sciences (Beijing), No. 4, pp. 80–85.

Leydesdorff, L. 2003. Can Networks Of Journal-Journal Citation Be Used As Indicators Of Change In The Social Sciences? Journal of Documentation, Vol. 59, No. 1, pp. 84–104.

Line, M. B. 1999. Social Science Information – The Poor Relation. 65th IFLA Council and General Conference. Bangkok, 20–28 August.

Liu, N. C., and Cheng, Y. 2005. The Academic Ranking Of World Universities – Methodologies And Problems. Higher Education in Europe, Vol. 30, No. 2, pp. 127–36.

Mara Hvistendahl, "Scientific Copy Cats: Is China's rash of plagiarism a deeply rooted cultural issue?" SEED, http://www.seedmagazine.com/news/2006/05/scientific_copy_cats.php

Michael Kahn.2010. Measure for Measure: Quantifying the Social Sciences. World Social Science Report. UNESCO. P.359.

Wei Lili.2010. Funding and Assessment of Humannities and Social Science Research in China. World Social Science Report. UNESCO. Chapter 7, pp 269-272.

[1] http://en.wikipedia.org/wiki/Bibliometrics

[2] Social sciences also included Economics, Econometrics and Finance, Business, Management and Accounting, Psychology.

[3] http://www.isiwebofknowledge.com

[4] See: Elsevier, Scopus Overview: What is it? http://www.info.scopus.com/overview/ (Accessed 1 -10- 2011.)