This project evaluates and ranks the scientific paper performance for the top 500 universities worldwide. Three criteria represented by eight indicators were used to assess a university’s overall scientific paper performance: research productivity (accounting for 20% of the score), research impact (30%), and research excellence (50%).
This annual ranking project began in 2007. Currently, aside from this project there are two other major university rankings: the QS World University Rankings by Times Higher Education Supplement (THES), and the Academic Ranking of World Universities by Shanghai Jiao Tong University (ARWU). In addition, Times Higher Education Supplement announced a ranking of the Asian universities in May, 2009. Different from THES that focuses on university ranking, and ARWU focusing on academic ranking, this project focuses on scientific papers performance ranking. The emphasis on current research performance makes the indicators used in this project more objective than traditional indicators such as a university’s reputation or the number of Nobel Prize winners affiliated with that university, which tend to favor universities with longer histories or universities in developed countries.
This project employs quantitative data drawn from Science Citation Index (SCI) and Social Sciences Citation Index (SSCI) to evaluate the scientific paper performances of world universities. Today, publishing in international academic journals is the predominant mode of scientific research output. Statistics on the articles published in international academic journals provide an objective representation of each university’s research performance. Furthermore, since the Institute for Scientific Information (ISI) began to publish SCI and SSCI in 1961, the two databases have grown over the years to include a good number of academic journals that are both international in scope and comprehensive in subject representation. However, it should be noted that results of this project may favor universities with better performances in sciences and social sciences and under-represent performances in arts & humanities research. Although ISI also publishes Arts & Humanities Citation Index (A&HCI), the database mainly indexes journals in English language, while arts & humanities researchers often publish in their native languages and in various forms of publications (e.g., books). Therefore, this project does not include the database because it may fail to objectively and accurately represent the research performance of arts & humanities researchers. Focusing on data obtained from SCI and SSCI allows for fairer comparisons across universities globally. The indicators used in this project have the following three characteristics:
1.Emphasize the quality of research - the indicators assessing research quality (research impact and research excellence) account for 80% of the performance score.
Research impact and research excellence evaluate the quality of a university’s research output. The calculation of each university’s score is based on the number of citations to its published articles, h-index of the last two years, number of Highly Cited Papers, and number of articles published in high-impact journals (Hi-Impact journal articles). These indicators will be explained further in the Indicators section.
2.Neutralize biases caused by the university size or faculty number.
Traditionally the size of a university affects its ranking when the number of articles is used as a sole indicator for research output. Because the number of articles is closely tied to the number of faculty members, rankings employing numbers of articles often favor larger universities. This project corrects that flaw by incorporating the average number of citations and h-index in the calculation of universities’ performance scores (explained below). The inclusion of the two indicators, which together account for 30% of the total score, balances the assessments of quality and quantity of research and provides a fairer representation of a university’s performance regardless of its size.
To further show the possible influences of university size on ranking, in addition to the original ranking this project also provides an adjusted ranking based on university size. Four indicators significantly affected by university size are normalized by each university’s number of full-time faculty; these include the number of articles of the last 11 years, number of articles of the current year, number of citations of the last 11 years, and the number of citations of the last two years. This project employs faculty numbers obtained from the following sources (listed by priority in usage): numbers of university faculty staff FTE obtained from the QS (Quacquarelli Symonds) Ltd, numbers of full-time faculty obtained from university Web sites, numbers of faculty registered at each country’s higher education administration, and numbers of faculty/staff obtained from university Web sites.
3.Take into account a university’s short-term research performance (constituting 55% of the score), thus ensuring a more objective comparison between universities of various lengths of history.
The indicators used in this project seek to represent both the long-term and short-term research performances of a university. The inclusion of indicators evaluating short-term performances corrects the flaws resulted from undifferentiating indicators that favor universities with longer histories. These short-term performance indicators include: the number of articles of the current year, the number of citations of the last two years, h-index of the last two years, and the number of articles in high-impact journals in the current year.
Through the use of the indicators, this project attempts to objectively compare the research performance and achievement of the universities worldwide. The relative strengths and weaknesses of a university revealed in the ranking provide insights into the higher education administration and resources allocation.