This project is to evaluate and rank the scientific paper performance for the top 500 universities worldwide. The performance measures are composed of eight indicators to assess a university’s overall scientific paper performance along three criteria: research productivity (accounting for 20%), research impact (30%), and research excellence (50%). Different from the QS World University Rankings by Times Higher Education Supplement which focuses on university ranking, and the Academic Ranking of World Universities by Shanghai Jiao Tong University which focuses on academic ranking, this project focuses on scientific papers performance ranking. The emphasis on current research performance makes the indicators a fairer one than some traditional indicators such as a university’s reputation or the number of Nobel Prize winners affiliated with that university, which tends to favor universities with longer histories or universities in developed countries.
This project employs quantitative data drawn from Science Citation Index (SCI) and Social Sciences Citation Index (SSCI) to evaluate the scientific paper performances of world universities. Today, publishing in international academic journals is the predominant mode of scientific research output. Statistics on the articles published in international academic journals provide an objective representation of each university’s research performance. Furthermore, since the Institute for Scientific Information (ISI) began to publish SCI and SSCI in 1961, the two databases have grown over the years to include a good number of academic journals that are both international in scope and comprehensive in subject representation. However, results of this project may favor those universities’ performances in sciences and social sciences and under represent their performances in humanities research. Although ISI also publishes Arts & Humanities Citation Index (A&HCI), this project excludes this database because it may fail to objectively and accurately represent the research performance of humanities researchers worldwide – A&HCI mainly indexes journals in English language, while humanities researchers often publish in their native languages and in various forms of publications (e.g., books). Analyses of the data obtained from SCI and SSCI allow fairer comparisons across universities globally. Generally speaking, the indicators used in this project have at least the following three features:
1.It emphasizes the quality of research; the indicators assessing research quality (research impact and research excellence) account for 80% of the performance score.
Research impact and research excellence evaluate the quality of a university’s research output. The calculation of each university’s score is based on the number of citations of its published articles, h-index of the last two years, number of Highly Cited Papers, and number of articles published in high-impact journals (Hi-Impact journal articles)(these indicators will be explained further in the Indicators section).
2.It balances biases due to a university’s size or faculty number.
Traditionally the size of a university affects its ranking when the number of articles is used as a sole indicator for research output. Because the number of articles is closely tied to the number of faculty members, rankings employing numbers of articles often favor larger universities. This project corrects that flaw by incorporating the average number of citations and h-index in the calculation of universities’ performance scores (explained below). The inclusion of the two indicators, which together account for 30% of the total score, balances the assessments of quality and quantity of research and provides a fairer representation of a university’s performance regardless of its size.
To further show the possible influences of university size on ranking, in addition to the original ranking, the 2008 project also provides an adjusted ranking based on university size for reference. Four indicators significantly affected by university size are normalized by each university’s number of full-time faculty; these include the number of articles of the last 11 years, number of articles of the current year, number of citations of the last 11 years, and the number of citations of the last two years. This project employs faculty numbers obtained from the following sources (listed by priority in usage): numbers of university faculty staff FTE obtained from the QS (Quacquarelli Symonds) Ltd, numbers of full-time faculty obtained from university Web sites, numbers of faculty registered at each country’s higher education administration, and numbers of faculty/staff obtained from university Web sites.
3.It takes into account a university’s short-term research performance (constituting 55% of the score), thus ensuring a fairer comparison between universities of various lengths of history.
The indicators used in this project seek to represent both the long-term and short-term research performances of a university. The inclusion of indicators evaluating short-term performances corrects the flaws resulted from undifferentiating indicators that favor universities with longer histories. These short-term performance indicators include: the number of articles of the current year, the number of citations of the last two years, h-index of the last two years, and the number of articles in high-impact journals in the current year.
The indicators employed in the 2008 ranking differ slightly from those of 2007. The 2008 ranking removes one indicator (“number of subject fields where the university demonstrates excellence”), and consequently the weighting of each indicator representing research excellence has been adjusted. The reason for the removal of the said indicator was that, starting from 2008, this project aims to provide annual ranking based on universities’ performances in specific subject categories aside from the ranking of the overall performances. The removed indicator is less useful in the subject-based ranking, and its significance is also lower for the overall performance ranking now that subject based rankings are conducted. Thus, the same indicators can be used for both overall and field performance for world universities. Research excellence still accounts for 50% of the total score as was the case for 2007. Consequently, two other research excellence indicators (“number of highly cited papers” and “number of articles of the current year in high-impact journals”) now account for a higher percentage of the total score (15% each; was 10% in 2007).
Through the use of the proposed indicators, this project attempts to objectively portray the current research development of the worldwide universities in comparison to universities worldwide. The relative strengths and weaknesses revealed in the ranking provide insights into the higher education administration and resources allocation