The measurements of the 2011 performance are composed of eight indicators. All the indicators represent three different criteria of scientific paper performance: research productivity, research impact, and research excellence. Table 1 lists the indicators and shows the respective weightings of each indicator.
Table 1: The Criteria, Indicators, and Their Respective Weightings Used for the Overall Performance Based Ranking
| Criteria |
2011 Overall Performance Indicators |
Weighting |
| Research productivity |
Number of articles of the last 11 years (2000-2010) |
10 |
20 |
| Number of articles of the current year (2010) |
10 |
| Research impact |
Number of citations of the last 11 years (2000-2010) |
10 |
30 |
| Number of citations of the last 2 years (2009-2010) |
10 |
| Average number of citations of the last 11 years (2000-2010) |
10 |
| Research excellence |
h-index of the last 2 years (2009-2010) |
20 |
50 |
| Number of Highly Cited Papers (2000-2010) |
15 |
| Number of articles of the current year in high-impact journals (2010) |
15 |
Research Productivity:
The number of articles published in peer-reviewed academic journals is frequently used to indicate the productivity of a research institution. To objectively represent a university’s on-going and current research productivity, this ranking system employs two indicators: the number of articles of the last 11 years (2000-2010), and the number of articles of the current year (2010).
“Number of articles of the last 11 years” draws data from ESI, which include 2000-2010 statistics of articles published in journals indexed by SCI and SSCI. “Number of articles of the current year” relies on the 2010 data obtained from SCI and SSCI, which were searched between January 1 and January 31, 2011.
Research Impact:
The number of citations to a particular academic article within a specific time frame is a commonly accepted indicator for that article’s impact. This ranking system considers both the long-term and short-term impact of a particular research and seeks to provide a fairer representation of a university’s research impact regardless of its size and faculty number. Thus, this ranking system measures research impact by: the number of citations of the last 11 years, the number of citations of the last 2 years, and the average number of citations of the last 11 years.
“Number of citations of the last 11 years” draws 2000-2010 citation statistics from ESI. “Number of citations of the last 2 years” draws 2009-2010 citation statistics from SCI and SSCI in WOS, which include citation statistics updated to the dates of retrieval. “Average number of citations of the last 11 years” is the number of citations in the last 11 years divided by the number of articles in the last 11 years.
Research Excellence:
This ranking system assesses each university’s research excellence by the following indicators: the h-index of the last 2 years, the number of Highly Cited Papers from ESI, and the number of articles of the current year in high-impact journals (Hi-Impact journal articles). “h-index of the last 2 years” measures both the quantity and quality of a university’s research via the use of the 2009-2010 data from SCI and SSCI. Employing Hirsch’s (2005) concept of h-index, a university has index h if h of its Np papers in the last two years have at least h citations each and the other (Np – h) papers have ≦h citations each.
“Number of Highly Cited Papers” utilizes data from ESI, which include statistics of “Highly Cited Papers” from 2000 to 2010. ESI defines Highly Cited Papers as SCI /SSCI-indexed papers that are cited most (in the top 1% of the total papers indexed in the same year) within the last 11 years.
“Number of articles of the current year in high-impact journals” employs data from JCR, which supplies the impact factor of each journal in its subject field. The impact factor of a journal is the number of citations to the papers published in that particular journal within the previous two years divided by the number of that journal’s papers within the previous two years. A journal with a higher impact factor means its articles are more frequently cited by other journals, thus suggesting its higher scholarly value. This ranking system defines high-impact journals as journals whose impact factors are ranked as the top 5% of the total journals within a specific subject category. With high-impact journal lists derived from JCR, this ranking system is able to count the numbers of each university’s articles published in high-impact journals by subject.
Reference
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output.
Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.