On the effects of institutional size in university classifications

the case of the Shanghai ranking

Domingo Docampo, Lawrence Cram

    Research output: Contribution to journalArticleResearchpeer-review

    12 Downloads (Pure)

    Abstract

    University rankings frequently struggle to delineate the separate contributions of institutional size and excellence. This presents a problem for public policy and university leadership, for example by blurring the pursuit of excellence with the quest for growth. This paper provides some insight into the size/excellence debate by exploring the explicit contribution of institutional size to the results of the Shanghai ranking indicators. Principal components analysis of data from the Shanghai ranking (2013 edition) is used to explore factors that contribute to the variation of the total score. The analysis includes the five non-derived ARWU indicators (Alumni, Award, HiCi, S&N and PUB) and uses the number of equivalent full-time academic staff (FTE) as a measure of size. Two significant but unequal factors are found, together explaining almost 85 % of the variance in the sample. A factor clearly associated with the size of the institution explains around 30 % of the variance. To sharpen the interpretation of the smaller factor as a measure of the effect of size, we extend the analysis to a larger set of institutions to eliminate size-dependent selection effects. We also show that eliminating outlying universities makes little difference to the factors. Our inferences are insensitive to the use of raw data, compared with the compressed and scaled indicators used by ARWU. We conclude that around 30 % of the variation in the ARWU indicators can be attributed to variation in size. Clearly, size-related factors cannot be overlooked when using the ranking results. Around 55 % of the variation arises from a component which is uncorrelated with size and which measures the quality of research conducted at the highest levels. The presence of this factor encourages further work to explore its nature and origins.
    Original languageEnglish
    Pages (from-to)1325-1346
    Number of pages22
    JournalScientometrics
    Volume102
    Issue number2
    DOIs
    Publication statusPublished - Feb 2015

    Fingerprint

    ranking
    university
    Principal component analysis
    university ranking
    alumni
    edition
    public policy
    leadership
    staff
    interpretation

    Cite this

    Docampo, Domingo ; Cram, Lawrence. / On the effects of institutional size in university classifications : the case of the Shanghai ranking. In: Scientometrics. 2015 ; Vol. 102, No. 2. pp. 1325-1346.
    @article{cf0c9502c7144d4e8752fc71f54a6daf,
    title = "On the effects of institutional size in university classifications: the case of the Shanghai ranking",
    abstract = "University rankings frequently struggle to delineate the separate contributions of institutional size and excellence. This presents a problem for public policy and university leadership, for example by blurring the pursuit of excellence with the quest for growth. This paper provides some insight into the size/excellence debate by exploring the explicit contribution of institutional size to the results of the Shanghai ranking indicators. Principal components analysis of data from the Shanghai ranking (2013 edition) is used to explore factors that contribute to the variation of the total score. The analysis includes the five non-derived ARWU indicators (Alumni, Award, HiCi, S&N and PUB) and uses the number of equivalent full-time academic staff (FTE) as a measure of size. Two significant but unequal factors are found, together explaining almost 85 {\%} of the variance in the sample. A factor clearly associated with the size of the institution explains around 30 {\%} of the variance. To sharpen the interpretation of the smaller factor as a measure of the effect of size, we extend the analysis to a larger set of institutions to eliminate size-dependent selection effects. We also show that eliminating outlying universities makes little difference to the factors. Our inferences are insensitive to the use of raw data, compared with the compressed and scaled indicators used by ARWU. We conclude that around 30 {\%} of the variation in the ARWU indicators can be attributed to variation in size. Clearly, size-related factors cannot be overlooked when using the ranking results. Around 55 {\%} of the variation arises from a component which is uncorrelated with size and which measures the quality of research conducted at the highest levels. The presence of this factor encourages further work to explore its nature and origins.",
    author = "Domingo Docampo and Lawrence Cram",
    year = "2015",
    month = "2",
    doi = "10.1007/s11192-014-1488-z",
    language = "English",
    volume = "102",
    pages = "1325--1346",
    journal = "Scientometrics",
    issn = "0138-9130",
    publisher = "Akademiai Kiado Rt.",
    number = "2",

    }

    On the effects of institutional size in university classifications : the case of the Shanghai ranking. / Docampo, Domingo; Cram, Lawrence.

    In: Scientometrics, Vol. 102, No. 2, 02.2015, p. 1325-1346.

    Research output: Contribution to journalArticleResearchpeer-review

    TY - JOUR

    T1 - On the effects of institutional size in university classifications

    T2 - the case of the Shanghai ranking

    AU - Docampo, Domingo

    AU - Cram, Lawrence

    PY - 2015/2

    Y1 - 2015/2

    N2 - University rankings frequently struggle to delineate the separate contributions of institutional size and excellence. This presents a problem for public policy and university leadership, for example by blurring the pursuit of excellence with the quest for growth. This paper provides some insight into the size/excellence debate by exploring the explicit contribution of institutional size to the results of the Shanghai ranking indicators. Principal components analysis of data from the Shanghai ranking (2013 edition) is used to explore factors that contribute to the variation of the total score. The analysis includes the five non-derived ARWU indicators (Alumni, Award, HiCi, S&N and PUB) and uses the number of equivalent full-time academic staff (FTE) as a measure of size. Two significant but unequal factors are found, together explaining almost 85 % of the variance in the sample. A factor clearly associated with the size of the institution explains around 30 % of the variance. To sharpen the interpretation of the smaller factor as a measure of the effect of size, we extend the analysis to a larger set of institutions to eliminate size-dependent selection effects. We also show that eliminating outlying universities makes little difference to the factors. Our inferences are insensitive to the use of raw data, compared with the compressed and scaled indicators used by ARWU. We conclude that around 30 % of the variation in the ARWU indicators can be attributed to variation in size. Clearly, size-related factors cannot be overlooked when using the ranking results. Around 55 % of the variation arises from a component which is uncorrelated with size and which measures the quality of research conducted at the highest levels. The presence of this factor encourages further work to explore its nature and origins.

    AB - University rankings frequently struggle to delineate the separate contributions of institutional size and excellence. This presents a problem for public policy and university leadership, for example by blurring the pursuit of excellence with the quest for growth. This paper provides some insight into the size/excellence debate by exploring the explicit contribution of institutional size to the results of the Shanghai ranking indicators. Principal components analysis of data from the Shanghai ranking (2013 edition) is used to explore factors that contribute to the variation of the total score. The analysis includes the five non-derived ARWU indicators (Alumni, Award, HiCi, S&N and PUB) and uses the number of equivalent full-time academic staff (FTE) as a measure of size. Two significant but unequal factors are found, together explaining almost 85 % of the variance in the sample. A factor clearly associated with the size of the institution explains around 30 % of the variance. To sharpen the interpretation of the smaller factor as a measure of the effect of size, we extend the analysis to a larger set of institutions to eliminate size-dependent selection effects. We also show that eliminating outlying universities makes little difference to the factors. Our inferences are insensitive to the use of raw data, compared with the compressed and scaled indicators used by ARWU. We conclude that around 30 % of the variation in the ARWU indicators can be attributed to variation in size. Clearly, size-related factors cannot be overlooked when using the ranking results. Around 55 % of the variation arises from a component which is uncorrelated with size and which measures the quality of research conducted at the highest levels. The presence of this factor encourages further work to explore its nature and origins.

    U2 - 10.1007/s11192-014-1488-z

    DO - 10.1007/s11192-014-1488-z

    M3 - Article

    VL - 102

    SP - 1325

    EP - 1346

    JO - Scientometrics

    JF - Scientometrics

    SN - 0138-9130

    IS - 2

    ER -