Thursday, 25 September 2014

Serious, Singapore NTU is Number One University; but ....

The United Nations agency, UNESCO, challenged the validity and reliability, and therefore the usefulness, of University Rankings.

Nanyang Technological University (NTU) in Singapore has secured top placing on a league table of the world's best young universities.  It has overtaken Hong Kong University of Science and Technology, who was No. one for the past two years, according to London-based educational consultancy Quacquarelli Symonds (QS). Of what value to Singapore is this NTU “achievement”?

Well, any good researcher would know that you will get what you measure, instead of what you want to claim the measure to mean.  So, what exactly does QS Rankings mean?

The United Nations agency, UNESCO, challenged the validity and reliability, and therefore the usefulness, of University Rankings:

“Global university rankings fail to capture either the meaning or divers qualities of a university or the characteristics of universities in a way that values and respects their educational and social purposes, missions and goals. At present, these rankings are of dubious value, are underpinned by questionable social science, arbitrarily privilege particular indicators, and use shallow proxies as correlates of quality.”

“These can often skew data in unexpected ways such as QS 2008's world rankings that helped put internationally-minded University College London ahead of Oxford, or the rise of Yale which has similarly been concentrating on its international programmes”.

Indeed, Universities Ranking is itself conceptually problematic.  It embraced an “idealised” model of University to be achieved and in so doing generalize the failure of most Universities to achieve it.  The World-Class University has NEVER existed as a concept, or as an empirical reality. The status of “World-Class University” as the gold standard is the normative social construct of the rankers themselves.

It makes little sense for all universities to aspire to a common “gold” standard, irrespective of socioeconomic needs, missions, goals, capacities and capabilities. The specific national conditions, realities and development challenges in their respective societies, and the diversity of social and educational purposes and goals that the universities in these societies must serve, require education systems and Universities to become characterized differentiated and diverse institutions. Institutional differentiation and diversity are valued over homogeneity with other Universities globally because such comparisons are immaterial and irrelevant to their unique Missions.  

Truth is, the term “University” now applies to various institutions with widely different functions, curriculum and characters”, and therefore, they can be expected to aspire to different “ideals” not captured by University Rankings purveyors.

In fact, even QS cautions against the use of the QS Ranking beyond its simple methodology and purpose “to serve the student consumer. Rankings allows the consumer to see how institutions stand against other universities." Adding: "As it became apparent that more and more undergraduate students were looking to study abroad, there was a need for an international comparison. We did not come about it from the point of view of an academic exercise with metrics."

This is a confession admitting to the fact that QS Rankings evolve around the metrics used to devise the tables including citations and peer review. The Rankers did not build their QS Rankings on any solid or vigorous foundation that would withstand the penetrative professional scrutiny of the Academics or Research Institutions which now used them to position themselves in spite of the lack of validity and reliability of these measures. Therein lies its fundamental conceptual and methodological flaw, confirming that the QS Ranking is therefore irrelevant and immaterial for any serious educational policy purpose.

In fact, QS rankers themselves were surprised at "the extent to which governments and university leaders use the rankings to set strategic targets. We at QS think this is wrong. Rankings are (just) a relative measure - if other universities do better and move up, you have to go faster."  It is just plain mindless stupidity, I may add.

QS launched the annual World University Rankings in 2004 with the then Times Higher Education Supplement (THES), at the time a Rupert Murdoch owned newspaper.  QS and THES parted company in 2005, and THES was renamed THE. Each now vies with the other to produce a new world ranking with the THE devising a new league table in collaboration with media group Thomson Reuters.

The QS World University Ranking Methodology 2014 involved Academic Reputation (40%), Employer Reputation (10%), Faculty/Student Ratio (20%), Citations per Faculty (20%), International Students Ratio (5%) and International Staff Ratio (5%).

It would be interesting to know whether the low number of Singaporean Professors was deliberate in order to achieve a higher QS score for International Staff Ratio. Or whether international students were given scholarships so as to increase their numbers to satisfy the QS requirements? Or whether it was reported that a low staff student ratio belies the fact that most tutorial classes pack 28-30 students to one Tutor?    

The use of Citations per Faculty is the most misleading measure of Research quality.

When it comes to the QUANTITY of Research, few can match the outputs of NTU Professors! The tremendous amount of publications by NTU professors (2,500+) every year must be validated by more concrete evidence of their impact on businesses and society. The QS Assessors of course did not care for ample proof, or the lack thereof, that the supposedly “new” knowledge contained in these journal and conference papers is in fact of some or significance public value.

By the way, does this measure include the many “Eminent” Professors, which include some past Nobel Prize winners, who are embedded as NTU Academic Staff during the period of evaluation? Including their understandably high citations could also skew this measure in NTU’s favour.

Clearly, should academic staff in any public-funded university like NTU be spending 70% to 80% of their time, paid for by public funds, to produce journal papers simply for improvements to their resume in preparation for their next job in another University?

A research study found that “specious and trivial” research resulted when people work with no goal other than that of attracting a better job, or getting tenure or higher rank. Such “specious and trivial” research makes little contributions to knowledge. As a result, in the past few decades, the need to secure a job in academia has certainly accounted for a fair amount of the useless material that's been published.

Overall, the QS Ranking Methodology is in essence an OPINION Survey and NOT an Objective Expert Evaluation of Universities. It is a “peer review” limited to a mere opinion poll among “advanced” researchers, whose identities are unknown. It does not involve experts visit to the Universities being ranked, which is a critical condition for any Quality Assurance Reviews.

QS Ranking is akin to nothing more than a Market Consumers Survey, much like how marketing agencies rank the Apple iPhone with other handphones by Blackberry, Nokia, ZTE, Samsung, Sony, Motorola, Lenovo and HTC. 

For the QS Exercise, unidentified Peers (Professors) or Employers choose the “Top” Universities from preselected lists, from which many universities and whole countries have been left out.

QS Rankings favour only Journal Papers in the English language were included, thereby ignoring many great Journals in the other European, Chinese, Indian, and Spanish languages. In so doing, it excludes many top Journals especially in the arts, humanities and social sciences which have important social, intellectual and cultural roles related to their local, regional and national societies.

Half of the overall score comes from two surveys, i.e. peer and employer reviews, both of which have low response rates; other data, with the exception of the number of citations, are provided by the universities themselves.

LEARNING and LEARNING IMPACT is EXCLUDED as a Measure. The only proxy used for teaching quality is the Staff/Student Ratio.  Never mind that NTU classes’ staff/students ratio is usually about 1 to 28 or 30 students.

The final QS rankings use an adapted methodology, which was never validated, to draw on the academic and employer surveys, as well as citations data. Regional weights are used to achieve balance between world regions. Data normalisation and Z-scores are used to calculate the final score.

A University’s contributions to society should be its sufficient measure.  The impact of education on people and community cannot be measured merely by the degrees of dubious excellence.  It can only be measured in terms of their contribution to the happiness and well-being of stakeholders and of the communities to which these persons belong.

What’s the IMPACT of NTU on Singapore?
Learn from Massachusetts Institute of Technology (MIT).
Commissioned in 1997 to conduct an impact study, BankBoston reported that the graduates of the MIT have, since its founding in 1862, created 4,000 firms which, in 1994 alone, employed at least 1.1 million people and generated $232 billion of world sales.  The combined revenues produced by “MIT-related” companies would make them, when taken together, the 24th largest economy in the world.

The MIT Report is an example illustrating the contribution of Universities to their respective dynamic economy.  The development of business enterprises is one concrete measure of the impact of the University and its professors through their students.

Academia, as a noble profession that prides itself for its appreciation of complexity and subtlety, and questions the face value of superficial fa├žade, should not embrace so easily and wholeheartedly such a simplistic technique as using a single dimension such as QS or other Rankings for the evaluation of its own performance and impact.   No University of excellence or professor worthy of the title will ever accept a single quantitative measure as an indicator, let alone a measure, of excellence.  

Whither NTU’s Impact on Singapore?  NTU President and University Management, as well as the Ministry of Education, should be more concerned about the need to increase NTU’s, and other universities’, contributions to society, instead of obsessing with the ranking game.  

Read More:



  1. Good on you Michael. I am afraid the obsession with rankings detracts from the true role of universities as sites of contemplation and reflection on the problems and concerns of the world of humaan affairs

    1. Thanks for yr comment, Robert.
      Catchup when in Singapore please.

  2. Will do Mike

    Good to see there are still some sensible people who are not overly enamoured with distorted representations of quality; its like eating the menu instead of the dish!

  3. A Professor once refers to the QS Rankings as a "Fraud on the Public":

    Somehow, I cannot imagine our University bosses, themselves Professors, and the Ministry of Education, did not know the hollowness of the QS Exercise. Are we so desperate to be No: 1 even to the extent of embracing a highly questionable Standard? Really sad indeed.

  4. We help students study Abroad in Uk , ireland , australia, singapore . We are a team of experienced study abroad consultants who can guide you through all your questions regarding education overseas in a foreign land .

  5. The objects and various other concerns have developed more of the better grounds for the students to start herewith. engineering personal statements

  6. These are the more possible values and provisions and favorably by the time would provide professionals with all such services.

  7. There is possibly still a lot needed to be done in this regard and hopefully for the future this would further proved to be much better for the individuals to proceed herewith. Check electronic gadgets for best electronic gadgets

  8. The idea behind writing a statement of purpose is to persuade the admissions committee to accept your application for admission. personal statement for dental school