HEC Ranking Fiasco Genesis

University Rankings in Pakistan

Dr. Syed Irfan Hyder, January 5, 2004

This is with reference to the letter to the editor of Dawn,
December 31, 2003 where the writer has highlighted the risks of jumping
prematurely in the area of university rankings.
Development of a ranking system is a good idea, but coming
up with a criteria that is complete, just, fair, valid, reliable and
professional is a non-trivial undertaking as explained below. Hurriedly put
together experiences of a few academicians in to a ranking system cannot be
expected to be fair, valid or even professional.

It is surprising that with all the emphasis on R&D,
there is a reluctance in investing in the research and subsequent development
of the proposed ranking system. The ranking initiative is therefore headed in
the same direction as the Model University Ordinance and Tenure Track
initiatives. Recently a questionnaire was sent out by HEC that lacks in
completeness as well as sufficiency. It is even without clearly laid out
objectives and has typographical and structural mistakes. It is surprising to
note the lack of thoroughness even in the booklet on Criteria for Establishing
New Institutes and Universities that has been published and widely circulated.
Institutions are threatened to be de-recognized if they do not conform to the
booklet Criteria in five years!
A proper ranking system for an institution of higher
learning should have the following characteristics without which it would not
be implement able:
  • Just:
    It should be applicable to all the institutions of higher learning
    including public and private universities. It should be general enough to
    encompass variations in engineering, medical, general sciences, business
    and other professional disciplines. It should incorporate and accommodate
    all finer differences such as those between the doctorate in medicine and
    doctorate in philosophy. It should be able to cater to mixing and matching
    of different disciplines in some universities as well as directed
    specializations preferred by some others. 
    It also must rationalize on the basis of the economic cost paid per
    student rather than the fee charged per student.
  • Complete:
    It should measure all the factors that contribute to the quality of the
    institute. It appears from the recent HEC questionnaire and the HEC
    criteria for setting up of new universities and institutes, that the
    questionnaire is biased towards brick and mortar evaluations. Evaluation
    of the following factors are conspicuous by their absence and must be
    included and given due weights: Financial discipline, accounting system,
    costing system, purchasing systems, grading systems, academic monitoring
    systems, exam and classes scheduling systems, syllabus management systems,
    fee management systems, faculty management systems, attendance monitoring
    systems, curriculum improvement systems, student records and student
    progress evaluation systems, examination systems, students complaints
    monitoring systems, industry interaction systems, etc.
  • Valid:
    Ranking system should be valid. That is, it should actually measure what
    it sets out to measure. Does it incorporate all the aspects of higher
    education? Does it measure the objectives of higher education for
    Pakistan? What does our economy require from the institutes of higher
    education. What is the economic demand for researchers, scientists,
    consultants, managers, office workers, professionals, academicians,
    responsible citizens from institutes of higher learning. What should be
    their proportion coming out from a given discipline. How often and how
    soon does the graduates switch from their area of specialization to an
    area of economic opportunity and what are the skills that they take into
    the new fields that they choose.
  • Criteria
    Testing and Pilot
    : Was a pilot done to identify reasonableness of the
    criteria. Was the criteria applied to a couple of universities to test its
    applicability, practicality and ability to yield the desired results. What
    shortcomings were noticed in the pilot application and how were they
    rectified. Unfortunately in Pakistan, government organizations tend to
    launch grand survey projects without the necessary pilot studies. One of
    the prime examples of failed surveys has been the business survey launched
    in 1998-99 for CBR and NADRA. We are still reeling from the lack of
    homework and pilot study done for the NIC project. More pertinent example
    is the Model University Ordinance that was applied without a pilot study.
  • Practical:
    Criteria should be realistically measurable. Is the asked for information
    readily available. Is it verifiable. In how many ways can it be
    interpreted. How much effort does it take to fill out the questionnaire.
    Would all institutes put a similar amount of effort. What would be the
    source of their data. Is the source of data reliable, authentic and
  • Reliable:
    Does multiple application of the criteria by different teams and
    different personnel and at different times on the same institute  yield the same result.
  • Professional:
    Has the questionnaire been prepared using established research
    methodologies. Selection of the survey technique, selection of the
    questions, wording and format of the questions, selection of the
    respondents, estimation of respondents’ effort, estimation of surveyors’
    effort, compilation effort, subsequent analysis effort and other related aspects
    need to be evaluated and addressed by professionals in business and
    analytical research.
A brief analysis of the issues raised above indicate, that
HEC needs to do a major research in developing the criteria before it can be
ready to actually start ranking the institutions. The development of the
criteria would have to be done in an open and transparent manner with
continuous interactions with institutions of higher learning to yield the
desired results. 
Dr. Irfan Hyder


Leave a Reply

Your email address will not be published. Required fields are marked *