University Rankings in Pakistan
Dr. Syed Irfan Hyder, January 5, 2004
This is with reference to the letter to the editor of Dawn,
December 31, 2003 where the writer has highlighted the risks of jumping
prematurely in the area of university rankings.
December 31, 2003 where the writer has highlighted the risks of jumping
prematurely in the area of university rankings.
Development of a ranking system is a good idea, but coming
up with a criteria that is complete, just, fair, valid, reliable and
professional is a non-trivial undertaking as explained below. Hurriedly put
together experiences of a few academicians in to a ranking system cannot be
expected to be fair, valid or even professional.
up with a criteria that is complete, just, fair, valid, reliable and
professional is a non-trivial undertaking as explained below. Hurriedly put
together experiences of a few academicians in to a ranking system cannot be
expected to be fair, valid or even professional.
It is surprising that with all the emphasis on R&D,
there is a reluctance in investing in the research and subsequent development
of the proposed ranking system. The ranking initiative is therefore headed in
the same direction as the Model University Ordinance and Tenure Track
initiatives. Recently a questionnaire was sent out by HEC that lacks in
completeness as well as sufficiency. It is even without clearly laid out
objectives and has typographical and structural mistakes. It is surprising to
note the lack of thoroughness even in the booklet on Criteria for Establishing
New Institutes and Universities that has been published and widely circulated.
Institutions are threatened to be de-recognized if they do not conform to the
booklet Criteria in five years!
there is a reluctance in investing in the research and subsequent development
of the proposed ranking system. The ranking initiative is therefore headed in
the same direction as the Model University Ordinance and Tenure Track
initiatives. Recently a questionnaire was sent out by HEC that lacks in
completeness as well as sufficiency. It is even without clearly laid out
objectives and has typographical and structural mistakes. It is surprising to
note the lack of thoroughness even in the booklet on Criteria for Establishing
New Institutes and Universities that has been published and widely circulated.
Institutions are threatened to be de-recognized if they do not conform to the
booklet Criteria in five years!
A proper ranking system for an institution of higher
learning should have the following characteristics without which it would not
be implement able:
learning should have the following characteristics without which it would not
be implement able:
- Just:
It should be applicable to all the institutions of higher learning
including public and private universities. It should be general enough to
encompass variations in engineering, medical, general sciences, business
and other professional disciplines. It should incorporate and accommodate
all finer differences such as those between the doctorate in medicine and
doctorate in philosophy. It should be able to cater to mixing and matching
of different disciplines in some universities as well as directed
specializations preferred by some others.
It also must rationalize on the basis of the economic cost paid per
student rather than the fee charged per student. - Complete:
It should measure all the factors that contribute to the quality of the
institute. It appears from the recent HEC questionnaire and the HEC
criteria for setting up of new universities and institutes, that the
questionnaire is biased towards brick and mortar evaluations. Evaluation
of the following factors are conspicuous by their absence and must be
included and given due weights: Financial discipline, accounting system,
costing system, purchasing systems, grading systems, academic monitoring
systems, exam and classes scheduling systems, syllabus management systems,
fee management systems, faculty management systems, attendance monitoring
systems, curriculum improvement systems, student records and student
progress evaluation systems, examination systems, students complaints
monitoring systems, industry interaction systems, etc. - Valid:
Ranking system should be valid. That is, it should actually measure what
it sets out to measure. Does it incorporate all the aspects of higher
education? Does it measure the objectives of higher education for
Pakistan? What does our economy require from the institutes of higher
education. What is the economic demand for researchers, scientists,
consultants, managers, office workers, professionals, academicians,
responsible citizens from institutes of higher learning. What should be
their proportion coming out from a given discipline. How often and how
soon does the graduates switch from their area of specialization to an
area of economic opportunity and what are the skills that they take into
the new fields that they choose. - Criteria
Testing and Pilot: Was a pilot done to identify reasonableness of the
criteria. Was the criteria applied to a couple of universities to test its
applicability, practicality and ability to yield the desired results. What
shortcomings were noticed in the pilot application and how were they
rectified. Unfortunately in Pakistan, government organizations tend to
launch grand survey projects without the necessary pilot studies. One of
the prime examples of failed surveys has been the business survey launched
in 1998-99 for CBR and NADRA. We are still reeling from the lack of
homework and pilot study done for the NIC project. More pertinent example
is the Model University Ordinance that was applied without a pilot study. - Practical:
Criteria should be realistically measurable. Is the asked for information
readily available. Is it verifiable. In how many ways can it be
interpreted. How much effort does it take to fill out the questionnaire.
Would all institutes put a similar amount of effort. What would be the
source of their data. Is the source of data reliable, authentic and
comparable. - Reliable:
Does multiple application of the criteria by different teams and
different personnel and at different times on the same institute yield the same result. - Professional:
Has the questionnaire been prepared using established research
methodologies. Selection of the survey technique, selection of the
questions, wording and format of the questions, selection of the
respondents, estimation of respondents’ effort, estimation of surveyors’
effort, compilation effort, subsequent analysis effort and other related aspects
need to be evaluated and addressed by professionals in business and
analytical research.
A brief analysis of the issues raised above indicate, that
HEC needs to do a major research in developing the criteria before it can be
ready to actually start ranking the institutions. The development of the
criteria would have to be done in an open and transparent manner with
continuous interactions with institutions of higher learning to yield the
desired results.
HEC needs to do a major research in developing the criteria before it can be
ready to actually start ranking the institutions. The development of the
criteria would have to be done in an open and transparent manner with
continuous interactions with institutions of higher learning to yield the
desired results.
Dr. Irfan Hyder
Leave a Reply