The Comprehensive Evaluation Indicator System for Academic Journals of Chinese Humanities and Social Sciences, which was recently released by the Chinese Evaluation Center for Humanities and Social Sciences (CECHSS) under the Chinese Academy of Social Sciences, represents a new breakthrough in adopting a more scientific approach to the evaluation of academic journals.

Based on the conceptual mode of thinking, the system is extraordinarily scientific in the following four ways.

 

More structured
Consisting of three primary indicators, namely “Attraction Power,” “Management Power,” and “Impact Power”, the indicator system divides each into varying grades of sub-indicators based on the actual situation, assessing academic journals in a structured manner. While some have five grades of indicators, some may only three or four.

“Journal supported by National Social Sciences Fund,” for instance, is a fourth-grade indicator subordinate to “National-level Award” (third-grade), but it has no sub-indicators below.

 

Comprehensive categorization
The Evaluation Report on Academic Journals of Chinese Humanities and Social Sciences published by CECHSS offers concise and clear-cut definitions of the three primary indicators.

“Attraction Power” refers to the external environment of the evaluation object. A good external environment can draw more resources and enhance the attractiveness of the evaluation object.

“Management Power” points to the ability of the evaluation object manager in managing and evaluating objects and promoting their development.
 

“Impact Power” directly reflects the strength of the evaluation object, ultimately embodying “Attraction Power” and “Management Power.”
 

Such explicit definitions ensure reasonable categorization of each primary indicator.

“Attraction Power,” for example, is assessed through such secondary indicators as “Academic Reputation,” “Inclusion,” “Author’s Profile” and “Status of Paper.”
 

At first glance, it is difficult to judge whether the division is reasonable or not, but by looking at indicators at the tertiary level and below, it can be seen that the four secondary indicators are parallel yet mutually exclusive.
 

Take “Academic Reputation” as an example. If the tertiary indicators under it are ignored, we can easily mistake it as overlapping the other three secondary indicators. However, the mistake can immediately be avoided if we take a look at how the indicator is further divided.

Categorized into “National-level Awards,” “Provincial- and Ministerial-level Awards,” “Awards to Papers in the Journal” and “Peer Review,” “Academic Reputation” is defined as an external evaluation that is separate from the other three indicators.

“Management Power” includes “Orientation Management,” “Management of Editorial Staff,” “Process Management,” and “Information-oriented Management” in full conformity to rules of conceptual division.
 

“Impact Power” is broken down into “Academic Impact,” “Policy Impact,” “Social Impact” and “International Impact,” which are all encompassing for the concept.

 

Innovative contribution
Drawing upon the conceptual way of thinking, scattered phenomena are integrated logically through structural levels. The indicator system exemplifies this mode of thought by containing all possible elements of all categories, which is an innovative improvement on previous evaluation standards.
 

For example, “Peer Review” includes “Review by Academic Experts,” “Review by Journal Editors,” “Readership Review” as sub-indicators, thus innovatively extending the range of ordinary “Peer Review.”

 

Room for improvement
Since conceptual thinking is historical, it must be tailored to changing needs. For example, is it adequate to incorporate “Internationalization of Editorial Board,” “Internationalization of Authors,” “International Citation Index,” and “Inclusion in International Retrieval Systems” into “International Impact”?

To answer this question, it is necessary to test and modify it in accordance with realities. Whether “Review by Foreign Readers” should be considered is worthy of examination. On this ground, with the expansion and deepening of international exchanges, indicators that represent “International Impact” will be expanded and deepened accordingly.
 

Theoretically, concepts should reflect objects. As objects are constantly developing, conceptual thinking is necessarily open.
 

To conclude, all evaluation indicator systems require adjustment to original structures—out with the old and in with the new—and the system created by CECHSS is no exception. It will develop in practice as well to be increasingly practical, scientific and complete.