Benefits and limitations
Bibliometrics is advantageous but also underlies several limitations. On the one hand, it is very important to apply the methods correctly but also make use of its outcomes in a reasonable manner. Below briefly methods and benefits but als limitations of bibliometrics are outlined are as wells as suitable in-depth reading is recommended.
Table of contents
For this purpose it is useful to make a rough classification according to desired insights. Evaluative bibliometrics is usually of higher importance as explorative bibliometrics that is more and more important for strategic decisions of the university's development and management.
Evaluative bibliometrics
Research interests
Evaluative bibliometrics means (standardised) evaluation of research output at different levels of abstraction (from countries, universities and sub-units, research groups up to individual researchers).
Objects of investigation
All types of (measurable) results of scientific output (especially publications) and their resonance (especially citations), which are used to draw conclusions about the influence of scientific work. The aim is to be able to make a statement about the quantity but also the quality of the research results.
Methods / Tools
- (summable) indicators at different levels: individual, publication and journal see Bibliometric Indicators
- Score Cards and Rankings
Benefits
- outstanding publications and citations can be one indicator of scientific excellence
- standardised procedures and thus intersubjective comparability
- less time- and ressources-consuming than peer review processes
Limitations
- field-specific publication cultures, practices and strategies as well as the characteristics of the individual indicators may encourage misinterpretations.
- Presence of false positive and false negative results (outstanding bibliometric values without scientific excellence and scientific excellence without outstanding bibliometric values)
- one-sided focus on maximising bibliometric indicators leads to undesirable side effects, e.g. "salami tactics" in publication behaviour as well as "citation cartels" and increases the "publish or perish" dilemma
Conclusion
- In view of the steadily increasing number of publications and growing specialisation, evaluative bibliometrics is a useful addition to qualitative assessment (peer review).
- In order to be able to interprete indicators resonably, specific knowledge of methods and the research dicipline is required (Wissenschaftsrat, 2011, p. 42).
- Field related definition of quality standards using a broad concept of performance to find and select suitable indicators.
Recommended in-depth readings
- Recommendations for evaluating and managing research performance
- Wilsdon et al. (2015): The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management
- Bornmann and Marx (2015): Bibliometrics in research evaluation - Background, significance, and limitations
- Smaldino and McElreath (2016): The natural selection of bad science
Explorative Bibliometrics
Research interests
The explorative bibliometrics focus on the identification of relevant research topics, central actors and trends within individual disciplines. In addition, subject-specific patterns of cooperation and communication structures may be identified.
Objects of investigation
- topic cluster
- research edge of scientific knowledge
- Interdisciplinarity
- Internationality
Methods / Tools
- Science Mapping
- Social network Analysis
Benefits
Gain knowledge of who cites which work in which context, in order
- to find future cooperation partners and/or
- to receive suggestions and perspectives for their own research.
Limitations
- Distinction of research fields.
- Statements about research contents is given only based on keyword level.
Conclusion
- Interesting method, which can provide insights into cooperation within and between researchers, disciplines and research institutions.
- Enables insights into emerging and upcoming new scientific topics/disciplines.