Open Universiteit

Please use this identifier to cite or link to this item:
Title: Exploring a prototype framework of web-based and peer-reviewed “European Educational Research Quality Indicators” (EERQI)
Authors: Mooij, Ton
Keywords: Quality indicators
Impact score
Prototype framework
Web-based versus peer-reviewed
Multimethod assessment
Educational research
Reviewer's bias
Structural modeling
Issue Date: 16-Sep-2014
Citation: Mooij, T. (2015). Exploring a prototype framework of web-based and peer-reviewed “European Educational Research Quality Indicators” (EERQI). Scientometrics, 102 (1), 1037-1055. DOI: 10.1007/s11192-014-1429-x
Abstract: Digitization, the Internet, and information or webometric interdisciplinary approaches are affecting the fields of Scientometrics and Library and Information Science (LIS). These new approaches can be used to improve citation-only procedures to estimate the quality and impact of research. A European pilot to explore this potential was called “European Educational Research Quality Indicators” (EERQI, FP7 # 217549). An interdisciplinary consortium was involved from 2008-2011. Different types of indicators were developed to score 171 educational research documents. Extrinsic bibliometric and citation indicators were collected from the Internet for each document; intrinsic indicators reflecting content-based quality were developed and relevant data gathered by peer review. Exploratory and confirmatory factor analysis and structural modeling were used to explore statistical relationships among latent factors or concepts and their indicators. Three intrinsic and two extrinsic latent factors were found to be relevant. Moreover, the more a document was related to a reviewer’s own area of research, the higher the score the reviewer gave concerning 1) significance, originality, and consistency, and 2) methodological adequacy. The conclusions are that a prototype EERQI framework has been constructed: intrinsic quality indicators add specific information to extrinsic quality or impact indicators, and vice versa. Also, a problem of “objective” impact scores is that they are based on “subjective” or biased peer-review scores. Peer-review, which is foundational to having a work cited, seems biased and this bias should be controlled or improved by more refined estimates of quality and impact of research. Some suggestions are given and limitations of the pilot are discussed. As the EERQI development approach, instruments, and tools are new, they should be developed further.
Appears in Collections:1. FEEEL Publications, books and conference papers

Files in This Item:
File Description SizeFormat 
Manuscript EERQI Scientometrics with abstr acknowl ref tables figures.pdf213.79 kBAdobe PDFView/Open

This item is licensed under a Creative Commons License Creative Commons