![]() ![]() To the best of our knowledge, this is the first time that anchoring effects are being investigated in the assessment of scientific work. In this preregistration, we present the proposal for a study that is intended to investigate various anchoring effects in the assessment of scientific work. Thus, in the example above, it is not only citation counts that could act as an anchor, but also numerical identifiers for the candidates (or any other arbitrary number) presented with the summaries. Values can even serve as anchors that are not directly related to the assessed objects. ![]() Research on anchoring effects has shown for various assessment contexts that presented values can have a substantial effect on judgments. Tversky and Kahneman introduced the anchoring-and-adjustment heuristic: “the heuristic maintains that anchoring bias is caused by insufficient adjustment because final judgements are assimilated toward the starting point of a judge’s deliberations … the anchoring effect is the disproportionate influence on decision makers to make judgments that are biased toward an initially presented value”. If the reviewer recommends this candidate for the professorship position, the reason for this decision may be the so-called ‘anchoring effect’. In thinking about the candidates, one of the reviewers is impressed by the very high citation counts of one candidate (someone from the field of financial economics) and observes that the other candidates are not on the same level. The citation counts are very different, since the candidates are from different fields within economy. When it comes to the decision regarding the candidates, the summaries for all candidates are provided to the reviewers with the total citation counts. Their previous research papers, funds received and teaching experience have been discussed in great detail. It is the last round of a selection process and the candidates are very similar in terms of their performance. ![]() Let us suppose a situation in which several candidates for a professorship position are compared. The use of metrics in situations with great judgment uncertainties may reach its limits, however. The reason for their use is understandable: since no researcher is an expert in all fields within a discipline, it appears reasonable to resort to metrics in cases of fields that are remote from those that are familiar. It is no surprise to read in the paper that bibliometric indicators (especially the popular h index) play a prominent role in the assessments of the candidates. In a recent study analyzing peer review reports, Hammarfelt, Rushforth identified criteria and strategies that reviewers used to decide on candidates for professorships in Sweden. The results of the study may have important implications for quality assessments of papers by researchers and the role of numbers, citations, and journal metrics in assessment processes. ![]() Thus, we are interested in whether possible adjustments in the assessments can not only be produced by quality-related information (citation or journal), but also by numbers that are not related to quality, i.e. In the statistical analyses, we estimate how (strongly) the quality assessments of the cited papers are adjusted by the respondents to the anchor value (citation, journal, or access code). The control group will not receive any further numerical information. Some authors will be assigned to three treatment groups that receive further information alongside the cited paper: citation information, information on the publishing journal (journal impact factor), or a numerical access code to enter the survey. The authors are asked to assess the quality of papers that they cited in previous papers. We shall undertake a survey of corresponding authors with an available email address in the Web of Science database. The design of our study is oriented towards the study by Teplitskiy, Duede. We are interested in the question whether citation decisions are (mainly) driven by the quality of cited references. In our planned study, we shall empirically study the assessment of cited papers within the framework of the anchoring-and-adjustment heuristic. ![]()
0 Comments
Leave a Reply. |