Wednesday, September 23, 2009

Research Assessment: Publish (in high impact journals!) or Perish

Have you ever wondered if the lead author in a collaborative paper is the brains behind it, or just lucky enough have his name first alphabetically? Or do you think that some authors have high citation counts simply because so many papers refute their work? Maybe there are even groups of researchers habitually citing each other to boost each other's rankings?

All of these questions are explored in "Communicating knowledge: How and why UK researchers publish and disseminate their findings" by the Research Information Network.

UK funding bodies use the RAE (Research Assessment Exercise) as a metric to allocate research grants. The RAE focusses on number of works published and their number of times cited. Many researchers feel that this system unfair, and rewards those who "play the game" best, rather than those producing the best, most innovative work. Many of the researchers quoted in Communicating Knowledge feel that the RAE is having an unhealthy influence on how they work, write, cite and publish.

Some researchers' comments from the report:
  • “With my collaborators and colleagues I have been organising so as to facilitate mutual citation.”
  • “Head of Research in my institute actively discouraged book chapters and reviews because they were not seen as prestigious for RAE.”
  • “Sadly, I find myself increasingly moving away from publishing in journals which are important and read by a lot of colleagues, to publishing in high status journals instead. This had led to much longer delays [and] thus adversely affects science, but I feel the pressure to do this in order to advance career wise.”
Would funding decisions in Ireland be fairer under an "objective" assessment like the RAE? Or is it meaningless and even damaging to try to compare research in this way?

1 comment:

  1. This is a valuable post with reference to an interesting report. I am not certain that the UK RAE process is correctly characterised however. It is based on peer review rather than bibliometric analysis. In disciplines where bibliometric analysis is normal and credible I would imagine that peer reviewers are likely to call for the data and to use it in support of their judgements. In my own discipline, law, the sub-panel made it clear that there was not role of bibliometric analysis and that judgements were made through reading of the maximum four submitted publications from each researcher and evaluating them in a manner that was independent of the outlet in which they were published. It has recently been reported that the successor to the RAE, the Research Excellence Framework, will retain substnantial elements of peer review with less of a shift to bibliometric analysis than had originally been suggested. I would add that I cannot see that the RAE encourages researchers to produce excessive numbers of publications by the standards of most disciplines - the pressure is to produce four internationally excellent publications within the time period of seven or so years.

    ReplyDelete