This is the month that universities in the UK make their submissions to the Research Excellence Framework (REF) 2014. The REF is a big deal, as I’ve written about before. It is also continuing to attract plenty of commentary, much of it critical. For some time Dorothy Bishop, Professor of Developmental Neuropsychology at Oxford University, has been using her personal blog to critique exercises in research rating. Her objections include their poor cost-effectiveness and the dangers of using journal impact factors as a proxy for the quality of individual papers. In his blog Peter Coles, Professor of Theoretical Astrophysics at Sussex University, has attacked the REF for becoming self-serving. Quoting from a Times Higher Education (THE) story he also writes of the practice in some universities of research-active academics not selected for REF return being shifted onto teaching-only contracts. This week, Professor Peter Scott from the Institute of Education writes in The Guardian that research assessment is now ‘out of control’ whilst the THE has recently reported on the case of Lancaster University historian Professor Derek Sayer who has appealed against the decision to include him in the REF on the grounds that the procedures used to exclude some of his colleagues have been discriminatory.

And so it goes on. In the REF proper, outputs (typically articles in journals) will be graded by experts as ‘world leading’ (4*), ‘internationally excellent’ (3*), ‘internationally recognised’ (2*), ‘nationally recognised’ (1*) or as either ‘sub-national’ or ‘not research’. These gradings will be made using the criteria of originality, significance and rigour. Universities get to select which of their staff will be included in their returns, drawing on their preparatory assessments of the quality of eligible outputs and underpinned by strategic ambitions of where they want to be in the HE firmament once the official REF results are published and institutions ranked. My guess is that, for most researchers and their employers, the most important distinction needing to be made will have been between outputs which are internationally excellent (3*) and outputs which are ‘only’ internationally recognised (2*). For reasons of reputation and likely future funding an article assessed as being at least the former is much more likely to be included in a REF submission than one which is not.

Quality assessments informing imminent REF returns will have been made by busy people with varying degrees of expertise in the (sub)areas in which the papers they have been reading lie. I’m going to speculate that there will be many hundreds (thousands?) of academics with outputs which will have attracted inconsistent scores from internal and external reviewers. Who knows, perhaps there are even some with individual outputs assessed by different people as simultaneously being ‘world leading’ and ‘unclassified’. Many will certainly have papers differentially judged as being 2* or 3*, leaving all sorts of tricky decisions to be made on submission or non-submission with all manner of possible consequences for both individuals and universities.

Back in the world of actually doing research, as opposed to the world of assessing research outputs and fretting over returns to assessment exercises, I am pleased to say COCAPP is now receiving questionnaires from service users and RISC is deep into phase 2. If you head over to this NISCHR page you’ll also find news of the Plan4Recovery project, led by Michael Coffey. This is a collaboration involving Hafal, and I’m very pleased to be a co-applicant along with Sherrill Evans and Alan Meudell. Plan4Recovery is advertising for a research officer, and is about to have its first advisory group meetings. Exciting times.

Follow @benhannigan