This is the month that universities in the UK make their submissions to the Research Excellence Framework (REF) 2014. The REF is a big deal, as I’ve written about before. It is also continuing to attract plenty of commentary, much of it critical. For some time Dorothy Bishop, Professor of Developmental Neuropsychology at Oxford University, has been using her personal blog to critique exercises in research rating. Her objections include their poor cost-effectiveness and the dangers of using journal impact factors as a proxy for the quality of individual papers. In his blog Peter Coles, Professor of Theoretical Astrophysics at Sussex University, has attacked the REF for becoming self-serving. Quoting from a Times Higher Education (THE) story he also writes of the practice in some universities of research-active academics not selected for REF return being shifted onto teaching-only contracts. This week, Professor Peter Scott from the Institute of Education writes in The Guardian that research assessment is now ‘out of control’ whilst the THE has recently reported on the case of Lancaster University historian Professor Derek Sayer who has appealed against the decision to include him in the REF on the grounds that the procedures used to exclude some of his colleagues have been discriminatory.
And so it goes on. In the REF proper, outputs (typically articles in journals) will be graded by experts as ‘world leading’ (4*), ‘internationally excellent’ (3*), ‘internationally recognised’ (2*), ‘nationally recognised’ (1*) or as either ‘sub-national’ or ‘not research’. These gradings will be made using the criteria of originality, significance and rigour. Universities get to select which of their staff will be included in their returns, drawing on their preparatory assessments of the quality of eligible outputs and underpinned by strategic ambitions of where they want to be in the HE firmament once the official REF results are published and institutions ranked. My guess is that, for most researchers and their employers, the most important distinction needing to be made will have been between outputs which are internationally excellent (3*) and outputs which are ‘only’ internationally recognised (2*). For reasons of reputation and likely future funding an article assessed as being at least the former is much more likely to be included in a REF submission than one which is not.
Quality assessments informing imminent REF returns will have been made by busy people with varying degrees of expertise in the (sub)areas in which the papers they have been reading lie. I’m going to speculate that there will be many hundreds (thousands?) of academics with outputs which will have attracted inconsistent scores from internal and external reviewers. Who knows, perhaps there are even some with individual outputs assessed by different people as simultaneously being ‘world leading’ and ‘unclassified’. Many will certainly have papers differentially judged as being 2* or 3*, leaving all sorts of tricky decisions to be made on submission or non-submission with all manner of possible consequences for both individuals and universities.
Back in the world of actually doing research, as opposed to the world of assessing research outputs and fretting over returns to assessment exercises, I am pleased to say COCAPP is now receiving questionnaires from service users and RISC is deep into phase 2. If you head over to this NISCHR page you’ll also find news of the Plan4Recovery project, led by Michael Coffey. This is a collaboration involving Hafal, and I’m very pleased to be a co-applicant along with Sherrill Evans and Alan Meudell. Plan4Recovery is advertising for a research officer, and is about to have its first advisory group meetings. Exciting times.
Ben Hannigan's blog 
Hi Ben, excellent blog and some really pertinent points made. My perspective, as ‘deputy’ UoA3 leader in my University, is that there appears to have been a lot of ‘scare mongering’ in the run up to this exercise. For example, the rumours largely consisted of tales such as; this exercise will be unlike any other and it will be much, much tougher this time round.
I suspect the dye was cast from that moment on and I wonder what impact these rumours may have had on any subsequent critical friend reviews across the UK. It would appear that, for many schools of nursing, the problem this time round has simply been getting out of the gates. Due to a combination of critical friend grades and University thresholds, I imagine that some nurses (many of whom were probably part of previous RAE submissions) were not returned this time round, and probably some schools have made no submission at all?
I’m sure we aren’t the only discipline in this position, but nursing does seem to have a track record in this area. My other concern, as pointed out in the above links, is that the REF process takes no account of those ‘ahead of the curve’ – ie, the researchers who are doing genuinly innovative work, but perhaps are viewed as non-tradional, and therefore risky. As I have heard said several times, REF is about the research of today, not tomorrow!
quite what the outcome of this exercise will be, remains to be seen but I’d be very keen for a post-2014 nursing and AHP debrief, just to estabslish if there are any lessons to be learnt for the future
Well, we’ll know soon enough which universities returned who as this is information which will be in the public domain sometime in 2015, if memory serves. Over on the REF website [http://www.ref.ac.uk/subguide/] I see that summary data has already been posted on the numbers of staff returned, and how this compares to numbers returned in the last RAE. A drop, but not by much.