LMS impact consultation

I have just posted the following comment on the LMS consultation blog on impact and the REF. At risk of boring my readers who have heard all this already, I am posting it here as well.

I will say briefly why I think that HEFCE’s rules for impact in the REF are on balance bad for mathematics. First, mathematics does have impact; second, this is not captured by HEFCE’s mechanism; and third, this will have an effect.

As preamble I should make clear that, since our research is funded by the taxpayer, it is entirely reasonable for the funders to have some influence on what we do. [I happen to believe that putting pressure on us to do “impactful” mathematics will actually harm the UK’s competitiveness in the long term; funders should take note of arguments for this. But that is another matter.]

First, mathematics does have impact. Everyone has their favourite examples of this. Mine are application of complex numbers to electrical circuit theory and hence to power transmission over long distances; application of number theory to public-key cryptography and hence to Internet commerce; and third, application of linear algebra to the Google page-rank algorithm, without which finding information on the Web would be much more difficult.

I think it is very important that we have good presentations of case studies on the impact of mathematics readily available for use in debates with funders and politicians. Several people, including Chris Rodger, Colva Roney-Dougal, and Peter Rowlett, have told me that they have such material. But it would be good to have a page which would come high on any Google search for “impact of mathematics”, with detailed descriptions of some and pointers to others. Chris Rodger maintains the CADCOM collection (featuring applications of discrete mathematics) at http://www.dms.auburn.edu/~rodgec1/cadcom/

My three examples make very clear that the impact of mathematics is a long-term process. HEFCE rules do not allow this. The allowed timeframe of 15 years is absurdly small; the definition of impact (economic and social, not even allowing academic impact as RCUK do) is far too narrow; both the research having the impact and the commercialisation of it must occur in the same institution. It seems clear that very little mathematics will have any impact at all on these rules.

Does this matter?

I have heard two arguments made. First, if mathematics units of assessment score essentially zero for impact, this won’t matter since everybody will still be equal. This is wrong for two reasons. First, it will depress the overall grades and give the impression that UK mathematics is in poor shape. Second, HEFCE assured RAE panels in 1996 that their decisions would have no effect on the total pot of money in their subject; when the funding formula was revealed, this was simply not true. So we cannot trust them.

Another argument is that, since you cannot take impact with you (a department can claim impact for a researcher’s work, if the other conditions are met, even if that researcher is now somewhere else), this will damp down the “transfer market”. This is a clear case of the tail wagging the dog. Departments need to make appointments, since people die, retire or leave; this is simply another way of penalising us.

Finally, if we think it is bad now, worse is to come. HEFCE explain that, as a result of comments from the community [including a petition signed by nearly 30000 academics, though they fail to mention this], they reduced the contribution of impact to the assessment from 25% to 20%, but they “intend to increase [this proportion] in subsequent exercises”.

About Peter Cameron

I count all the things that need to be counted.
This entry was posted in maybe politics and tagged , , . Bookmark the permalink.

7 Responses to LMS impact consultation

  1. Doormat says:

    The following is really cynical, but that’s the mood I’m in right now. Let’s play Devil’s advocate. Suppose you really believe that university research should be contributing to the economic and social well-being of this country. Then why would you fund basic maths research, especially when other countries will? If it’s going to take years to have impact, then you can just buy the journals (or, more likely, textbooks) which have already been produced (or, in the future, will be produced by other countries) and use that knowledge to develop something applicable.

    Of the examples you give– all that research can be found in many, many textbooks for very little money. Much cheaper than employing an expensive mathematician! About the only argument I can make is that someone had to teach this to, say, Brin and Page so they could invent the Google Page-Rank algorithm. Unfortunately, that’s an argument for having maths teachers, not researchers (or, at least, not for paying people on the basis of their research).

    I do genuinely come around to the view expounded by comedian David Mitchell: http://www.guardian.co.uk/commentisfree/2009/sep/27/david-mitchell-pointless-studies-survey If an idea has obvious economic impact, then business should be paying for it (and, frankly, business _will_ pay for it, because it will pay them back). The whole point of government funding is to pay for research which wouldn’t be funded otherwise. I do think there’s a strong argument for doing academic research for the hell of it– it’s a mark of civilisation that we allow our curiosity to lead us.

    Yep, in the short term, we have to play by the HEFCE rules. But I do think that, some of the time, we do need to continue to suggest that the very idea the rules are based upon is wrong.

  2. Yiftach says:

    First, like Doormat, I believe that governments should pay for basic science. If money can be made directly out of research, then obviously some business will be happy to fund it. Why should the government compete with the private market? It is well accepted that governments are inefficient. Notice that this is true not just for mathematics! (I took this argument to the extreme, but the REF’s measure and use of impact is also extreme.)

    Second, with the way that impact is measured by the REF, it seems to me that there is no reasonable way to predict in advance who and what research will have impact (at least in mathematics). Moreover, it is not transferable with the researcher. So how are universities supposed to change their behavior to achieve more impact? Moreover, the time scale is so large that most people that make the decisions will not see the result of their decisions before they retire, so why bother? In addition, who can know how research is going to be evaluated in 10-15 years from now?

    Third, I think one of the main points that are missed about impact of mathematics, is that much of it is indirect. Namely, people who finish their undergraduate or graduate degree in mathematics move to another area of research or technology but carry mathematical values, approached and knowledge into the new area. This is clearly not measured. What values will be carried by people that are educated by mathematicians that are focused on funding research rather than doing research?

    Fourth, I think also that unfortunately the REF’s impact is going to have impact. It is going to add lots of noise to the system. The impact measured is focused on very few individuals and is so arbitrary that it is likely that its variance is going to be quite broad and as I said quite arbitrary (a department might be lucky to have someone with high impact or might not be lucky). On the other hand, the variance of papers and research environment is not going to be so large. Thus, we will suddenly have two similar departments that will be ranked very differently due to the impact.

    My final point is that we can still do something about it. I believe that if the LMS and other professional organizations in other science areas would have recommended to their members not to take part in REF committees unless the impact is canceled, then this would have changed things. Politicians wouldn’t like to be seen as fighting with the experts. I honestly doubt if they really care about the impact. They just want to be seen doing the “right” thing.

  3. A small footnote to Yiftach’s fourth point. There is some evidence already from the impact dry runs that a department which has someone with journalistic skill who can put the right spin on a piece of research is likely to do better than one that hasn’t. This doesn’t seem like a good way to encourage research.

  4. From the first editorial in this week’s Nature:

    … it typically takes any technology some 20 years to emerge from the lab and be commercialised — and even then it can succeed only with sustained effort from researchers, industry and funding agencies

    How much longer, then, from mathematician’s notebook to commercialisation?

  5. The Isaac Newton Institute have this to say about impact.

    The Research Excellence Framework recognises the uncertainties of translation difficulties in measuring the impact of research on economic and social well-being, and is permitting a window of 15 years for reporting. Mathematics is particularly difficult in this respect. It is not possible in law to patent mathematics and many mathematical contributions go simply unacknowledged. Much work is curiosity driven. For example, it took many decades for the importance of discoveries in early number theory to be recognised as absolutely essential to modern cryptography, granting security in financial and other communications. In many cases, in this process the language of the original mathematics can be lost in its appropriation through other academic disciplines. Equally, mathematics has the potential for rapid transferral; this is illustrated by the take-up of the Itô calculus for stochastic systems. Equally, the deep long-term impact of incremental changes should not be underestimated.

  6. Peter Rowlett’s collection of impact case studies is in the current (14 July 2011) issue of Nature: volume 475, pp. 166–169, entitled “The unplanned impact of mathematics”.

  7. Impact is an inexhaustible topic. Here is a comment by Tim Harford, The Undercover Economist, on his blog: http://timharford.com/2011/09/new-ways-with-old-numbers/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.