A post that reiterates, to me, deep flaws in how academics (both formally and informally) think about impact, “evidence-based” policy, and change. (See the work of University of Stirling’s Paul Cairney and his “1000 word” summaries of key policy theories for much better takes on how policy change happens.)
Re-posted without alteration under Creative Commons license. Original linked in the title below.
Policymakers frequently fail to use research evidence in their work. Academia moves too slowly for the policy world, and its findings do not translate easily into policy solutions. Using the Department of Health as a case study, Jo Maybin outlines how research most likely has an impact as a result of personal interactions between individual researchers and policymakers. But this can limit the range of knowledge being used to inform policy, and be problematic when individuals change or leave jobs.
Within busy government departments, who is it that policymakers speak to and what happens in those conversations? And why are these interactions such an effective and appealing means of learning for policymakers? I spent 18 months studying civil servants working on high-profile policy documents and legislation in England’s Department of Health, observing them in meetings, reading what they were reading and writing, and interviewing them about their work.
Researchers were indeed one important knowledge source for these policymakers but time pressure, combined with the delicate process of alliance building that constitutes policymaking, meant that authority, proximity and trust were key implicit criteria when it came to selecting which individuals to speak to.
The qualities of the knowledge brought by individual researchers (in comparison to research documents) made it particularly well suited to the civil servants’ interests, because it was seen as up-to-date, candid, synthesised and editorialised. Conversations enabled the civil servants to ‘drill down’ into what they were most interested in, and to discover ‘unknown unknowns’. But one major downside of ‘embodied’ knowledge like this is that policymakers move jobs and leave, which risks severing the relationships through which research knowledge flows. A succession of recent and planned cuts to the Department’s own staff brings this weakness into sharp relief.
Insiders and contacts-of-contacts
When they needed to learn about a new policy topic civil servants drew, to a large extent, on the accumulated knowledge of colleagues within the Department. Interviewees talked in abstract terms about how the best civil servants were those with generalist policy skills, and not specialist subject knowledge. Yet in practice these generalists relied heavily on the insights of individual colleagues who had built up expertise in a particular policy area by dint of staying in a team or on a topic for a longer period of time.
Attempts at capturing this knowledge (and other aspects of ‘organisational memory’) in a formal document-based knowledge management system seemed to have been largely ineffective: the civil servants wanted to talk to the person, and not to read the notes they had logged on the system intended to serve that purpose.
The civil servants did also speak to outsiders, who were most commonly individuals from:
- Professional representative or membership organisations, such as the Royal Colleges and the British Medical Association
- Think-tanks, such as the Nuffield Trust and The King’s Fund
- Patient charities, such as Diabetes UK or Rethink
- GP practices, hospitals and local health commissioning organisations
- Charities involved in providing health services
Having a sense for who are the ‘relevant organisations’ and ‘big players’ in any particular policy area was seen as an important policy skill. The civil servants identified outside individuals through contacts of contacts: the recommendations of colleagues, and in turn of the outside contacts themselves.
Why did the civil servants favour colleagues and contacts-of-contacts? The decision on who to speak to was partly a matter of expediency given the time pressure the civil servants often worked under. Studies of organisational learning describe how it is common for individuals to engage in ‘local’ rather than ‘general scanning’ for information, and communication theories show us how we find it easier to communicate with people who share similar frames of reference.
But this was also a matter of trust. To have a meaningful conversation about an issue, the civil servants had to reveal something about emerging policy thinking on the topic. Policy formulation was a craft of delicate alliance building. Particularly in the early stages of a policy’s development, the civil servants were most comfortable divulging sensitive information about a policy’s possible content with individuals who shared an interest in protecting the Department’s work and its reputation, or at least in maintaining good relations with the Department in an effort to secure future influence.
The limits and potential of personal networks
The problem with this strategy is that the Department may be drawing on only a very small pool of knowledge, views and experiences. The civil servants described with embarrassment how they had sometimes approached particular outside individuals only to find they had already had a number of other recent contacts from others in the Department. It was also telling that the civil servants often struggled when it came to wanting to engage directly with the ‘public’, who were partly defined by their lack of existing connection to the Department.
Yet using the ‘contacts of contacts’ strategy to source knowledge need not necessarily be limiting; think of the ‘six degrees of separation’ theory that underpins the Kevin Bacon Game. Social theories of information diffusion tell us that if this method is pursued for a number of iterations, and with the aim of seeking out acquaintances rather than ‘strong ties’ of each individual (eg. not a close professional or personal contact), then the approach can lead you to varied and innovative sources of knowledge. The risk comes if contact is limited to those with strong links who are in regular contact with one another.
Why people? Why interaction?
Again, expediency is part of the story here. The knowledge that people bring is already synthesised. The individual may have years of experience which, if written down, could fill a library shelf at least. But by meeting with a civil servant, and hearing about what they are interested in knowing, the individual can quickly select the most relevant items of their knowledge to share.
People can also offer their judgement. This was really prized by civil servants who wanted to know, given everything that individual had learnt about this topic, what did they really think? As carriers of knowledge people brought not just ‘facts’, but also opinions, and new ways of seeing and thinking about issues.
People also have (in theory at least) the most up-to-date knowledge on a topic, whereas documents, even electronic ones, may become dated as soon as they are drafted. And people are the holders of certain kinds of practical and risky knowledge that doesn’t get ‘committed to paper’. For example, the civil servants often needed to understand how a particular system, or process, or set of relationships works in practice. There are rarely documents that describe such practices, partly because they are seen as too mundane to record in that way, but also because they are variable, ever-changing, and often run more or less counter to some official policy about how things should be done. The civil servants felt they got more candid accounts from people by meeting them in person.
Talking together with others also allowed for ‘simultaneous translations’, helping civil servants to quickly learn the language of new topic areas by being able to pause or interrupt to ask about particular terms and acronyms. And the distinctive qualities of dialogues allowed the civil servants to discover ‘unknown unknowns’. As one interviewee said: ‘In talking about it you get to the point where you think “aha! That’s what I was after!” You might not have known it yourself when you first sat down’. In the most constructive conversations, the civil servants and their dialogue partners were not simply telling each other what they already knew, but were rather ‘creating something new together’.
The shortfalls: when people leave
One of the problems with this ‘embodied knowledge’ is that people move on. They change jobs or careers, or stop working altogether. The Department has recently announced the latest in a long series of cuts to its staff numbers, and each of those staff will take with them a body of knowledge about particular policy areas and a host of contacts outside of the Department.
It is now even more incumbent on those who are left behind to challenge and support themselves and each other to pursue contact chains beyond the usual suspects, through pursuing a series of ‘weak ties’. This will maximise the chances of health policy being informed by a true breadth and depth of knowledge and experiences.
Jo Maybin is a fellow in health policy at The King’s Fund, where she leads research on patient, carer and staff experiences of healthcare in England. Her book, Producing Health Policy: Knowledge and Knowing in Government Policy Work, which draws on her ethnography of policy-making in England’s Department of Health, was published in 2016.