This event convened leaders from major practitioner, policy and professional bodies with an interest in the adoption and use of high-quality evidence across their organisations and sectors. The day was a partnership of the Alliance for Useful Evidence (A4UE) and Iriss, and invited senior representatives from the worlds of health and social care, justice, housing and education.
Background
Iriss has recently been involved in the Improving Use of Evidence Strand of the Vision and Strategy for Social Services in Scotland. One of the early messages from a workshop held as part of this strand, which brought together practitioners from across social services, was the need to learn from elsewhere about evidence use and understanding. In addition to this, the strand has also been developing a shared Evidence Statement for social services that explores what evidence use looks like and what we mean by ‘evidence of all types’. At the same time, the A4UE with Carnegie UK Trust, published The Scottish Approach to Evidence, which effectively provoked debate around what is good evidence in the context of the Scottish approach to public policy, which emphasises people and partnership. The A4UE supports cross-sectoral and cross-UK learning and recently held a UK Professions Summit in London, where Heads of Professions committed members to take ‘full account of evidence and evidence informed guidance in their daily decisions and advice to individuals and organisations’. The A4UE wanted to convene a similar event with leaders in Scotland. Therefore, together we arranged the Scottish Evidence Summit to share and explore current evidence use across sectors, and what this, aspirationally, may look like in the future.
What we did
Thirty senior representatives, invited because of their influence on the worlds of social policy and community wellbeing, attended the event. As well as hosting presentations and discussions on the day, we also collected survey responses from participants where we asked the following questions:
- Thinking about your current sector, how much importance is placed on encouraging evidence use at all levels?
- In your area of work, can you give an example of when evidence has been used effectively to make a difference to practice?
- For your sector, what do you think will be the future priorities/challenges for evidence use?
The following event report collates the key messages from these responses, as well as the contributions from the speakers and discussions that took place. Thanks must go to the speakers for their input and to Professor Nick Watson from What Works Scotland for chairing the day.
Speaker contributions
- Karyn McCluskey - Chief Executive, Community Justice Scotland
We also heard from representatives from across the attending sectors.
- Richard Whetton, Head of Partnerships and Collaboration, Police Scotland
- Sally Thomas - Chief Executive, Scottish Federation of Housing Associations
- Theresa Fyffe - Director, RCN Scotland
- Jim Thewliss - General Secretary, School Leaders Scotland
- Susan Taylor - President, Social Work Scotland
The speakers all highlighted why evidence is important: it can support change. The default mode of commissioners, leaders and practitioners should be to look for the most effective approaches to achieving the change they want to see and using evidence to support them in doing that.
As Karyn emphasised, to be efficient and effective we need to know and understand what has already been done well before; and how a good understanding of the evidence base for any intervention can help with this. However, it is not always easy to know where to find relevant evidence, or how to assess its quality and make best use of it.
Participants also made the point that evidence doesn’t speak for itself. We should take into account the context, and people’s experiences and priorities, when developing evidence-informed services and practices: a point strongly made by Theresa. Service leaders need to collaborate with citizens and service users in order to establish what outcomes are meaningful to them. These vital conversations and explorations can also be called evidence. As Susan said, ‘evidence has to take many forms’.
Speakers, generally, highlighted the benefit of professional collaboration. Susan highlighted the value of increasing collaboration between the professionals from medical and social science backgrounds. Sally made the point that effective use of evidence needs collaboration between sectors, for example between researchers producing evidence and the commissioners and service managers. She also spoke about how, as with other relationships in our lives, this isn’t always easy, but when persevered with can be hugely rewarding.
Richard spoke of how this can be challenging, as an organisation like Police Scotland needs to make change on the move, to ‘run fast’, and make ‘decisions as we go’. This dynamic doesn’t always sit easy when trying to pause and consider current and emerging bodies of evidence. Jim pointed out that to do all of this, to be open to embracing discussion around evidence and research, you need to keep communication channels positive and not shut down new and emerging ideas. When phrases like ‘let’s not reinvent the wheel’ are used, what this often does is close down the whole conversation, when actually that conversation may usefully have challenged false or entrenched assumptions that were no longer true.
Using evidence… ‘free cheese and a challenge’
Karyn asked the question - what do leaders see when there is potential to make practice stronger and do things differently? Would they look at a mousetrap and see risk and stay well away, or would they see ‘free cheese and a challenge’. Some of the examples from attendees show that senior leaders do want to do things differently and embrace that challenge. Specific examples given of where evidence use has made a difference were:
- Narrative evidence; service users’ voices, have played an important role in changing perceptions, and then service delivery. For example, the evidence of people in recovery from domestic abuse has helped to shift the thinking of professionals around domestic abuse (Iriss Insight - Domestic Abuse and Child Protection)
- The literacy coaching programme to support pupils’ reading abilities, which was designed in partnership with Renfrewshire council, teachers and Strathclyde University and drew on high-quality research evidence from the Education Endowment Foundation Teaching and Learning Toolkit
- The collection and analysis of local data in Her Majesty’s Prison Polmont defining new, agreed quality indicators, has developed staff and improved outcomes for prisoners
- An evaluation of Street Triage Projects - a university led evaluation of nine pilot schemes in England - supported the set-up of similar partnership work across Police Scotland
- The 2015 Report on Affordable Housing Need in Scotland contributed to the current Scottish Government £3.2bn five year affordable housing programme across the country, which has increased grant rates for its delivery
- Evidence through survey, questionnaire and qualitative interviews published in the Scottish Care report 'Voices from the Nursing Frontline' resulted in recommendations that have made a significant contribution to the re-appraisal of care home nursing
Survey respondents stated that many ‘practice areas’ have been significantly developed using evidence, including housing and homelessness; community and criminal justice; the role of public libraries; the heritage industry; public engagement; violence against women and girls; and money advice. However, people made the point that for evidence use to become a part of daily decisions we require a behavioural and culture shift.
Participants suggested a number of ways of impacting this culture change, which included: truly enable staff by folding evidence use through their roles - don’t make it an add-on; give people protected time to have these conversations; don’t consider evidence as abstract - make sure it is seen as something that can be applied and used to make services and lives better and; make use of national level data and research alongside local level information to inform tailored solutions. A number of participants mentioned that the actual word ‘evidence’ needed to be folded through organisational mission statements, visions, plans and objectives to reinforce the value attached to the use of evidence to inform decision-making. In some sectors this seems to be reflected at policy level, but not necessarily further down the line and feeding into frontline practice and behaviour.
What needs to happen as we go into the future?
There was a broad consensus that increasing the use of evidence needs strong leadership, not least because there are several conundrums which have to be tackled openly. Participants at the workshop identified that support structures are needed to help leaders ‘map their way through’ the evidence landscape and that leaders need to be courageous enough to embrace that. The Ingenuity Gap was discussed - the book about the critical gap between our need for practical, innovative ideas to solve complex problems and our actual supply of those ideas - and that leaders need people to help them bridge this gap.
Related to this, survey respondents pointed out that leaders need support to develop their capabilities specifically around evidence use, for example in knowing when to use what evidence: this reinforced the development role for intermediate evidence and innovation organisations, like Iriss, A4UE and others that work within and across sectors.
What one participant wanted was to work towards a future where there was transparency about how decisions were made and the role that evidence played. Folded within that needs to be an acknowledgement from leaders that they don’t always know the answer, but they are prepared to seek out and enable others to gather and process knowledge that could help them.
Many of the challenges that were explored in the Scottish Approach to Evidence remain. We still don’t know how to co-produce evidence well (even if we can co-design and deliver services). Organisations seem to be increasingly considering views of service users, but not co-producing evidence of effectiveness and impact. Respondents identified there are often gaps in available evidence. Firstly, sometimes there is a lack of causal evidence around a particular intervention decision-makers might be interested in.
Another sort of evidence that might be lacking is cost-benefit analysis. Costing needs to go hand-in-hand with evidence and decision-making. A survey respondent fron COSLA made the point that more cost-benefit analysis would support the ambition of providing elected members with evidence relevant to decision-making within constrained budgets.
It was discussed that often, when evidence is produced through a long research process (academic or otherwise), the findings might come too late to be useful. To generate more timely and actionable evidence respondents saw a place for ‘practice-based evidence’, as well as ‘evidence-based practice’. They noted that for this to become a reality, practitioners needed support and encouragement in this field from their managers and throughout their career development. Again, this requires a more holistic understanding of where and who evidence comes from.
At the local level, we sometimes lack evidence on the impact of policies and projects. Do we make too many assumptions, when really there is a need to test if that is the case? Increasingly, with a growing focus on outcomes, leaders are considering how they can best use their data (facts, statistics, items of information) to develop a local evidence base, or to prove or disprove the impact of their services. As one respondent said:
A real data challenge is not only how data is collected (consistently, routinely, etc.) but also how that data is analysed and how partners have confidence and trust of other partners in that analysis / reporting.
Leader in policing/social justice
This resonates into completing the improvement cycle of ‘plan, action, do reflect’. It is in the completion of that cycle that we often see a crack. Not enough time, money and value is given to planning evaluations from the start and making plans to reflect and act upon this, whether that results in doing something new or stopping doing something that already exists and isn’t working.
Even in circumstances where easily accessible, synthesised evidence is available, there are barriers to evidence use. For example, respondents pointed out that in some circumstances policy is ‘as likely to be driven by ‘politics’ as by evidence’. This is something that is widely accepted, hence ‘evidence-informed practice’ gaining popularity over ‘evidence-based practice’. Effective evidence use involves regularly reflecting on the evidence and using this to inform how we think, act and make decisions. We recognise that other factors, such as context, ethics and values, and the nature of our relationships have a central role to play in positive and progressive practice. For example, in our Evidence Statement for Social Services we write:
Effective evidence use involves regularly reflecting on the evidence and using this to inform how we think, our actions and decisions. In the context of social services we also recognise that other factors, such as context, ethics and values, and the nature of our relationships have a central role to play in positive and progressive practice.
We acknowledge that evidence doesn’t happen in isolation, but that it should be used to enlighten and inform future direction and action.
What next for us?
This event has highlighted several barriers to evidence use. Common themes covered the need for research evidence to be timely, useful and relevant. Another widely discussed point was that potential evidence users’ need to know where to find evidence, and how to use it alongside service user input. Services such as the Iriss Evidence Search and Summary Service, the evidence summary Insights and process tools are useful resources that offer various levels of support to access existing evidence, but also to help create evidence. Later in 2018 Iriss, along with other partners associated with the Social Services Vision and Strategy, will be building on this event by hosting a seperate day for social services practitioners to develop action around effective evidence use. Please contact Stuart at Iriss if you would like any more information.
The Alliance for Useful Evidence is committed to supporting leaders across social policy sectors to increase their use of evidence. For example, convening roundtables on evidence standards and outcomes-based policy. It is a partner in a new project sharing learning from the What Works Network of evidence-centres across the UK, and would be delighted to work with professionals in policing, social care, education, housing and local government, to support their access to existing research and knowledge, and to provide ongoing opportunities for cross-sectoral collaboration, including between academia and practice.
The A4UE runs masterclasses and seminars to support practitioners and policymakers with addressing some of the challenges mentioned above. It has a programme of Evidence Champions, who are people that have found evidence useful to them, and want to encourage others to use evidence. The A4UE supports Evidence Champions to discuss common challenges and look for solutions. Please contact the A4UE if you’d like more information.
This event report was written by: Pippa Coutts, Alliance for Useful Evidence and Stuart Muirhead, Iriss
Representatives at the event included individuals from
Health
- NHS Lothian
- Royal College of Physicians of Edinburgh
- Royal College of General Practitioners
- Royal College of Surgeons of Edinburgh
- Royal College of Nursing Scotland
Social Services
- Scottish Social Services Council
- Scottish Care
- Iriss
- Care Inspectorate
- Social Work Scotland
Justice
- Police Scotland
- Community Justice Scotland
Housing
- Planning Aid Scotland
- Scottish Federation of Housing Associations
- Royal Incorporation of Architects in Scotland
- Association of Local Authority Chief Housing Officers
Education
- School Leaders Scotland
- Education Scotland
- General Teaching Council for Scotland
Cross sector organisations/representation
- Scottish Government
- What Works Scotland
- Alliance for Useful Evidence
