Improving use of research in the third sector: A journey with Apex Scotland

Improving research use in the third sector
Published in Case studies on 19 Nov 2012

Apex Scotland is a third sector organisation working throughout Scotland to improve the lives of people who have offended or are at risk of offending. It is involved in a range of interventions: from developing employment skills, running a school inclusion unit, and providing drug and alcohol treatment and support.

This case study describes a project to improve research use in Apex Scotland, and to share the learning with others. Further details about this project, can be found on Improving research use in the third sector: Project report​.

The problem we addressed

Third sector organisations are increasingly under pressure to use evidence to demonstrate the contribution they make, justify continued funding, inform spending cuts, improve performance and maximise efficiencies. At the same time academics are under increasing pressure to demonstrate their 'impact' on the economy, society, public policy, culture and quality of life (HEFCE, 2009). Therefore, both third sector organisations and academics producing research of relevance for the sector have an interest in improving use of research in the third sector. However, there are a range of barriers to optimal research use, including:

  • Research findings (especially in the social sciences) are often equivocal with uncertain conclusions (Coleman, 1991)
  • The speed of the policy process contrasting with the sluggish pace of the research process means that research becomes less relevant than anticipated (Weiss and Weiss, 1981)
  • Outside academia research can be a low priority; non-academics lack time to consult research and it can be difficult to find or access research (Ritter, 2009, p71)
  • There are perceptions of research not being relevant, and other sources of information can be more highly valued (Nutley, 2003)

For academics aiming to support research use outside academia, barriers include: lack of resources (time and money); lack of skills to engage with non-academics and lack of professional credit for activities focused on non-academics (Nutley, 2003)

This project sought to address some of these barriers and improve research use within one third sector organisation (Apex Scotland), and to share the learning with others (particularly with third sector organisations).

What we did

Between July 2011 and July 2012 a member of Iriss, Claire Lightowler, worked with Apex Scotland to help them reflect on and improve their use of research. Throughout the year, Claire spent a day a week working from Apex Scotland offices, most frequently from the Head Office based in Edinburgh. The placement involved three main stages:

Stage 1. Pre-intervention audit of evidence use
Work was undertaken to better understand research use across the organisation prior to any intervention. This was to:

  1. Produce a baseline understanding of research use at Apex Scotland so we would have a better understanding about the impact of the placement
  2. Inform the shape of the placement activities designed to improve research use.

This involved surveying all Apex staff and exploring issues with a sample of staff who had been selected to be 'research champions'. 

Stage 2. Improvement activity
We tried out a range of activities to improve research use, including:

  1. Value of research: Held a workshop about the value of research and evidence for managers across Apex Scotland.
  2. Data profile report: Worked in partnership with one unit manager to produce a report about their geographical area of responsibility, Forth Valley. This was to be of direct use to those working in this location and designed to demonstrate the range of freely available data available to those working in other areas.
  3. Research reading exercise: Circulated carefully selected research and asked people to read, watch or listen to them, and complete a survey based on their response to the exercise.
  4. Service list: With one service manager developed a list of services in their local area, helping them to think through where to look for information, what to search for and then how best to record findings.
  5. Evaluation: Held a workshop about evaluation at Apex for senior managers.
  6. Project planning documentation: Produced sample project planning documentation, which explicitly includes a requirement for evidence.
  7. Evidence summaries: Arranged for Apex Scotland staff to peer review an evidence summary about criminal justice.
  8. Evidence based report: Mapped out the content of a short report to clearly articulate Apex Scotland's vision, activities and the evidence on which this was based.
  9. Ad hoc support: Provided support and advice about research and evaluation, and as part of this set up meetings with other contacts who could further assist with the development of plans and ideas.

The placement fellow was supported in her work by a nominated representative at Apex Scotland, Aidan McCorry, Director of Corporate Services and Planning. In addition eight Apex champions were nominated by their managers to play a leading role in the development of placement activities.

Stage 3. Post-placement survey about the impact
This involved surveying staff about research use following the placement, the usefulness of placement activities and interest in potential next steps for Apex Scotland.

Challenges

1. Redundancies at Apex Scotland
The organisation saw a significant number of redundancies during the placement period, resulting in the loss of around one-third of its workforce. In such a climate there were difficulties engaging people with issues around research and organisational improvement given they faced an uncertain future personally and organisationally. Overall, Apex employees were keen to engage with research and were committed to improving and strengthening Apex, often seeing that better using research could help this agenda. However, the redundancies affected a number of key players in the organisation who had a research related role. Specifically, a number of those who had been identified as 'research champions' for the project left the organisation during this period. In particular, as we began work around improving communication and engagement with the outcomes data collected by Apex the employee responsible for data collection faced redundancy.

2. Lack of 'research champion' role clarification
As discussed above several of the 'research champions' intended to support the placement were made redundant during the year. In addition, given that the 'research champions' had been nominated to fulfil the role by their managers, this meant that several of those recruited lacked a direct commitment to that role. It was also difficult to keep in touch with the 'champions', and given the range of experiences, roles and locations we were never able to really clarify exactly what these individuals could and should do to champion research. The geographical spread of the 'champions' also ensured that they were never able to meet as a group to really work through what their role should be.

3. Role change
In between submitting and being awarded the third sector placement, the award holder (Claire Lightowler) moved jobs, from the Scottish Centre for Crime and Justice Research (SCCJR) to Iriss (Institute for Research and Innovation in Social Services). Whilst there were advantages of this for the placement, enabling Claire to bring in additional expertise from Iriss (around social media and data visualisation) it meant was more difficult to share the learning derived about the implications for academics, with such knowledge sharing occurring at the end of the placement in more formal settings, rather than on an on-going and more informal basis. This ensured the opportunity for strengthening the direct relationships between Apex Scotland staff and academic colleagues at SCCJR was less successful than would have otherwise been the case.

4. Geographical spread and meeting opportunities
The geographical spread of Apex Scotland and the infrequency of whole organisation meetings presented a challenge to support activities with all staff. When events about research were devised as stand-alone events they were subsequently cancelled due to lack of attendance. As a result, where possible research events were organised as part of existing meetings and events (organising sessions in management development days or management meetings). Whilst this ensured attendance was high, these meetings were largely for the senior staff, inevitably meaning that these learning opportunities were not open to all levels of staff. There is potentially also something here to reflect on about whether this had the unfortunate consequence of encouraging a perception that research was for senior staff.

5. Keeping Fridays clear
Finally, it was a significant challenge for the placement holder to keep the allocated placement day clear from distractions. Although based at Apex Scotland offices, it was not uncommon for the Monday-Thursday job to invade on this time. On rare occasions this encroachment was unavoidable, given the need to respond to issues urgently. However, the majority of the time there was simply a need to maintain boundaries, turn off the work mobile and email. However, the placement holder found this to be a considerable personal challenge, and thus had to spend additional days working on Apex placement activity at other times to make up for the distractions on Friday. Although we have no evidence either way, on reflection, we wonder whether undertaking the placement on say a Monday may have improved the situation, perhaps helping to ensure the PI would undertake the placement day with a clearer head!

What we found

  • The project helped to raise awareness of research evidence and the associated benefits it can offer.
  • For individual staff members, the project helped raise awareness of specific pieces of evidence and helped individuals work through potential implications and ways of using relevant evidence to inform their work.
  • There was a real interest and appetite for using research across the organisation. In particular, in general, Apex staff enjoyed being asked to consult research and many did so when asked.
  • Research champions have emerged and have self-identified throughout the placement period. These are generally not the initial champions recruited before the project began, who were largely nominated by managers.
  • The project provided support for those interested in championing research evidence, and energised those with this interest.
  • Research is still seen as an additional task, often not seen to be part of people's responsibilities, not for everyone and not a priority. Therefore, people do not perceive they have time to reflect on research.
  • It was difficult to ensure research activities were jointly owned, let alone led by, Apex staff (with the placement holder tending to fulfil this role).
  • On reflection, more could have been done to link the need for better evidence use with wider developments across Apex around organisational development, improvement and change, which may have helped to secure commitment for specific activities.
  • There is real potential for the work undertaken during this placement to help Apex further develop and grow as a learning organisation, though to fully capitalise on the work to date would require an organisational commitment to take this forward.

For further details about the impact of the project see the Improving research use in the third sector: Project report​.

What next for Apex Scotland

The evidence presented in this report suggests that during the placement period many people in Apex benefited from the increased attention given to research and evidence use. However, as was expressed by some survey respondents, it is important for this activity to be built upon and further developed to achieve real change beyond those who were directly involved in activities during the placement period. A number of potential next steps emerged for Apex to consider:

1. Formation of "Research and Apex" group: The placement holder has offered to host quarterly meetings for a "Research and Apex" group. This group would discuss research findings and support the use of research across Apex, including exploring how to take forward the other suggestions identified below. Five people have indicated interest in being on this group, and the plan is to set up an initial meeting for these interested parties in January 2013. We also plan to continue to track Apex's experience of improving their research use, and to share this learning with others through posting regular updates on the project webpage.

2. Nominated lead: There may be value in designating a nominated lead for embedding research across the organisation. Nominating someone relatively senior in the organisation would ensure clear leadership around research use, whereas, there may be alternative benefits if the nominated lead were someone on the front-line to really demonstrate how research is for everyone throughout the organisation. To enable the nominee to fulfil this role it may be worth considering how to ensure they are given time and space to do it (whether by reducing existing responsibilities or securing additional support).

3. Induction process and job descriptions: There was some evidence throughout the placement period that many Apex staff did not consider consulting research or evidence to be part of their job. There may be something to reflect on here in terms of introducing an evidence component in job descriptions, appraisal and promotion criteria, and/or the induction process. Through these mechanisms, should Apex wish to, it may be possible to strengthen an expectation that staff in particular posts should keep up to date with evidence. However, if this were to be seriously considered, thought would need to be given to how this expectation would be supported.

4. Direct engagement with evidence producers: The value of direct relationships with research producers was highlighted during the placement, and is supported by wider literature. Therefore, perhaps opportunities to strengthen direct engagement with those involved in producing research could be encouraged. One suggestion about how to facilitate this is through encouraging greater academic representation on the Apex board, which could help raise awareness of the evidence base and lead to other opportunities (research projects, placements, etc). The high value placed on events and relationships also suggests the potential value for Apex in supporting and encouraging staff to attend research related events and networks, emphasising the importance of this for future organisational as well as personal development. Although time out of the office, and cost for some events, will be a factor, as a simple step and initial first step there may be value in more systematically raising awareness by circulating details of events organised or promoted through organisations such as SASO, SCCJR, SCCCJ and CJ Scotland, and for senior managers at Apex to be seen to be promoting and raising awareness of such opportunities.

5. Documentation to improve evidence use: During the placement two documents were drafted to explore and demonstrate potential ways of supporting the use of evidence. These were a business case document which included a requirement for evidence, and an initial plan for a document to map out the evidence base for Apex as an organisation. To improve their use of evidence as an organisation there may be value in Apex reflecting on whether they want to support the further development of these documents and ultimately look to support their use across the organisation.

6. Future research activities and workshops: One rather pleasing, but surprising finding was that when Apex colleagues were asked to consult a piece of research a large number of them did so. This highlights that if people across Apex were to take on the role of championing research, simply circulating relevant research material and occasionally asking people to consult and/or discuss it, it is likely that a large proportion of Apex staff would do so. There would also be potential for continuing to arrange research, evidence or evaluation inputs into future staff development days, perhaps contacting relevant academics to explore whether they would be willing to do an input as well as continuing the relationship established with Iriss.

Apex have responded to these suggestions indicating their commitment to implement all the suggestions, see Apex's response to the project. It is the intention that the Apex research group will coordinate future activity to progress these suggestions, which Claire Lightowler will initially support.

Reflections for third sector organisations

Whilst this project generated a range of lessons and suggestions for Apex Scotland to explore, wider lessons and ideas have emerged that other third sector organisations interested in improving their use of research, may find useful.

  • Champions: There can be value in identifying, encouraging and supporting research champions, who have energy and commitment to improve research use. These champions can be either members of the third sector organisation or from external organisations, and can play a leading role in demonstrating the value of research across the wider organisation. If there is an obvious champion within their organisation they may be willing to take on the role of a nominated lead for improving research use, and potentially coordinate and support a range of research related activities. If there are several champions with an interest in supporting improvements in research use there may be value in setting up a working group to explore research findings and their potential implications for the organisation, and to inform and coordinate activities to improve research use.
  • Research tasks: There may be an unknown appetite for reflecting on research findings in your organisation which has never been uncovered because people have never specifically been asked to consult research. Therefore, a potentially quick and easy way to encourage greater research use amongst individuals in your organisation may be to ask your staff to consult a piece of research, and perhaps discuss it at a future team meeting or write down and share their reflections on what the research may mean for them.
  • Research is for you!: People working in third sector organisations, particularly those in service delivery roles, often don't think research is something for them to engage with, either thinking its not part of their job, it's just a task for senior members of staff or it's for policy focused staff. There also appears to be something about the term 'research' which is a barrier for people, perhaps particularly front-line practitioners. Therefore, there can be value in paying particular attention to demystifying what research is, and specifically encouraging and supporting research use amongst different types of staff at all levels in the organisation.
  • Organisational commitment: There are a range of methods third sector organisations could employ to demonstrate their commitment to being evidence-informed, such as:
    • Create a report setting out the evidence base for your organisational aims, outcomes and activities.
    • Add keeping up to date with research evidence as a specific requirement in job descriptions, criteria for promotion and encourage this to be identified as a developmental area in appraisals.
    • Ensure the induction process makes it clear if consulting research is considered to be part of staff roles, and if so provide training or other forms of support.
    • Invest in building internal research capacity through funding posts (or applying for funding) to support the use of research and evaluation, or look to commission research and evaluation of direct relevance to your organisation. There is potential here to explore ideas with university and college staff who may be looking for projects for their students.
    • Make reference to the evidence base a requirement in key documents: business case, project planning and other related documentation.
    • Provide dedicated time for staff to consult research or conduct small scale research projects.
  • Away days and team meetings: Use existing meetings as spaces to consider further research reflection, perhaps encouraging discussion of particular pieces of research in team meetings or using away days to allow deeper reflection away from the day-to-day work pressures.
  • Engage with researchers: There can be value and mutual benefits in increasing engagement with researchers who are interested in similar issues to your organisation. Academics in particular are currently under pressure to demonstrate the impact of their work outside universities, so may be interested and willing to engage (which does not always need to involve payment). There are different ways to strengthen relationships, some potential ideas could include, encouraging researchers to become board members, developing joint projects with researchers (perhaps jointly applying for funding), offering student placements, undertaking job shadowing, and delivering lectures to students about your work in exchange for academic input into a workshop for your organisation.

Reflections for academics

Academics and other researchers who are interested in improving the use of their research across the third sector may find some of the ideas and reflections below of interest.

  • Time is short (for everyone!): So resources which are short and clearly explain the implications of research for policy and practice are really valuable to those outside academia. Obviously time is also short for you, so there are simple and effective ways of sharing your research, such as recording a 5 minute talking head or podcast to explore the potential implications of your research findings for the third sector. Once you've produced one, this becomes a much quicker way to communicate than writing briefing papers or other resources.
  • Raise awareness of your research: There is often a lack of awareness of and/or an ability to access research resources in the third sector. Although it can feel a little like self-promotion, colleagues in the third sector often value being notified when you produce research which may be of interest to them (through email, twitter, word of mouth, etc.).
  • Step into third sector shoes: It can be difficult for researchers to maintain awareness about the problems facing the third sector and to clearly understand what their research could mean for those working in policy and delivery roles. To strengthen this understanding, there can be value in building relationships with third sector colleagues, whether through sitting on third sector boards, undertaking formal or informal work shadowing, secondments and placements, engaging in joint projects (research or knowledge exchange), applying for funding with a third sector partner, or asking third sector colleagues to read your work and comment on its relevance to them.
  • Valuing other forms of knowledge: Those working in third sector organisations in policy and practice roles have knowledge and experience of different forms. There is value in academics reflecting on this and using engagement opportunities to learn as well as to share their knowledge. Traditional dissemination models, such as lectures, do not lend themselves to this form of exchange and sharing, so there may be value in thinking creatively when planning events.
  • Direct contact is important: Building one-to-one relationships is key to supporting the use of research in the third sector. Therefore, there can be real value in investing time to meet with third sector colleagues face-to-face.
  • Improving research and increasing impact: It can be the case that investing time in supporting third sector colleagues to understand and use your research does not just improve the impact of your work but this also has the potential to inform the production of better research; for instance, through providing challenge and feedback about your findings.

References

Coleman D (1991) 'Policy research-who needs it?' in Governance: An International Journal of Policy and Administration, Vol. 4(4)

Nutley SM (2003) Increasing research impact: Early reflections from the ESRC EvidenceNetwork, Working Paper 16, ESRC UK Centre for Evidence Based Policy and Practice, Queen Mary, University of London (Accessible via www.EvidenceNetwork.org)

Ritter A (2009) How do drug policy makers access research evidence? International Journal of Drug Policy, Vol. 20, 70-75

Weiss J A and Weiss CH (1981) Social scientists and decision makers look at the usefulness of mental health research, American Psychologist, 36(8), 837-847