Case study: Embedding evidence at Includem

Benefits, barriers and lessons learnt
Published in Case studies on 14 Dec 2011

Includem is a third sector organisation supporting some of Scotland's most vulnerable, high risk and challenging young people. It provides relationship-based support to help troubled young people achieve better lives, working in partnership with local authorities, carers, families and other agencies. Includem's tailored support and supervision service includes intensive community based one-to-one support, 24-hour support, short-term crisis support in the community, tapered transitional relapse prevention support, and restorative justice approaches.

Evidence-informed rationale

Includem has embedded the use of evidence from its inception. Right from the beginning, the shape of the services and the organisation was based on the research about "What Works?". The underlying theories and research on which Includem was established are set out in Includem's framework for intervention (PDF). The framework sets out the key messages from the evidence base about a range of issues faced by those Includem supports, for example, pro-social modelling, child development, attachment, trauma, resilience, desistance research etc. The evidence base on these topics is used to develop services, to inform staff materials and work tools, and for training and development purposes.

Embedding evidence

As the organisation has developed, the use of evidence has continued to be embedded. Most commonly, evidence is identified and accessed by "speaking to people who know". Includem staff and managers keep up to date with developments, and often use policy documents and research reports as a useful starting point in team meetings and practice sessions. The organisation has always utilised external expertise, and regularly employs consultants to feed in a specific specialism or knowledge about developments elsewhere. Service developments are always grounded in the research about how to affect change with vulnerable young people. In the development of services, external specialists are often brought in to provide support and guidance about the shape of new developments.

Includem's access to, and awareness of, research is facilitated by the inclusion of academics on the Includem Board, as well as people with a range of other expertise and knowledge (retired sheriffs, accountants, HR experts etc). The academic's contribution is partly about feeding in recent and emerging evidence. Performance and evaluation information is reported to Includem's Service and Quality Committee, which uses the information to scrutinise performance, improve services and identify areas for further development. This committee then feeds in their analysis to the Board. In this way, Includem ensures that the data it collects is used to question and improve performance.

Evaluation

From the establishment of the organisation it was clear that evaluation needed to be built in to the organisation. Includem originally commissioned a team from Glasgow University to evaluate and review projects, and after a number of years implemented evaluation in house. Includem's in-house research team coordinates and develops evaluation and monitoring, and is viewed as one of the core parts of the organisation.

Outcomes focus

When a local authority refers young people to Includem, it asks the local authority to specify the outcomes it wants them to work towards with the young person. An assessment is then undertaken with the young person, which focuses on what the young person wants to change and what they want from Includem. Includem then works to connect these outcomes with those identified by the local authority and develops a support plan, which is reviewed every eight weeks by the young person, social services, Includem staff and other agencies.

At the assessment stage and prior to each eight weekly review, the young person and their Includem worker have a discussion about the outcomes the young person identified they wanted to work towards, and the progress made on this.

Includem uses an outcomes star tool to record progress against the outcomes, which is completed on the basis of this discussion. The worker will structure these discussions in liaison with the young person, often using visual tools and when appropriate, asking the young person to complete an outcomes star based on their perceptions, and sometimes filling one in with them. The Includem worker completes an outcomes star based on their assessment of the young person's progress, and it is this outcomes data, which is used to monitor performance across the organisation. This data is then logged in a central database.

To further improve understanding of the outcomes for young people, Includem is also thinking about surveying parents, carers, and lead workers from the referring organisation, to gain insight into the outcomes achieved.

Satisfaction with services

In addition to the outcomes star, which focuses on the outcomes for those supported, Includem also monitors the satisfaction of those they support by using an online feedback tool known as Viewpoint, which visually represents data and is specifically designed for securing feedback from young people.

Includem shares its satisfaction data with staff and young people. This data is also included in the organisation's annual report and, as with the outcomes data, feeds into its Service and Quality Committee. Includem is currently working to make this information clear through better visual representation (utilising data visualisation support from Iriss). In the future, once they have developed a clear way of communicating the data, Includem wants to add this performance information to the information packs it provides.

Barriers and challenges

Includem staff's focus is always on the young person they support, and as evaluation involves paperwork and administration, it is difficult for staff to find the time to do it, or to prioritise it, despite being aware that it needs to be done. Therefore, Includem continues to devote time and resources to championing and promoting evaluation.

Includem has also faced a range of technical and organisational teething problems around getting the systems in place to facilitate the recording of outcomes and satisfaction data. Systems are improving but a significant amount of time, effort and patience has been required to get them in place.

Staff development

Includem has a learning and development strategy that requires staff to engage in five days of learning and development activity based on their needs and preferred learning style. At Includem, all staff are required to undertake a HNC in Social Care and an SVQ level 3 in Health and Social Care (Children and Young People).

Therefore, the pursuit of these qualifications is one of the key mechanisms through which Includem ensures that staff have a good understanding of the relevant research evidence. The Improvement Manager at Includem, who circulates evidence around team managers for them to disseminate as appropriate, further encourages the use of evidence by way of cascading information.

Resources to facilitate evidence use

Includem has also developed an evidence-based toolkit, called A Better Life (PDF). This toolkit outlines Includem's approach to working with young people. There are 11 modules within the toolkit, outlining approaches and activities that can be used to challenge a young person's offending and risk taking behaviours. Subjects covered include drugs, alcohol and sex, and it includes activities to probe the young person on their feelings towards others, particularly their family and friends, and victims of their actions.

The toolkit is based on a combination of Includem's experience and on the evidence base. The resource has been edited by an external specialist in youth justice.

Every member of staff is provided with this toolkit, and four days initial training and on-going continuous professional development workshops support the use of the toolkit. The toolkit is constantly being improved, for example, at the moment a new section is being developed around working with the families and carers of young people.

Includem has also invested in evidence use by employing skilled people to facilitate the use of evidence across the organisation. It has a monitoring and evaluation team (two people) and an Improvement Manager who has responsibility for learning and development and for organisational development, based on evidence.

Benefits of an evidence-based approach

Includem believes that being evidence based improves its work by:

  • Strengthening confidence in its approach, 'you know what you are doing is right'
  • Enhances its integrity because it has a clear picture, and is open about, its actual performance and outcomes
  • Ensuring it is able to continuously improve
  • Allowing it to prove what it does, the value for money it provides and the difference it makes
  • Ensuring it stays focused on the young people, not on organisational needs.

What Includem has learnt along the way

  • Build evidence in at a very early stage, making it part of what you do and not an add-on.
  • If you are generating evidence, use it! If you want to create a culture of evidence use, you need to use the evidence to improve
  • Don't drown in data! Be wary of gathering too much so that you are not able to use everything you gather
  • Get all your staff on board. They need to see the evidence being generated and how it is being used.