On evaluation

Please note: this section and its summary page are new; and much of the material included is also either entirely new, incomplete, or under-going substantial re-writing. Many of the links also will not work - or may be misleading.

Services working to meet the needs of others will usually have a sense of purpose, or mission, which motivates their work, from within. For those working with individuals with very complex needs, and those who have experienced - or are at risk of - exclusion, rejection and systemic failure, evaluation of our efforts is challenging - and yet crucial.

So central is evaluation, that some have suggested it might be seen as a hallmark of a well-developed PIE. Yet the sheer complexity of complex needs, and complex needs services' work, ensures this is probably only true if we adopt an especially broad interpretation of 'evaluation', to encompass all the areas that we here have instead called a culture of 'learning and enquiry'.

This is an attitude within any organisation, which needs to pervade all areas of the network or service's work, and not come into play solely at the end (see the distinction here between formative and summative evaluation).

When considering the evaluation of psychologically informed environments for this population, we need to bear in mind a number of issues which bear upon the suitability of different evaluation approaches to different circumstances or focusses of action .

In this section, we explore a range of issues in evaluation of services as PIEs.

  • Service evaluation by outcomes: HERE
  • Evaluations of specific interventions: HERE
  • Whole systems evaluation: HERE
  • Formative vs summative evaluation: HERE
  • Outcome and process assessments: HERE

Finally, as many services are keen, and some are now even required, to operate 'as a PIE', we turn, or return, to issues specific to the evaluation of services as PIEs - that is, the extent to which a service is using this framework to guide its development.

  • PIEs assessment: what is the point?: HERE

 

Further reading, listening and viewing

PIEs assessment

PIEs assessment - what's the point? : HERE

The Pizazz on paper and on screen : HERE

The Pizazz - what is it NOT? : HERE

The PIE Abacus - an on-line Pizazz (summary) : HERE

 

On evaluation per se

  • Service evaluation by outcomes: HERE
  • Evaluations of specific interventions: HERE
  • Whole systems evaluation: HERE
  • 360 degree evaluation : HERE
  • Beyond Outcomes - a discussion with Joy MacKeith : HERE

 

Library items

  • Formative vs summative evaluationHERE
  • Outcome and process assessmentsHERE

On complex needs evaluation and research issues generally

  • Learning and enquiry, and the cycles of evidence-based practice : HERE
  • Annie Danuco, on formative vs summative evaluation: HERE
  • Becky Rice and Juliette Howe on person-centred research for complex needs: HERE
  • Grant Everitt on the range and sheer complexity of data in work with complex needs: HERE
  • Stephanie Barker and Nick Maguire on the lack of studies researching peer support: HERE
  • Sophie Boobis on researchers learning from a dialogue with evolving practice (video): HERE
  • McDonald & Tomlin: on mindfulness evaluation with young people, with cautions over a premature preference for meta-analysis: HERE
  • Emma Belton: on the challenges in researching behaviour change in young people; and the search for alternative evaluation approaches: HERE
  • Mental Health Foundation: Progression Together, a report with honest comments on difficulties with evaluation studies: HERE
  • Robin Johnson: 'Do complex needs need complex needs services? (Pts 1&2):  HERE
  • Zack Ahmed on using Participatory Appraisal in involving users in local area needs research: HERE
  • Collaborate/Newcastle University Business School on complexity and a new paradigm HERE and (excepts): HERE
  • Robin Johnson (in conversation) on outcomes measurement: HERE