Whole systems evaluation

Whole systems

Granted the complexity of need, the multiplicity of services to meet specific needs, and the need for effective partnership for those with complex needs, we are increasingly seeing attempts at encouraging processes of systemic integration between (primarily) local services.

A more 'whole systems' approach, towards what is sometimes called a 'services eco-system', or 'a PIE of pathways, means, in turn, an attempt at 'whole systems' evaluation, and measures of systemic integration between local services.

Particularly valuable, in this context, is recent work done by Collaborate and colleagues at Newcastle (now Northumberland) University ( see panel right: 'A whole new world: funding and commissioning for complexity') on the need to develop a very different way of working.

This approach challenges the 'contracts culture' of the past 20+ years; and proposes a 'new paradigm' in which partnership and trust, rather than competition and monitoring, is the way forward.   We see this development as being very much in line with the both the 'action learning' and the wider systemic overview which was consciously included in PIEs; and this wider view has been built in to the PIEs 2.0 framework.

It is also built into the Pizazz assessment process - both 'on paper' and on-line'. It is, nevertheless, too soon to say whether the ambition for the PIE Abacus to provide a communications medium for exchange of views at locality level will be realised.

Meanwhile, a (still relatively new) UK-based research centre, the Centre for Homelessness Impact, has focussed with increasing depth on a wider range of practical issues in the use of data and evidence to encourage change at systems level, primarily but not solely at local level.

Pivoting, for the COVID era, to on-line conferences, their Impact Festival programme for 2020 (HERE) provides a wealth of useful insights and examples, and the 'What works' community (HERE)  promises to expand for 2021.


Further reading, listening and viewing

PIEs assessment

PIEs assessment - what's the point? : HERE

The Pizazz on paper and on screen : HERE

The Pizazz - what is it NOT? : HERE

The PIE Abacus - an on-line Pizazz (summary) : HERE


On evaluation per se

  • Service evaluation by outcomes: HERE
  • Evaluations of specific interventions: HERE
  • Whole systems evaluation: HERE
  • The Pizazz as a research tool: HERE

Library items

  • Formative vs summative evaluationHERE
  • Outcome and process assessmentsHERE

On complex needs evaluation and research issues generally

  • Learning and enquiry, and the cycles of evidence-based practice : HERE
  • Annie Danuco, on formative vs summative evaluation: HERE
  • Becky Rice and Juliette Howe on person-centred research for complex needs: HERE
  • Grant Everitt on the range and sheer complexity of data in work with complex needs: HERE
  • Stephanie Barker and Nick Maguire on the lack of studies researching peer support: HERE
  • Sophie Boobis on researchers learning from a dialogue with evolving practice (video): HERE
  • McDonald & Tomlin: on mindfulness evaluation with young people, with cautions over a premature preference for meta-analysis: HERE
  • Emma Belton: on the challenges in researching behaviour change in young people; and the search for alternative evaluation approaches: HERE
  • Mental Health Foundation: Progression Together, a report with honest comments on difficulties with evaluation studies: HERE
  • Robin Johnson: 'Do complex needs need complex needs services? (Pts 1&2):  HERE
  • Zack Ahmed on using Participatory Appraisal in involving users in local area needs research: HERE
  • Collaborate/Newcastle University Business School on complexity and a new paradigm HERE and (excepts): HERE
  • Sophie Boobis: Evaluation of a Dialogical Psychologically Informed Environment: HERE
  • Brett Grellier: report on a mindfulness programme in three homelessness hostels:  HERE
  • Sophie Boobis on evaluation of facilitated PIEs training: HERE
  • Robin Johnson (in conversation) on outcomes measurement: HERE


3: On PIEs assessment specifically

  • The Pizazz as a research tool: HERE
  • The iAbacus team on the IAbacus process - developing the questions: HERE
  • 'Useful questions' the Pizazz process handbook: HERE