Whole systems evaluation and the public health paradigm

Whole systems

Granted the complexity of need, the multiplicity of services to meet specific needs, and the need therefore for effective partnership for those with complex needs, we are increasingly seeing attempts at encouraging more systemic integration between (primarily) local services.

Meanwhile, for a fully systems focussed analysis, we need to look beyond the evaluation of single, and specific interventions, and at public health as the more relevant paradigm for complex systems evaluation. The infographic of obesity is often cited as an example of the sheer complexity of causal connections to be addressed.

Particularly valuable in this context is recent work done by Collaborate and colleagues at Newcastle (now Northumberland) University (see panel right: 'A whole new world: funding and commissioning for complexity'); and likewise the recent report on Enabling Help from the Triangle consultancy team (HERE) .

Both focus on the need to develop a different way of thinking for a very different way of working. This approach challenges the 'contracts culture' of the past 20+ years; and proposes a 'new paradigm' in which partnership and trust, rather than competition and monitoring, is the way forward.


The PIEs 2.0 framework and local whole systems evaluation

We see this development as being very much in line with the both the 'action learning' and the wider systemic overview which was consciously included in PIEs; and this wider view has been built in to the PIEs 2.0 framework.

A more 'whole systems' approach, towards what is sometimes called a 'services eco-system', or 'a PIE of pathways" (HERE) , means, in turn, an attempt at 'whole systems' evaluation, and a search for measures of systemic integration between local services.

This possibility is built into the Pizazz assessment process - both 'on paper' and on-line', with the PIE Abacus. Both can now be brought to bear upon these more systems-focussed elements of the framework (see: HERE) .


The PIE Abacus in local whole systems evaluation

Despite the inherent interlocking complexity of these 'pervasive interactions', the Abacus in particular now allows the possibility of asking specific questions of all participating agencies, to focus attention on manageable priorities in a local needs and strengths audit (HERE).

It is too soon to say whether the ambition for the PIE Abacus to provide a communications medium for exchange of views at locality level will be realised. But we are keen to encourage local commissioners and planners to consider gaining their own Abacus, with which they can then invite the views and suggestions of any participating services, service users or groups to suggest where the pinch points, and the opportunities, may lie.


A further PIE framework specifically for systems?

In several pages on this site we explore the possibility that a more systemic focus in the future might even call for a revision of some of the elements in the PIEs 2.0 framework (HERE),  and what that might then entail in practice (HERE).



Further reading, listening and viewing

System coherence and its opposites

'Joined up thinking' : HERE

The New Public Management approach: HERE

360 degree appraisal : HERE


PIEs assessment

PIEs assessment - what's the point? : HERE

The Pizazz on paper and on screen : HERE

The Pizazz - what is it NOT? : HERE

The PIE Abacus - an on-line Pizazz (summary) : HERE


On evaluation per se

  • Service evaluation by outcomes: HERE
  • Evaluations of specific interventions: HERE
  • Whole systems evaluation and the public health paradigm: HERE
  • The Pizazz as a research tool: HERE

Library items

  • Public health and social housing - a natural alliance? : HERE
  • Formative vs summative evaluationHERE
  • Outcome and process assessmentsHERE

On complex needs evaluation and research issues generally

  • Learning and enquiry, and the cycles of evidence-based practice : HERE
  • Annie Danuco, on formative vs summative evaluationHERE
  • Becky Rice and Juliette Howe on person-centred research for complex needsHERE
  • Grant Everitt on the range and sheer complexity of data in work with complex needs: HERE
  • Stephanie Barker and Nick Maguire on the lack of studies researching peer support: HERE
  • Sophie Boobis on researchers learning from a dialogue with evolving practice (video): HERE
  • McDonald & Tomlin: on mindfulness evaluation with young people, with cautions over a premature preference for meta-analysisHERE
  • Emma Belton: on the challenges in researching behaviour change in young people; and the search for alternative evaluation approachesHERE
  • Mental Health Foundation: Progression Together, a report with honest comments on difficulties with evaluation studies: HERE
  • Robin Johnson: 'Do complex needs need complex needs services? (Pts 1&2):  HERE
  • Robin Johnson: Public health and social housing - a natural alliance ? : HERE
  • Zack Ahmed on using Participatory Appraisal in involving users in local area needs researchHERE
  • Collaborate/Newcastle University Business School on complexity and a new paradigm HERE and (excepts): HERE
  • Sophie Boobis: Evaluation of a Dialogical Psychologically Informed EnvironmentHERE
  • Brett Grellier: report on a mindfulness programme in three homelessness hostels:  HERE
  • Sophie Boobis An evaluation of facilitated PIEs trainingHERE
  • Robin Johnson (in conversation) on outcomes measurement: HERE


3: On PIEs assessment specifically

  • The Pizazz as a research tool: HERE
  • The iAbacus team on the IAbacus process - developing the questions: HERE
  • 'Useful questions' the Pizazz process handbook: HERE