Evaluations of specific interventions

Evaluations of specific interventions

A major area of evaluation is research on the effectiveness of  specified interventions - specific activities (such as CBT, mindfulness, motivational interviewing) where there is a general belief, or simply an assumption, that consistency in  application of the intervention in question is valuable in itself.

Here the principal concern for complex needs service research is to ensure that evidence on such interventions that are proven effective for the majority is still relevant to those most excluded (See Johnson, right panel, on complex needs services). Typically standard treatments need considerable revision, adaptation and customisation ('personalisation") to meet more complex needs; and this may take their actual use far from the standardised original.

Other approaches, more suited to complex and entrenched needs and exclusion, are harder to craft, and they tend to produce softer, more 'qualitative' conclusions. But by being free-er to take into consideration all the elements of context that standardised evaluations must exclude, they may be more relevant.

These considerations have influenced the development of the PIE Abacus, as a research tool. The addition of a specific, perhaps highly specified, intervention as a "bespoke field" in a study with actual services which also record a wide range of other features of their service could provide a platform for the analysis of the effects of specified interventions, which does attempt to include the wider environment.

The 360 degree appraisal approach can also be used to evaluate specific interventions. Here a number of key partners - typically referring agencies or those that work alongside a particular service - are asked to give their views on the impact of a specific intervention as applied or implemented by a single service. The provide a 'rich description', which may be useful feedback for the individual service.  Collating a large number of such appraisals, to assess the intervention on a larger scale, would reuire a sophisticated data share and analysis tool, such as the PIE Abacus.


NB: Some approaches, such as Housing First, combine a number of different elements, as a package of interventions, with some easier to define than others. Despite some early concerns for 'mission drift', those who adopt and promote Housing First have generally been able to manage the dilemma by distinguishing those areas where strict standardisation is expected from others where a high degree of customisation and personalisation is allowed - even required.

(We are promised another paper - 'soon' - from Robin Johnson on how HF and PIEs can best fit together, in research)

Further reading, listening and viewing

PIEs assessment

PIEs assessment - what's the point? : HERE

The Pizazz on paper and on screen : HERE

The Pizazz - what is it NOT? : HERE

The PIE Abacus - an on-line Pizazz (summary) : HERE

360 degree appraisal : HERE


On evaluation per se

  • Service evaluation by outcomes: HERE
  • Evaluations of specific interventions: HERE
  • Whole systems evaluation: HERE
  • The Pizazz as a research tool: HERE

Library items

  • Formative vs summative evaluationHERE
  • Outcome and process assessmentsHERE

 On PIEs assessment specifically

  • The iAbacus team on the IAbacus process - developing the questions: HERE
  • 'Useful questions' the Pizazz process handbook: HERE
  • Robin Johnson (in conversation) on outcomes measurement: HERE