You don't fatten a pig by weighing it.

So the saying goes. What, then is, the point of 'assessing' a service as a PIE?

In this section of this site we explore a range of issues in assessment and evaluation of complex needs services, as these are the central focus of this site. In particular we aim highlight the difference between 'formative' and 'summative' evaluation approaches.

Even so, such terms as 'assessment' or 'evaluation' seem to imply an 'external' (and therefore supposedly objective) view of a service, technique or policy, according to predetermined criteria or standards. This in itself is problematic, in this context.

Appraisal : a more constructive alternative

We suggest instead that the experience and views of a wide range of participants and other stakeholders offers a better approach for a more nuanced recognition of the successes, the strengths and weaknesses in context, of complex interventions for complex needs in a complex world.

For this we suggest the practice and the term 'appraisal' - as in '360 degree appraisal' - as a more suitable and constructive alternative; and several pages and papers here explore this alternative paradigm of 'horizontal accountability'.

The rest of this section is then given over practical ways to structure that 'in the round' feedback, for maximum effect; and to outlining our own approach to the assessment and appraisal of PIEs, using the Pizazz -  both the 'pen and paper' version, and now the still un-tapped potential in the software - the PIE Abacus (HERE).

On evaluation

In 'On evaluation' we look first at some of the more abstract or theoretical issues in the various approaches and styles of evaluation, and their relevance - if any - to the assessment of PIEs.

It's possible that this area is mainly of immediate interest to researchers looking for an appropriate methodology for evaluation.  It does, however, offer some word of caution for others, such as policy makers and funders, who want the services they fund to be 'evidence-based', and so these might be seen as the 'end users' of evaluation methods.

So it matters to the services they fund, to be aware of these differences, and to see the strength of the alternatives.

On evaluation : specifics

• Service evaluation by outcomes: HERE
• Evaluations of specific interventions: HERE
• Whole systems evaluation: HERE
• Formative vs summative evaluation: HERE
• Outcome and process assessments: HERE

The Pizazz on paper and on screen : HERE

360 degree appraisal : HERE

The Pizazz on paper

In the pages on 'The Pizazz', we then illustrate an alternative approach, formulated especially for services that wish to develop using the PIEs framework.

This approach was first outlined in principle using 'the Pizazz on paper' - the version of this process that was published in 2018. We have generally very positive feedback on how useful it has been, though this is still largely anecdotal.

This is partly a reflection of the fact that there is no central organisation to collect and collate feedback more consistently. But it is primarily a reflection of the nature of the Pizazz process and philosophy itself, which aims to radically decentralise the process of assessment, to make it fully integrated into services' self development planning.

• • The Pizazz on paper - the paperwork : HERE
  • The Pizazz approach to assessment: HERE
  • The coffee break Pizazz: HERE
  • The Pizazz - what is it NOT? : HERE
  • The Pizazz as a research tool: HERE

The PIE Abacus 

The PIE Abacus - the Pizazz process in software form, but with many other additional features - aims to preserve and to extend the radically decentralised nature of the 'Pizazz on paper' assessments, and the full integration into service planning. But the PIE Abacus, as software, then also lends itself to sharing and pooling of assessment, forming a bigger picture.

The PIE Abacus can then be used 'internally' - that is, within any organisation or network, when the participants - staff and any others invited to comment - wish to share their views on the service's degree of development as a PIE.

But it can also be used to gather feedback from a range of 'external' agencies,  partners in collaboration efforts to meet the needs of those with the more complex needs that may have - or have had - many service involved.

Those 'other' agencies are best place to give the most meaningful feedback not just on the overall success of an agency, but on all or any of ther areas where the agency's work, and working relationships, could be improved.

The PIE Abacus

The simplest, most effective way to evaluate and improve your PIE

(from Daniel O'Brien on Vimeo.)

For more on the PIE Abacus, see:

• The PIE Abacus - an on-line Pizazz (summary) : HERE
• Ambition and modesty : HERE
• The PIE Abacus - FAQs : HERE
• PIE Abacus use and terms of use : HERE

Relative strengths

For a little more on the relative strengths of each - the Pizazz 'on paper' and the Pizazz 'on screen' - see :  HERE

Meanwhile, for more on the business model underlying the PIE Abacus' aims to bring about an active shared, community of practice approach to developing the PIE approach, in future, see:

• PIE Abacus use and terms of use : HERE
• Costs as income: HERE
• PIE Abacus - balancing cost factors: HERE

The PIE Abacus – text and video briefings for PIE leads : HERE