Complex needs require complex interventions, often highly personalised, and typically from flexible and holistic services, working with or alongside others. It is therefore relatively rare* that any one such service's performance is well evaluated by any single, standardised and quantified measurement. The requirement to produce such measured 'outcomes' is more a function of the commissioning models introduced under the New Public Management philosophy than of the needs to the service or its users.
By contrast, the modern knowledge economy has evolved more sophisticated techniques to assess performance in complex, fluid (or 'agile') business environments where the best efforts of any one individual or team in the service are only one contributing part to the success of the whole. Of these, one that seems most interesting for the PIEs approach is 360 degree appraisal - part of the 'formative evaluation' school of methods, providing on-going information for an extended process of change.
More commonly applied to performance appraisal of individuals, a 360 degree appraisal approach to service or team performance consists in an invitation to a range of colleagues and other stakeholders, customers etc to give feedback on one or more aspects of the performance of a service. The key themes in 360 appraisal tend to reflect the values of the agency; and the process tends to accentuate the effectiveness of communications and relationships, both within the service and between the service and wider context.
As with any more comprehensive audit, 360 degree appraisal is considerably more time-consuming than quantified evaluation by agreed outcomes or similar measures, but it provides a 'richer description', more suitable for complex services and complex needs, and more individualised outcomes. Yet is also provides more useful, in depth information on what can be changed.
It has also been criticised for tending towards a more wary culture within an organisation; but this may be reduced when applied to a whole team rather than to an individual. In general it nevertheless still relies on some judgement to weigh up differing accounts and their relative importance to the agency.
Pizazz and the PIE Abacus in 360 degree appraisal
Fortunately the Pizazz and the PIE Abacus are able to work well with this evaluation approach. In effect it simply extends the practice of peer review - the final stage of the Pizazz process - to a wider range of stakeholders, usually but not necessarily local, whose own efforts are impacted by the performance of any particular service or agency. It is a working example in practice of the suggestion of the 'horizontal' or 'lateral' accountability when embedding the PIE approach through a large organisation.
The team discussion that lies at the heart of a Pizazz team assessment, using the PIEs 2.0 framework, can provide a framework for an assessment 'in the round' of a complex and multi-faceted service; and peer review, complemented by service user feedback, can then enrich development planning with a fresh viewpoint on a service.
In a large agency or network, a fully elaborated 360 degree evaluation is even more effective using the PIE Abacus software. Here it can complement the more unstructured conversations in action learning sets with more specified detailed accounts that can be shared and pooled. In sufficiently large numbers, the 'quantising' that is built into the assessment even allows a degree of clarity in summary that may be sufficient for most practical purposes, to identify systemic weakness and a need for change in some aspect of the agency's work.
Probably the most truly effective and reliable form of 360 degree evaluation nevertheless is one that reaches out beyond any particular staff team or service, and invites contributing opinions from other colleagues, partners and stakeholders, whether within the agency or outside it. Although initially intimidating, this need not be rushed; it can be phased in gradually and tentatively.
- A notable exception to this general rule is Housing First. By focussing on the length of a sustained tenancy as the key measure, and treating this as a proxy for all other more individual achievements, Housing First has been able to establish a track record of success. In order to achieve this consistency, however, Housing First excludes from its core model any analysis of the context, or of more effective in-put, of the kind that the PIEs framework in practice provides.
