PIEs and evidence

There are still some that protest that there is little 'hard' evidence for the effectiveness of PIEs - although the same charge is then laid against Trauma-Informed Care, which is endorsed by governments the world over. This might suggest that the criteria for 'evidence' adopted in such analyses might be in urgent need of revision.

Nevertheless, whilst much of the PIEs approach and practice fits most comfortably with the action learning model of 'inner-directed' services change, Evidence-Generating Practice was retained as one of the areas to be encouraged. In fact, one of the original intentions and ambitions for the PIEs 2.0 framework, in moving on beyond PIEs 1,  was that it might provide a more sound basis for general research into the effectiveness of PIEs, and related issues.

Meanwhile, looking forward, the wider potential in the relationship between PIEs and research cuts both ways.

On the one hand, we have the ambition to see research into PIEs as such - the development and effectiveness of the approach, and the framework. Here, hopefully, we can go beyond merely measuring 'outcomes', for the sake of contract compliance, to start to delve into what really works, and what matters, in a PIE.

On the other, we have the PIE framework as a tool for research into any aspects of practice with complex needs and responses.

Here we have the more radical aspiration that the PIEs concept, and the PIEs framework and the PIE Abacus in particular, can open the door to approaches and subjects - for examples, TIC - that are otherwise poorly served by narrowly quantitative studies that demand uniformity in  any interventions studies, and cannot deal with (and may simply ignore) complexities, relationships and contexts.

 

Scoping the area : 1  - Research into PIEs

Research into PIEs, as the subject of study itself, may take several forms.  We can explore the value, in a wide range of dimensions, of adopting the framework and the underlying ideas; and/or of using the Pizazz and PIE Abacus more specifically. In the longer term, we can then hope to look at the enhanced effectiveness in improved outcomes in services that do adopt the PIE approach, as a whole or in part.

Much such research currently exists only in small scale studies, often by practitioners with or without the assistance of research specialists; and much of what little is published exists in the form of what academic researchers call 'grey literature'.

Nevertheless, we CAN have the ambition to bring together what research there is currently, of whatever status; and even to develop a PIEs research special interest group. This is currently one of our aspirations for the next steps in 'the PIEs project'.

 

Scoping the area : 2  - PIEs as a research tool

But equally we can explore the extent to which the PIEs framework is now able, in itself, to offer a basis for future research into all or any aspect of work with complex needs and environments.

That is, we can apply the framework not just for those consciously developing as PIEs, but any services or ways of working, even for single interventions, that really need 'rich' descriptions of the context of the work, to be truly valid.

In conjunction with the Pizazz processes and the PIE Abacus, as 'quantised self-report' assessment and development tools, the PIE framework can provide greater clarity and 'richer' descriptions, without sacrificing all of the precision needed for comparability.

For more discussion on this opportunity, see PIE and the PIE Abacus as a research tool : HERE

 

Consistency : sufficient and suitable?

For such larger scale research to be valid, we must address a key question on the level, and the role, of consistency in the terms used. The framework itself as an example of 'conceptual pragmatism' - the validity being measured by the utility.

The primary use and purpose of the PIE framework - and the Pizazz assessment process - is to allow services to identify their own work, in a common vocabulary, customised to their circumstances. It is argued that this is necessary for a research tool, because any study that imposes additional data-collecting demands on a service, over and above what it useful to the service itself, will tend to get fewer adherents agreeing to take part, and more cursory compliance from already busy staff.

The methodological question therefore is whether there is then sufficient consistency between these uses of the PIEs 2 terms, as applied in each particular situation, for legitimate comparison. But the proper measure of success here is not whether there is perfect consistency; it it to compare this attempt at consistency with those approaches that make no attempt at all to take into account local context.

 

PIEs 2.0 as 'a research-ready' framework? 

It remains to be seen to what extent the PIEs framework can live up to the aspiration to provide a tool for more formal and large scale research. What we can argue, however, is that we urgently need to find an alternative or counterpart to standardised measurements for standardised interventions that are required by their methodology  - the ceteris paribus assumption - to ignore all such complexities of context.

The PIElink community's research and evaluation SIG : HERE

• PIEs 2.0 as a 'research-ready' framework? : HERE
• The PIE Abacus as a research tool : HERE
• The PIE Abacus in communities of practice and research : HERE
• Ethnography as method : HERE

Developing as a PIE :  HERE

On evaluation : HERE 

• Approaches to evaluation : HERE
• Evaluation of specific interventions : HERE
• Formative vs summative evaluation : HERE
• Evaluation by outcomes : HERE
• Whole systems evaluation : HERE

PIEs and the Pizazz approach to assessment : HERE

• PIEs assessment - what's the point? : HERE
• How far do you want to go? : HERE
• How's things? (a.k.a 'The coffee break Pizazz') : HERE
• Peer review stage in the Pizazz assessment process : HERE

'A brief future of PIEs' (video) : HERE