How does the way we do things around here, influence the way we make a difference over there? Mike Corcoran, Associate Consultant with the Co-Production Network for Wales, investigates.
Our Inspiring Impact event series has been an opportunity to bring together a wonderful group people from all over Wales, the UK and around the world, to support one another, share our ideas and experiences, and interrogate the big questions we all face as evaluators. Questions such as:
- How do we strike a balance between the quality of evidence our evaluations must produce, and the quality of experience we owe to all of those they involve?
- How do we ensure our evaluations simultaneously meet the multiple and diverse needs of our organisations, our funders, and our communities?
- How do we evaluate successfully when operating in complex, uncertain and rapidly changing environments?
- How can simple, everyday questions be a launchpad into meaningful, person-centred evaluation for all?
Each of these questions were inherently valuable to ask, and each conversation which followed was rich, challenging, and insightful – but something is missing.
WE CANNOT SEPARATE THE EVALUATION FROM THE EVALUATOR
Evaluation does not exist in isolation. Without evaluators there is no evaluation, and without evaluation, there are no evaluators! Evaluations are undertaken by people, and those people work within organisations – therefore, if we want to improve our evaluation practice, consideration of our personal attributes, our behaviours, and our organisational cultures, must be part of the story.
Can we ask the right questions of those who benefit from our services if we do not ask the right questions of ourselves? Can our services be truly person-centred if our internal processes are not, and what can we do to ensure that our organisational cultures help, not hinder, or journey towards better evaluation for all?
To explore these questions, we were joined by three guests with diverse and invaluable perspectives on the challenge.
- Andy Roberts – Founder and CEO of Weekly10, a software platform built around regular reflection to engage employees, empower managers, and build transparency and visibility throughout organisations.
- Kate Hamilton – Programme Director of Renew Wales, a practitioner-led programme which helps communities in Wales to reduce their carbon footprint, adapt to the impacts of climate change and live more sustainably. Renew Wales are currently working with the Co-production Network for Wales in undertaking their own process of ‘organisational reflection’ and ‘action learning’.
- Rory Cahill – Psychotherapist, Trainer and Lecturer at Warwick University, on a mission to see compassion-focused approaches become the norm across organisations of all shapes and sizes.
Throughout our conversations, three lessons leapt out at me, as ones which can help any of us, no matter where we are, and no matter what we are evaluating.
OUR CULTURE INFORMS OUR EVALUATION, AND OUR EVALUATION INFORMS OUR CULTURE
The relationship between our evaluations and our organisational cultures is not one directional. The culture of our organisations will directly inform the types of evaluation we are likely to favour, but equally, the types of evaluation we undertake can surreptitiously shape our organisational cultures over time. For example, if our organisational culture is highly risk adverse, then co-produced, participatory evaluations will be harder to achieve, and if our cultures are loose and informal, rigorous evaluation may not be as high up on our agenda. But equally, it is not uncommon to see those who operate in compliance driven sectors, with strict processes and a heavy burden on proof, see this impact upon the way in which they converse with one another, share information, and challenge the status quo to drive forward change.
The first step is to open our eyes to this, and to reflect on whether the cultures and evaluations we have, support one another, and support the change we want to see.
YOU CAN EVALUATE THE WRONG QUESTION WELL, AND THE RIGHT QUESTION BADLY
Reflection matters because it helps us ask better questions. An evaluation process may be designed and carried out to perfection, but if we ask the wrong question, it will not give us the information we need. The University of Edinburgh’s ‘Reflection Toolkit’ defines the goal of reflection as to ‘surface learning about oneself and the situation, and to bring meaning to it in order to inform the present and the future.’ If our organisational cultures allow us the space and the time to reflect, and if as practitioners we build our own abilities in critical reflection, we will see the questions we ask of ourselves improve, and the results our evaluations enabling us to increasingly enhance the impact we can achieve.
RISK CAN BE A GOOD THING!
Evaluation is about more than proving – but the shift in culture from prove to improve is a significant one. Implicit within proving is a culture of looking for what’s working and celebrating success. Implicit within improving is simultaneously looking for what else might be happening, embracing failure, and learning from it. With that comes the need to explore the ‘unknown unknowns’ – to ask the questions we have not asked before, and to accept that sometimes we simply cannot define the questions our evaluations must answer before we start. The uncertainty breeds risk, and risk breeds fear – but we should not be afraid. Whilst we cannot always define our questions at the outset as we leap into the unknown, our values, our ways of working, and the established methods we will draw upon if and when we need them, can combine to give us a framework* which keep us safe (and give our funders confidence!) as we explore – and it is through exploring that we can transform our services for the better.
*(Our ‘Measuring What Matters’ approach, is an example of one such framework).
WHAT NEXT?
So, what next? There are big, complex challenges, and knowing where to start can be a challenge. Our guests shared their own words of wisdom.
For Andy “The first step is always transparency. Be open about what you’re doing and why you’re doing it and encourage feedback and conversation. Transparency is a powerful tool because it gives people a voice, makes them accountable, and flattens an organisation”, bringing everyone closer together.
For Kate, the first step is to ask “How are you going to find out the stuff that you didn’t already know about? Whatever you do, it should create something you didn’t expect.”
For Rory, we must begin “recognising that each individual can be an agent of change” and remembering that “we can all have a significant impact.”
And for me, these events themselves have reminded me of the tremendous value of being part of a supportive group of peers, asking difficult questions, and remembering that I am never too busy to stop and think! I’d encourage everyone to do the same, and to join us when we next come together to continue the conversation.
Mike Corcoran, Associate Consultant with the Co-Production Network for Wales, joined WCVA and evaluation practitioners from across the UK, for the final instalment in the ‘Inspiring Impact’ event series, on Tuesday 9 February 2021.
Mike works with organisations across Wales and around the world as an advisor on engagement and impact. He is a long-term associate of the Co-production Network for Wales and led the development of the Network’s ‘Measuring What Matters’ evaluation tool and WCVA’s ‘Simple Questions’ evaluation framework.
Sign up to the Inspiring Impact mailing list to keep up-to-date with upcoming events.