<pBy Fran Demetriou
The theme of resonated with me. I鈥檓 relatively new to evaluation and it鈥檚 been an intense journey over the last two years in learning about what evaluation is and how to go about it well. This conference (my first ever evaluation conference) was a pivotal point in that journey.
As an 鈥榚merging evaluator鈥�, my first question was鈥� 鈥榳hat does that mean?鈥� I participated in one of the , where one of the facilitators, Eunice Sotelo, did some excellent miming of the concept (I can鈥檛 justify it with text, so you鈥檒l have to ask her nicely to demonstrate it). An audience member in the session called us caterpillars, following on from butterfly references in Michael Quinn Patton鈥檚 inspiring . I鈥檓 not sure we have a working definition of transformation yet, but I鈥檝e got some good imagery.
This caterpillar came to the conference with a good grounding in evaluation, but with a lot more to understand, including where I was at and what I needed to do to develop.
Here鈥檚 what I鈥檝e taken away from my first 重口50度灰conference:
Community spirit and failing forwards
I was struck by the diversity of content in . There is so much to learn about and so much innovation underway to enable us to better address complex social problems. This felt overwhelming as a newcomer, but I was comforted to find a community of evaluators at the conference who wanted to share, collaborate and learn from one another.
It was great to have so many interactive sessions to enable those connections. As an emerging evaluator, I also appreciated the focus given at the conference to welcome us into the community, focus on our development, and provide platforms to hear our perspectives on opportunities to develop the sector.
The emphasis on learning from failure was valuable. One of my conference highlights was Matt Healey鈥檚 interactive session ( where, under Chatham House rules, evaluators with various backgrounds, specialisations and levels of experience shared some of those facepalm moments. It was comforting to know others had made similar mistakes to me, but even more beneficial to learn from others鈥� mistakes to help mitigate them in my own practice.
I learned that, as we continue to transform our practice to tackle complex problems, there are going to be failures along the way 鈥� and that鈥檚 ok, so long as we recognise them, learn and adapt. I went along to the panel session and listened as a highly experienced team shared the challenges they have encountered implementing systems change through the Beacon Foundation in Tasmanian schools. For me, it helped surface the importance of having strong relationships with partners and funders who are willing to fail forwards with us.
We have power! Let鈥檚 share it, empower others and be ready to let go
The conference reiterated for me the power that we hold as evaluators. We have the power to influence who is included in evaluations, and how 鈥� and we need to push back to make sure those who are affected by decisions are involved meaningfully in the process.
Through some enlightening role play, the (Tracy McDiarmid and Alejandra Pineda from the International Women鈥檚 Development Agency) session helped me to reflect on the ever-present power dynamics between evaluation stakeholders, and how to critically assess and address this to ensure stakeholders are included.
I learned that power isn鈥檛 just about how you include stakeholders, but what you bring to each evaluation through your own identity, and the often unstated cultural values you hold. A challenge I will be taking back to my practice is to be more critically aware of my own identity and the impact it has on evaluations I work on.
These conversations and discussions were summed up for me in with the galvanising question: 鈥淲hen will YOU challenge the power when it is denying inclusion?鈥�
It鈥檚 all about values
Very much connected to power is whose values are heard and counted in an evaluation. I went to several sessions dedicated explicitly to values in evaluations. It was exciting to see both the development of theory and the sharing of practical tools for eliciting values in evaluations.
In their plenary, Sharon Gollan and Kathleen Stacey provided a reminder that the benchmark for doing evaluation has been defined by the dominant culture. This was a powerful insight for me 鈥� it seems obvious, but it鈥檚 something easily overlooked. The way we undertake evaluation has cultural values embedded deep within it, and we must take care to think about the suitability of our approaches especially with Indigenous communities.
Being able to elicit values in each stage of an evaluation is a separate challenge altogether from understanding they are important. It was great then to have several sessions focused on identifying different types of values, articulating values approaches, specifying where values fit into an evaluation (at the start, and then they permeate everything), and how to work with these values, especially in culturally appropriate ways.
We like food metaphors
And finally, we must be a hungry bunch, because the sessions were peppered with food references.
Some savoury metaphors included policy being described as spaghetti, with evaluation making it a bento box (Jen Thompson in ), and a key takeaways slide with a pizza image (Joanna Farmer in ).
Pudding was offered up by Jenny Riley鈥檚 and Clare Davies鈥� appetisingly named and Matt Healey鈥檚 ignite session on evaluators as cake .
My favourite food reference, reflecting the importance of power and values, was from Lisa Warner, who was quoted by a panellist in : 鈥淚f you鈥檙e not at the table, you鈥檙e on the menu鈥�.
What鈥檚 next?
I don鈥檛 know about you, but I certainly feel well nourished!
I鈥檒l be transforming my work to better address values, power and inclusion, and I look forward to the Emerging Evaluators Special Interest Group kicking off soon to continue learnings with and from others.
Thanks for a great first conference, and I look forward to seeing you in Sydney next year!
Fran Demetriou works at Lirata Consulting as an Evaluator, and volunteers as an M&E advisor for the Asylum Seeker Resource Centre鈥檚 Mentoring Program.
LinkedIn:
Twitter:
------------
Please note that the original version of this article incorrectly quotes a Developmental evaluation in Indigenous contexts: transforming power relations at the interface of different knowledge systems panellist for saying: 鈥淚f you鈥檙e not at the table, you鈥檙e on the menu鈥�. In fact, the panellist was quoting Lisa Warner who said this in her STEPS team presentation. The post has been updated to reflect this.