重口50度灰

by Sarah Oxford

The 重口50度灰 recently held , a week of online activities to celebrate evaluation. This year鈥檚 event included more than 1,100 registrants with attendance of about 3,000 people across 23 sessions. The engaging session topics ranged from themes addressing practical application such as capability and capacity building in evaluation to encouraging evaluator reflection and behaviour change. FestEVAL gave us ample opportunities to begin this work by platforming experts with diverse life experiences in evaluation.

The topics that caught my interest were the ones that made me look inwards and ask, what can I do to lead evaluations with inclusive and anti-racist practices? As evaluators a lot of our work involves exploring human behaviour change. But what happens when our behaviour and the systems that we work in and through need to change? How can we improve ourselves, our evaluation practices and the systems at large?

A provocation to decolonise evaluation

During the FestEVAL launch, First Nations woman and evaluator discussed that the space in evaluation between theoretical and practical application is where systemic racism rears its ugly head. Skye explained that the commonly discussed importance of self-determination and applying First Nations ways of 鈥�knowing, doing and being鈥� initiated in the early stages of evaluation with goodwill frequently falls flat when the actual work begins.

Why is it so outrageous that an Aboriginal woman would stand up today and say my path here so far has been really hard and lonely and that is not good enough?
鈥� Skye Trudgett 

From Skye's experiences, racism arises in this space when there is a misalignment of values, which are then subjected to processes and systems underpinned by the dominant cultural hegemony. Here, First Nations ideas and practices are driven to align with methodological concepts created and sustained by the dominant White culture. As such, she argued, there is a continual push for First Nations people to do evaluation in the 鈥渞ight or White way鈥�.

Why is it so outrageous that I should suggest that patriarchal colonialism continues to show its face within evaluations of our First Nations practice?
鈥� Skye Trudgett

Building on her experiences conducting evaluations within a system that prioritises the dominant White culture, she made a strong call to action, asking those who are uncomfortable to ask themselves 鈥榳hy鈥� and to reflect on their own positionality. Next, she asked those interested in eradicating racism to dig into that interest because interest is a core ingredient to social change.

In addition to challenging evaluators to be better, Skye suggested practical and tested ways to tighten the space between theory and application.

A good place to begin education and engagement is the 重口50度灰First Nations Cultural Safety Framework. This important document was launched during FestEVAL. The authors are well known facilitators Sharon Gollan and Kathleen Stacey; the project was led by the AES Indigenous Culture and Diversity Committee with a cross-cutting First Nations project team. The Framework will provide crucial guidance to evaluators seeking to undertake their work in a manner that is culturally safe across all phases of the evaluation process. The  重口50度灰First Nations Cultural Safety Framework can be viewed via /first-nations-evaluators

Other suggestions included:  

• Listening
• Employing people from community who hold significant knowledge about local processes and contacts and pay them the same as non-local consultants
• Sharing power and leaving decision making with the community
• Spending time in the community and focusing on building social relationships, not outputs

In addition to extensive efforts in community, she highlighted exemplary non-racist and Indigenous-focused work for 重口50度灰members to learn from:

• for applying a flexible approach
• for establishing an Aboriginal Knowledge Circle
• for creating a 鈥渒nowing doing being鈥� impact framework and demonstrating what success looks like.

An open discussion about 鈥榯he space鈥� and allyship

The session facilitated by , , and thematically built on Skye鈥檚 provocation. This session, designed as an interactive open discussion, encouraged participants interested in engendering social change to engage in dialogue and share their experiences and ideas.

The session focused on four key questions:

• What are the strengths of multicultural evaluation in the areas where you work?
• What are the challenges of multicultural evaluation in the areas where you work?
• What are our aspirations for multicultural evaluation?
• What do you think you need to equip you to undertake evaluations that authentically respond to local cultural contexts?

Through dialogue, the panellists defined multicultural evaluation as a network of like-minded individuals who integrate cultural consideration and context into theory, measures and practice, and as a result, the evaluation is culturally responsive and promotes equity, diversity and inclusion. Participants then joined the discussion.

I think one key thing for me is that multicultural evaluation needs to not just use a good, culturally responsive approach/process, but what is valuable to and valued by different communities need to be infused into what and how we evaluate.
- Panellist

Participants openly discussed challenges when conducting evaluations. One example included negotiating conflicting goals between the funder interested in quick outputs and the evaluator wanting to apply participatory approaches to better serve the community.

It鈥檚 often a trade-off of the priorities of the funder and the community being evaluated. Ensuring that the values of the evaluated community are prioritised or at least equally considered is important - often doesn't happen!
鈥� Participant

Donors' demands for quick results contrast with equitable approaches. Participants argued the importance of setting expectations and pushing back when donors' needs do not align with community values.

It's about setting expectations - the pressure from decision makers above for quick evaluations/timeframes are sometimes based on previous experience or a lack of push back (sometimes from fear of repercussions/reputational risk). In the space of behavioural science in the justice sector in [location], we've been pushing back on this by showing the value of observations, and the importance of spending time and investment to be immersed in the communities that we are wanting to work with to truly understand their context. We try to get those on the ground who know their communities best to be part of the engagement and co-design of solutions to ensure that those voices (conflicting or not) are not lost.
鈥� Participant

Another issue addressed was conducting evaluations in highly diverse settings where groups of people have conflicting needs. Fostering equity across and among people and groups creates challenges for many evaluators.

One of the most challenging and important elements in multicultural evaluation must be the recognition of intersectionality - gender, sexuality, age, disability, economic status - all of which are so challenging in working on communication and data collection in the COVID pandemic, etc. Important not to assume 鈥渃ommunity leaders鈥� are always reflecting the experience of the community 鈥渓ed鈥� - important in relation to gender and age at the very least.
鈥� Participant

To assist with this challenge, one participant suggested that looks at the various interest groups participating in the evaluation. Other suggestions from the panellists included using the 重口50度灰First Nations Cultural Safety Framework, finding culturally appropriate allies, adopting the principle of 鈥榙o no harm鈥� and paying attention to nuance (e.g. generational family change, intersections across identity, diversity within a community). While making these suggestions, panellists openly acknowledged that trying to understand the complexity of context, tension and nuance is a significant task and very challenging, and there is much room for future conversations and learnings on the subject.

Many strategies were discussed as pathways forward for re-imaging multicultural evaluation. For example, a participant suggested focusing on learning local value systems and working to understand the evaluation through the lens of the local communities, as well as focusing on listening, sharing and making decisions with intention. Intersectionality and equity were also discussed. refers to the ways in which different aspects of a person鈥檚 identity can expose them to overlapping forms of discrimination and marginalisation. Participants suggested intersectionality should be applied at the fore of evaluation. is when fairness is considered, and conditions allow each individual to reach their full potential. For example, the evaluator should use if literacy levels vary among participants. All themes discussed are highlighted below.

I was drawn to these sessions because, as a white person, I often reflect on what good 鈥榓llyship鈥� means and my privilege as an evaluator who reports and presents findings on people鈥檚 experiences. These sessions reminded me that this is hard but essential work, and as evaluators, we must keep these conversations going and take the time to commit to self-reflection and improving our evaluation practices.

----------------

Dr Sarah Oxford is a Senior Research Officer (Evaluation) at the Institute for Safety, Compensation and Recovery Research (ISCRR), Monash University

• @SarahWOxford