Introduction

There are well-established practices in evaluation for generating findings and lessons to inform midcourse corrections. Certain conditions must exist for these to be applicable, including clearly operationalized and feasible to measure outcomes; leadership commitment to learning; a structure and process for accountability; and good facilitation. When we want to practice evaluation in service of racial equity, these conditions need even closer attention and additional capacities. This article describes the attentiveness needed, where people tend to get stuck, and tips for moving forward based on our experience.

Five Conditions Requiring Closer Attention

Condition #1: A common understanding about the definition of racial equity and power, as well as what they both look like when there is progress must be actively discussed and reached by funders, implementation partners, representatives from communities of interest, and other stakeholders. These terms are used a lot as strategy outcomes in projects that aim to address racial disparities. We often see words like these as the impact in logic models: an equitable system, health equity, historically disadvantaged populations have power, or equity for families and children in City X. Stakeholders tend to get stuck after implementation begins because typically, the strategy is not powerful enough to make meaningful progress. One reason for this is the lack of common definitions of equity and power. Stakeholders assume that everyone understands equity and power in the same way, which is not true.

Here’s a tip to keep moving forward. Take the time up front to struggle through these definitions with all the stakeholders, even if it means delaying implementation. You can start with these definitions that we use at Community Science:

We have equity when people, regardless of their race first and foremost, gender, sexual identity, disability, socioeconomic status, and any other demographic characteristic, and place of residence have:

  • fair access to the resources and opportunities they need to reach their full potential;
  • the capacity to take advantage of these resources and opportunities;
  • the rights to these resources and opportunities; and
  • the freedom from any discrimination to obtain these resources and opportunities as respected by institutions and the law.

A community’s power is reflected in the ability of its members to work together to get what they want from larger systems. c

We suggest inviting stakeholders to describe how the strategy will help to advance racial equity and build power without using the words “equity,” “inequity,” “justice,” or “power;” and specifically ask who is experiencing inequity and lacking power without using the general terms “communities of color,” “historically disadvantaged communities,” “immigrants,” or “residents of County or City X.” This exercise can help people break down and articulate what they mean in everyday terms. By identifying the populations most affected, people are more likely to develop tailored interventions that consider different communities’ histories and circumstances versus assuming a “one size fits all” approach.

Condition #2: The strategy must explicitly focus on root causes from the start. Closing the disparity gap is often perceived as the same as achieving equity, but they are not the same. The gap could close for many reasons that have little to do with equity. Stakeholders tend to spend too much effort analyzing and synthesizing the data, and not enough on understanding the root causes for the disparity in the first place. Consequently, they become stuck later because the solutions are not sufficient to affect the conditions that contribute to inequity for communities that have been historically marginalized and disadvantaged.

Here’s a tip to keep moving forward. We recommend using the “systems tree” c to methodically identify the root causes of the disparities of interest. The approach begins with analyzing and disaggregating the disparity data as much as possible by race, gender, and other demographic variables; identifying patterns and relationships across different populations and outcomes that contribute to the disparity (e.g., poor housing conditions create unhealthy environments that are not conducive for learning, thus contribute to poor health and education outcomes among young people); and examining the root causes (i.e., policies, practices, and relationships that hold together the systems that create the disparities, and the narratives and mindsets that shape perceptions and drive decisions about policies, practices, and relationships). We suggest stakeholders spend most of their effort understanding the root causes and how they cause similar experiences and outcomes for two or more disadvantaged groups.

Condition #3: Participants experiencing the disparities must be engaged from the start to guide and inform the evaluation, learning, and continuous improvement strategy. Funders, implementation partners, and evaluators would agree that this condition is necessary for any good evaluation. It requires additional intentionality and thoughtfulness when doing evaluation in service of racial equity to ensure that relationships between different historically disadvantaged groups are not harmed, and that funders and evaluators don’t inappropriately use their power in the process. People get stuck here because the engagement process might take longer to set up and complete, causing delayed implementation and baseline data collection.

Here’s a tip to keep moving forward. We suggest stakeholders engage the evaluator early on during the strategy design stage, not after the strategy has been announced and expectations have already been established and communicated. Invite participants for whom the strategy is being designed to meet the evaluator in person or virtually, ask the evaluator questions about their philosophy and approach, and discuss how the participants would like to be involved in the evaluation, learning, and continuous strategy improvement process. Let participants decide how they want to engage without feeling burdened and to be able to hold the funders, implementation partners, and evaluators accountable for the strategy’s continuous improvement and success. Build this step into the design process from the start and ensure meetings with participants are culturally and linguistically appropriate.

Condition #4: Leadership must be held accountable to learning and improving the strategy in two ways — acting on the lessons generated by the evaluation, and working with partners (including grantees) to identify what capacities (i.e., knowledge, skills, resources, and relationships) they need to be successful. Leaders of funding organizations that provide and receive funds must appreciate the value of evaluation and the role it plays in generating lessons and insights — the good, bad, and ugly. They must be curious about past research and experiences and be committed to using what they learned to improve the strategy. Stakeholders tend to get stuck here because leaders are sidetracked by debates about the value of quantitative versus qualitative methods and data, their desire to see a “pet project” to the end, and their lack of time to process and make thoughtful decisions to improve the strategy. Also, evaluators and researchers don’t always produce actionable insights and recommendations.

Here’s a tip to keep moving forward. From the start of the strategy, we suggest you design, articulate, and share the structure and process for developing a learning agenda with all the stakeholders, including grantees. This builds everyone’s capacity to achieve the desired outcomes. We also suggest delivering the evaluation findings to decision-makers and holding them accountable to reflecting on the implications and improving the strategy. Illustrate clearly the lines of communication and accountability between who shares the findings and insights, who receives them, and who decides what to do with them. Determine how the communication will take place, for example, regular presentations and discussions with leadership and/or a memo that synthesizes the implications of the findings and provides recommendations for improvement. Identify champions in the organization who can deliver the feedback and recommendations to the decision-makers. Make the structure and process known to everyone, from the decision-makers to the implementation partners, evaluators, and communities participating in the strategy.

Condition #5. Everyone involved must be willing to be curious and uncomfortable. Everyone must be willing to be curious about each other’s perspectives and experiences with racial inequity and injustice even if they disagree, and to engage in uncomfortable conversations about the root causes of racial and other disparities in the United States. Stakeholders tend to get stuck here because discussions about racism and other forms of discrimination make people uncomfortable, and most people, including evaluators, are not skilled in calling out biases, challenging assumptions, shifting mindsets (our own and those of others), and in dealing with conflict. Consequently, it may be difficult to have authentic and deep discussions about solutions to tackle root causes and about the continuous improvements needed based on the lessons and insights generated by the evaluation and the learning agenda.

Here’s a tip to keep moving forward. Evaluators can partner with professionals who know how to facilitate dialogues about structural racism, recognize what “resistance” could look like, and work with participants to overcome the resistance. Evaluators can engage these professionals early on in the design as “learning facilitators” and build their time into the contract or grant.

Conclusion

The five conditions described above can be challenging to cultivate, support, and maintain. There are no simple solutions, and it is easy to get stuck at any point in the evaluation, learning, and continuous improvement process. The tips in this article are initial suggestions to help funders, implementation partners, evaluators, and participating communities moving forward in a process that supports building power and achieving racial equity.

a Community Science thanks the W.K. Kellogg Foundation for its funding and collaboration to develop the Practice Guide Series: Doing Evaluation in Service of Racial Equity, which allowed us to hone our thinking and articulation of what it means to practice evaluation, learning, and continuous strategy improvement in a way that supports progress toward racial equity.

b A more detailed articulation of the relationship between equity, power building, and power, and their measurement will be forthcoming in the next newsletter issue in April 2023.

c See the Practice Guide on Doing Evaluation in Service of Racial Equity, Diagnose Systems and Biases, for a more extensive explanation about the systems tree.

About The Author

Kien S. Lee, Ph.D., Vice President of Consulting, has expertise in promoting equity, inclusion, and cultural competency for health, food security, civic engagement, and leadership development.

She brings experience as a capacity builder, evaluator, and researcher working with federal and local government agencies, foundations, and nonprofit organizations that are striving to effect community and systems change. Kien is seen as a thought leader in work that occurs at the nexus of evaluation and racial equity and presents and writes extensively about how evaluation can be scientifically rigorous and supportive of racial equity at the same time.