Rachel Dannefer is an experienced public health evaluator and researcher, and a CUNY Urban Food Policy Institute Fellow and PhD student at CUNY. She has dedicated her career to conducting public health evaluations in New York City. We sat down with Rachel to discuss the power of evaluation for strengthening public health and food systems projects.
Katherine Tomaino Fraser (KTF): HI RACHEL, THANKS FOR JOINING ME TODAY. YOUR BACKGROUND IS IN PUBLIC HEALTH EVALUATION WORK, AND YOU’RE NOW PURSUING YOUR PHD, A DEGREE THAT IS TYPICALLY ASSOCIATED WITH ACADEMIC RESEARCH AND RESEARCH METHODS. TO START, CAN YOU PLEASE DESCRIBE TO OUR READERS HOW EVALUATION IS DIFFERENT FROM ACADEMIC RESEARCH, AND IN WHAT WAYS THEY INTERSECT?
Rachel Dannefer (RD): In general, the goal of most evaluation is to understand how well an intervention works. Evaluation explores questions like—What is the impact of this policy? Is this program working in the way we expected? What’s working well and what are areas for improvement? Is the program working differently for different populations?
In contrast, the primary goal of academic research is to generate generalizable knowledge—this knowledge is often useful for planning programs or advocating for policy.
These two approaches can overlap—for example, if we are evaluating or testing out a new program and aiming to generate knowledge about that approach, this can be considered research as well as evaluation.
Also, these approaches can—and should—be used to inform each other. For example, when I was at the Bureau of Harlem Neighborhood Health at the NYC Health Department, our program evaluation data showed high levels of food insecurity among community residents who participated in various programs connected with the Bureau. Based on this evaluation data, we developed a research project to better understand experiences with food insecurity and food access among neighborhood residents. Our findings from that study were in turn used to develop new programmatic approaches around food security.
KTF: TYPICALLY, IN THE PEER REVIEW LITERATURE, WE SEE LOTS OF PUBLISHED RESEARCH STUDIES AND FEWER EVALUATION STUDIES (THOUGH THE NUMBER OF PUBLISHED EVALUATIONS HAS INCREASED IN RECENT YEARS!) WHY IS THIS THE CASE? AND, WHAT CAN FOOD SYSTEMS AND PUBLIC HEALTH PRACTITIONERS DO TO HELP INCREASE THE EXTENT TO WHICH EVALUATION STUDIES ARE CONDUCTED SHARED WITH THE GENERAL PUBLIC?
RD: Sharing evaluation findings is so valuable—it’s an essential way to build our collective knowledge about food programs and policies. Being able to learn from other evaluations helps shape and strengthen our approaches, adopt and adapt best practices and avoid reinventing the wheel. I always spend time reviewing both peer reviewed and non-peer reviewed publications as a key step to starting any new project or evaluation.
But as you note, there are challenges to getting evaluation findings out there. I think many factors are at play—for one thing, a lot of evaluation is done for internal purposes and with a limited audience in mind. For example, organizational leadership and staff often think about how to improve efficiency or outcomes when they look at evaluation findings, rather than thinking about broader dissemination. Also, publishing can be very time consuming and expensive, and many smaller organizations may not have the staffing, budget, time, or capacity to pursue publishing. Another challenge is that peer-reviewed journals typically require review by an Institutional Review Board (IRB), and many places doing evaluation (like nonprofits or foundations) don’t have easy access to an IRB.
Organizations can share evaluation studies outside of the peer review process, for example releasing reports or results on their websites. But even doing this requires institutional support to give people the time and resources to write a public-facing report. I think the more we as practitioners can communicate the value of having access to this information, and share how we use it, the more we can help to build support for sharing findings publicly. Also, finding ways for larger and more resourced agencies to support smaller organizations in sharing these findings, for example through partnerships with academic institutions like CUNY, can be an effective strategy.
KTF: HOW DID YOU BEGIN YOUR CAREER AS A PUBLIC HEALTH EVALUATOR? WHAT HAS MOTIVATED YOU TO STAY IN THE FIELD FOR AS LONG AS YOU HAVE?
RD: Before I started my Master of Public Health (MPH) program, I had the opportunity to collaborate with an external evaluator through my role coordinating a national initiative to support immigrant and refugee farmers. That exposure made me really interested in evaluation, and even apart from that, I had become interested in how to understand and measure the impact of the food and farming work I had been involved in. During my MPH program I focused on building my knowledge in evaluation and research, both through courses and part-time research positions that gave me experience with things like surveys (I spent hours surveying families in a pediatric emergency room) and data analysis. Right after I graduated I was fortunate to get a position at the NYC Health Department evaluating a food retail program, and I have been doing evaluation work ever since.
There are many things I enjoy about evaluation—I really love the collaborative and applied nature of the work. I enjoy partnering with program teams to identify the right evaluation questions for their programs. I love sharing data back to stakeholders to collaboratively make sense of the findings and to use findings to strengthen their work. I also find it very rewarding to help programs better understand their impact. I am 15 years in and still loving this kind of work!
KTF: TO ME, EVALUATION IS SUCH A POWERFUL TOOL BECAUSE WHEN DONE WELL IT CAN ACHIEVE THREE IMPORTANT OUTCOMES: 1) EVALUATIONS CAN HELP ORGANIZATIONS ANSWER QUESTIONS ABOUT THEIR WORK AND DETERMINE THEIR IMPACT; 2) EVALUATIONS CAN GENERATE DATE AND INFORMATION THAT SUPPORTS THE SCALING OF GOOD PRACTICE; AND 3) EVALUATIONS CAN CONTRIBUTE TO GENERALIZABLE SCIENTIFIC KNOWLEDGE IN THE FIELD. WHEN YOU APPROACH AN EVALUATION, HOW DO YOU DESIGN AN APPROACH THAT BALANCES THOSE PRIORITY OUTCOMES?
RD: I love the way you’ve described what evaluation can do. For me the focus of an evaluation really comes from the needs and goals of the program that I’m working with. The evaluation approach will be driven by what is being prioritized among these different outcomes. In my experience, these outcomes are often approached in stages, with an evaluation approach starting with addressing the first outcome you list above about understanding the impact of the work. Once there is evidence for impact, subsequent evaluations could be developed that focus more on scaling and generalizability.
KTF: YOUR EXPERIENCE IN PUBLIC HEALTH AND FOOD SYSTEMS WORK HAS FOCUSED ON SOCIAL JUSTICE AND EQUITY. WHAT ARE THE UNIQUE CHALLENGES AND OPPORTUNITIES OF FOOD SYSTEMS WORK? AND WHAT DOES IT MEAN TO APPROACH THESE IMPORTANT ISSUES WITH AN EQUITY LENS?
RD: One big challenge is just the complexity of the food system and all of the ways that food intersects with health and social justice. It is a lot to untangle! As for opportunities, I think just the nature of focusing on food is an opportunity – it is something we all have in common, and food can be such an amazing way to build community and create bridges across different language and cultures, and to start conversations about social justice.
For me, an important part of using an equity lens is framing and context. For example, contextualizing health inequities by highlighting the role of structural racism, wealth inequality, and policies like redlining and urban renewal in shaping our society and our neighborhoods—and therefore our food environments—serves to remind us that the inequities we see in our country didn’t just come out of nowhere. Another aspect of using an equity lens is bringing community voices into the process. Community members are experts in their own experiences, and when possible, we need to compensate them for their time and expertise. Sharing results back to people who participated in the evaluation process to confirm that findings reflect their experiences, and to make the process more transparent and less extractive is another way to bring equity into evaluation. This is an area where I always want to do more, and I’m hoping to take on more participatory evaluation projects in the future. Finally, as a woman who is cisgender and white, I try to have self-awareness about the power differentials in the work, especially when I’m not a member of the community being served by the program I’m evaluating.
KTF: IF AN ORGANIZATION OR INDIVIDUAL DOING FOOD SYSTEMS WORK IN NYC IS INTERESTED IIN EVALUATION, BUT DOESN’T HAVE ANY EXPERIENCE HOW CAN THEY GET STARTED? WHAT ADVICE DO YOU HAVE FOR YOUNG FOOD SYSTEMS EVALUATORS ENTERING THE FIELD?
RD: There are some really great resources out there – in NYC, the New York Consortium of Evaluators offers networking and professional development opportunities. It’s the regional affiliate of the American Evaluation Association, which is another great resource. Their blog, AEA 365, provides daily evaluation tips on a wide range of topics, and you can sign up to have them delivered to your email. Resources on equity and data include this paper on data for equity, this paper on structural racism and health, this publication on culturally responsive and equitable research and evaluation, and this toolkit on the spectrum of community engagement to ownership. Finally, a personal passion of mine which is useful for anyone who deals with presenting and summarizing data is data visualization—some great resources for that are Evergreen Data and Data Depict Studio.
Rachel Dannefer is a CUNY Urban Food Policy Fellow and a PhD student at CUNY SPH. She has more than 15 years of experience in public health evaluation and research with a focus on social justice and equity. Previously, she worked at NYC DOHMH where she conducted evaluation and research on food security, nutrition education, healthy food retail interventions, and neighborhood food environments. Earlier in her career she worked to support immigrants and refugees in starting their own small farms in NY and nationally. Rachel currently works as an independent consultant and her recent food-related work includes serving as an evaluation consultant for Edible Schoolyard NYC and supporting Harvest Share, a multisector collaborative program to offer culturally-tailored food boxes to diverse communities in Sunset Park, Brooklyn housed in the Section for Health Equity at the NYU Grossman School of Medicine.