SCENE

Strengthening Capacity for Equity in New England Evaluation Collaborative

Upcoming Event: April 2024 | Care + Connect

Click here to sign up for the mailing list and hear about future events and updates. 

SCENE

About SCENE

Evaluation and applied research play significant roles within our social service, education, and health sectors. Government agencies, philanthropies, non-profit organizations, and other groups routinely commission evaluations to answer critical questions about the effectiveness, outcomes, and impacts of policies, programs, and services. Yet, too often, evaluations do not examine whether and how interventions mitigate social and environmental inequities and injustices. Moreover, the people and communities who are supposed to benefit most too often have little or no voice in the evaluations. This threatens the validity and utility of evaluations and risks perpetuating inequity.

This initiative explores how applied research and evaluation can center social and environmental equity and justice. We bring together professional evaluators and researchers, faculty, graduate students, and community organizations to go on a collective learning journey. Drawing on research on place-based work to generate change, we focus on New England inviting those across health, education, social services, youth development, and other areas within the region to join us. Our ultimate goal is to improve the capacity of evaluators, applied researchers, and students to contribute to more equitable and just social, health, and education sectors within the region.

The initiative consists of several virtual opportunities to learn and network: 

  • 6 virtual events with guest speakers and discussions/3 per academic year
  • Participant-led collaboration streams to develop and pilot new practices
  • Private Slack workspace to facilitate connections and sharing questions/resources
  • Social system mapping of participants who opt-in to facilitate connections

Click here to sign up for the mailing list and hear about future events and updates. 

Leadership

SCENE is coordinated by a core group of faculty at Boston College community coordinators and guided by input from a dynamic group of advisory members.

Faculty and Community Coordinators

Emily F. Gates, Assistant Professor, Measurement, Evaluation, Statistics, and Assessment

Emily F. Gates, Assistant Professor, Measurement, Evaluation, Statistics, and Assessment

Dr. Emily F. Gates is an evaluator, social scientist, and mixed methodologist. Her research focuses on the role of evaluation in addressing complex problems and changing systems. Driven by a democratic vision for evaluation, she advances evaluation theory, methods, and practice that use systems thinking and approaches, make values explicit, and center equity. She recently co-authored a book, Evaluating and Valuing in Social Research (Guilford Press) and guest co-edited a special issue of New Directions for Evaluation exploring systems approaches to evaluation.


Andrés Castro SamayoaAssistant Professor, Educational Leadership & Higher Education

Andrés Castro SamayoaAssistant Professor, Educational Leadership & Higher Education

Dr. Andrés Castro Samayoa is assistant professor of higher education at the Lynch School of Education and Human Development at Boston College. Born and raised in Merliot, El Salvador, his research examines equitable policies and institutional practices in postsecondary institutions within the United States, with a particular interest in issues of social inequities in educational settings. He is the editor of multiple volumes, including A Primer on Minority Serving Institutions (Routled, 2018). His research on racial inequality and postsecondary education has been supported by The Spencer Foundation, the Association of Institutional Research, and AccessLex. In 2021, he was named a Mellon Emerging Faculty Leader by the Institute for Citizens & Scholars.


Lisa Goodman, Professor, Counseling, Developmental & Educational Psychology

Lisa Goodman, Professor, Counseling, Developmental & Educational Psychology

Dr. Lisa Goodman’s research engages communities to illuminate and transform the lives of intimate partner violence survivors. Her expertise encompasses community-based participatory research, community-based responses to intimate partner violence, domestic violence program evaluation, and social justice teaching.


Patricia 'Trish' Dao-Tran, Founder & Principal, Resonance Data Collective

Patricia 'Trish' Dao-Tran, Founder & Principal, Resonance Data Collective

As a community data convener, Trish coaches social innovators to create equitable change through people-centered, community-led, and participatory approaches to data and storytelling. She has led data strategy and evaluation efforts for cradle-to-career, social emotional learning, school health, food system, and collective impact initiatives in diverse non-profit, government, and cross-sector contexts. Trish is most energized by community-centered collaborations that utilize evidence to shift systems, structures, and mindsets to forge an equitable and just world.


Min Ma, Founder and Principal of MXM Research Group

Min Ma, Founder and Principal of MXM Research Group

An independent research and evaluation practitioner, Min draws from a broad toolkit of evaluation, applied social science, and human-centered design methods. Her areas of focus currently include community health, coalition and network development, and data equity. She is experienced in working with programs supporting people on the move in the US and abroad, including migrant workers, refugees, and survivors of human trafficking.


Bianca Montrosse-Moorhead, Associate Professor of Measurement, Evaluation, and Assessment, University of Connecticut Neag School of Education

Bianca Montrosse-Moorhead, Associate Professor of Measurement, Evaluation, and Assessment, University of Connecticut Neag School of Education

In addition to her position on the faculty, Dr. Montrosse-Moorhead is the program coordinator for UConn’s Graduate Certificate in Program Evaluation. As an evaluation researcher, educator, and practitioner, Dr. Montrosse-Moorhead specializes in evaluation theory, methodology, practice, and capacity building. Her work investigates the demarcation of evaluation’s knowledge base, gaps between evaluator education and the demands of on-the-ground practice, the validity of evaluative arguments, and the importance of establishing professional standards for public evaluation work. She also has an active evaluation practice and has directed several evaluations at the local, state, and national level.


Regional Advisory Group

Jade Caines Lee, Assistant Professor, Clark Atlanta University (formerly at University of New Hampshire)

Jade Caines Lee, Assistant Professor, Clark Atlanta University (formerly at University of New Hampshire)

Dr. Jade Caines Lee is an Affiliate Assistant Professor with the Leitzel Center and an Assistant Professor in the UNH Education Department. Her research areas include assessment development, the application of validity/evaluation frameworks to education assessments, and the evaluation of interventions that improve teaching and learning for underrepresented student groups. Active research projects include a summer reading intervention for struggling adolescent readers, an application of an evaluation framework to examine policies developed to use student achievement test scores to evaluate teachers and teacher preparation programs, and an investigation of factors related to teachers' participation in informal learning opportunities for their students (i.e., museums and science centers).


Cynthia Char, Principal, Char Associates

Cynthia Char, Principal, Char Associates

Dr. Cynthia Char, Principal of Char Associates in Montpelier, Vermont, has over 40 years of experience in educational evaluation and the design of educational programs for children, adults and families. Prior to establishing her educational consulting firm in 1996, Dr. Char had been a senior associate in research and design at Bank Street College’s Center for Children and Technology in New York, and at the Education Development Center (EDC) in Newton, Massachusetts. Particular areas of expertise include formal and informal learning settings (schools, homes, museums, libraries, zoos, parks, and other community-based settings); curriculum development in science, mathematics, and literacy; educational technologies and media; whole-school reform initiatives; professional development; and collaborative partnerships.


Cindy Crusto

Cindy Crusto, Professor, Deputy Chair for Diversity, Equity, and Inclusion, Department of Psychiatry, Director of Program Evaluation and Child Trauma Research, Yale University

Dr. Cindy Crusto has more than 20 years of experience in developing, implementing, and evaluating preventive interventions in schools and community agencies. She also has extensive experience providing training and technical assistance to schools and to community-based organizations on the evaluation of prevention programs. She is interested in culturally relevant interventions for children from racial/ethnic minority and low-income backgrounds and in school-based behavioral health services. Dr. Crusto's research examines the impact of family violence on children and the ecological influences on child and family well-being, and includes rigorous evaluations of community-based initiatives involving substance use and violence prevention, and the promotion of school readiness.


Cynthia Roberts, Evaluator, Rhode Island Coalition Against Domestic Violence

Cynthia Roberts, Evaluator, Rhode Island Coalition Against Domestic Violence

Cynthia Roberts is the Evaluator for all programs across the Rhode Island Coalition Against Domestic Violence (RICADV), with a primary focus leading evaluation planning and implementation for the RICADV’s federally- and locally-funded primary prevention strategies, particularly, the Centers for Disease Control and Prevention (CDC)-funded DELTA Impact project. Additionally, Cynthia provides empowerment evaluation support to the Newport Health Equity Zone and technical assistance to local, state, and national partnering organizations. Prior to joining the RICADV in 2014, she was an evaluator for the RI Department of Health’s Statewide Tobacco Control Program.


Andresse St. Rose, Director of Research & Evaluation, Education Development Center

Andresse St. Rose, Director of Research & Evaluation, Education Development Center

Dr. St. Rose is an experienced non-profit leader and education researcher focused on using research and data to promote organizational success, learning, and equity in education. She produces accessible research with credible and actionable findings that benefit the field, and is particularly proud of her ability to translate and communicate complex and nuanced findings to a range of audiences. Dr. St. Rose is committed to doing work to improve education opportunities and access and to transform systems in service of those who have been historically underserved and discounted in education.


Lisa Watts Natkin, Vermont Evaluation Network

Lisa Watts Natkin, Vermont Evaluation Network

Dr. Lisa Watts Natkin is fascinated by how people’s experiences shape their perspectives and worldviews. She loves listening to people’s stories to understand their viewpoints, experiences, and values. She is committed to using her teaching, evaluation, and research skills to help address issues of social justice and climate change. Lisa enjoys empowering people through education and designing curriculum to align learning outcomes and assessment strategies. Dr. Natkin appreciates the field of evaluation’s ability to contribute to change, increase equity, and address global challenges.



Drew Koleros, Senior Researcher, Mathematica

Drew Koleros, Senior Researcher, Mathematica

Drew Koleros is a senior researcher at Mathematica with over 15 years of experience in designing and delivering mixed-methods evaluations and program monitoring, evaluation, and learning (MEL) systems for social and economic development projects. He brings particular expertise in using theory-based approaches that integrate complex concepts and systems thinking into program and evaluation design processes. Drew’s current work focuses on providing evaluation and learning services to programs aimed at addressing disparities in the social determinants of health and advancing health equity. 

Upcoming Event

CARE+CONNECT: Convene a meal and conversation

April 2024

Invite YOU to get to know new or familiar colleagues better over good food (on us!).

Set a date/time and invite 3-5 others; add to shared Miro board

Each person receives a $70 gift card (to reimburse food/transit expenses)

Meet, eat, talk, reflect 

Interested? Questions? Send an email to scenecollab@bc.edu and we’ll follow up :)

Past Event

Co-creating Contractual & Organizational Support

Event SlidesEvent Transcript

Date: Thursday, April 7, 2022 | 3-5 pm

Description: Whether evaluators can center equity in their work often depends on the timeframe, scope, and budget for the evaluation. These contractual details are influenced by those funding and commissioning the evaluation and, sometimes, by the organizations where evaluators work. This session will host dialogue between commissioners and practitioners of evaluation and share strategies to build contractual and organizational support for equity-focused evaluations.

The session will be split into two parts:

  • 3- 4 pm: An interactive panel with commissioners and seasoned evaluators to learn about some of the challenges and creative strategies they see to integrate equity into evaluation contracts
  • 4- 5 pm: Small affinity group discussions to explore challenges and possibilities wherever we are situated professionally, such as external evaluators, internal evaluators, independent consultants, evaluation commissioners/funders, students, etc.

 

Here are a few suggested resources to reflect on this topic:

  • Evaluation as an Ecosystem AEA365 blogpost
    • Considering ways to reframe philanthropic impact to center grantee success
    • Convening evaluation consultants to begin building a collaborative rather than competitive environment
    • Identifying evaluation strategies, tools, and approaches that best support and help us understand intentional collaboration (e.g. ripple effect mapping, outcome harvesting, participatory evaluation, culturally responsive evaluation)
  • We All Count’s Introduction to the Funding Web
    • Understanding the interplay between money, data, and decision making is essential to measuring the equity balance in your data projects
    • Creating a funding web makes it easy to visually organize and understand all project stakeholders and the flow of money, data, and influence between them
    • Funding webs can reveal power gaps and imbalances and how they can be corrected. They are also a systematic way to brainstorm more creative and effective project structures before considering methodologies, project designs, or scope and scale
  • From Contractors to Conduits: An Exploratory Dialogue among Funders and Evaluators
    • A conversation among funders and evaluators centered on factors shaping the philanthropic evaluation landscape that resulted in a roadmap for action
    • Addressing persistent challenges in evaluation will require greater collaboration, and sharing of knowledge and resources among evaluators and with funders, and less market redundancies and unnecessary competition.
    • Improving collaboration, knowledge- and skill-sharing, and diversification and retention of talent can improve quality, increase the application of our evaluation findings, and advance field-level learning that will increase social investments impacts and drive social change.
  • Developing More Intentional Budgets AEA 365 blogpost
    • Redefining ‘Level of Effort’ (LoE) formulas to include variables such as uncertainty, complexity, and ideological alignment of evaluation projects
    • Budgets that account for energy-depleting uncertainty and complexity, and energizing alignment between projects and core values, increase confidence and excitement to take on resource and energy intensive projects
  • Radical Reimagining podcast episode with Rory Neuner
    • Deven Wisner, Tiffany Smith, Libby Smith, and Rory Neuner talk about radically reimagining relationships between evaluators, funders and grantees

Cultivating Self and Teams

Date: Thursday, January 27 2022, 3:00pm - 4:30pm 

Purpose: Invite critical reflection and discussion of how to cultivate ourselves and teams to support equity-focused evaluation and research

Description: Centering equity begins with reflecting on our intersectional identities and differing degrees of privilege and power. We will explore our personal and collective histories and identities and how we can cultivate ourselves and teams.

Issues to explore: 

  • Reflecting on white privilege and white supremacy culture within ourselves and workplaces 
  • Recognizing risks and additional challenges/burdens evaluators of color face
  • Setting up processes to surface our biases and get critical feedback on our work 
  • Building partnerships and new relationships to seek out those evaluators from different cultural backgrounds and lived experiences as well as young and emerging evaluators
  • Bringing community members and intended beneficiaries into evaluation processes in meaningful ways, including developing capacity and compensating them


Suggested pre-readings:


 

Kick-off for Strengthening Capacity for Equity in New England Evaluation Collaborative (SCENE Collab)

Date: October 22, 2021, 11:30am-1:00pm

Description: We invite evaluators, applied researchers, and students within New England to join us for a collective learning journey to center equity in our work. In this kick-off, we will introduce our purpose, share highlights from a recent study that led to this initiative, and connect with each other in small groups to envision the journey ahead. Input from this session will inform how we shape the learning and networking opportunities in the rest of the series.

Slack Channel

Join our growing community of individuals interested in maintaining the conversation in between our various events!

Please click here to join our list and receive an invitation for the Slack channel.

Slack

Social System Map

The SCENE Collab Map represents the ecosystem of professionals and students in New England who are participating in our community of practice around equitable evaluation. We hope this map will help to:

  • Foster relationships and self-organizing: Use the map to find others to connect and collaborate with. Anyone can interact with the map by setting expertise and interest filters.
  • Visualize the community: Give us all a big picture view of the community and explore changes over time.
  • Inform research and field-building: This simple idea of mapping regional evaluation communities is pretty new! We hope to share what we learn with the wider evaluation field.

    To join the map, sign up for the SCENE mailing list. You will receive an invitation in our updates

Supporters

SCENE is generously supported by both Boston College’s Institute for the Liberal Arts & the Barr Foundation. The Greater Boston Evaluation Network provides additional coordination support.
 

Barr Foundation

Barr Foundation's Sector Effectiveness grantmaking "aims to increase the impact of the social sector across New England communities." Learn more here.

GBEN

Greater Boston Evaluation Network "promotes excellence, innovation and equity in evaluation among professional and aspiring evaluators, and those who align their work with the discipline of evaluation in Greater Boston." Click here to learn more and join their network.