Measuring What Matters

Measuring What Matters
By Kathryn O'Neal Dunham, Chief Operating Officer, Philanthropy New York

My early career as a nonprofit development professional included some fairly unforgettable moments.  One such moment included downloading the interim report for a particularly generous funder of my nonprofit organization.  The printer jumped to life, spitting out page after page – in all, the report contained 8 pages, 12 point font, of questions.  Weeks later, I mailed 20 pages, 12 point font, of typewritten prose to the funder (it was the late 90’s – many a tree were sacrificed).  Imagine my surprise when I joined the ranks of philanthropy overseeing a $3.5 million fund to find that grant reports like these were read, carefully filed, and never again seen by anyone other than me, the program officer.  I still feel a flush of embarrassment as I write this: in my 8 years as a funder in two different roles, I read hundreds of grant reports.  But, I never used those reports to ask a bigger question of my work: how and whether we were having a larger systemic impact on the issues we funded.   

Funders often ask grantees to have assessment strategies in place – everything from bullet-pointed outcomes of the numbers of students served in an after-school study to the perennial favorite – the five year, longitudinal cohort study. However, funders don’t always practice what they preach in terms of evaluation. Even when they do evaluate aspects of their work, many foundation staff often believe they lack the experience to effectively contribute. According to a recent survey by the Center for Effective Philanthropy and the Center for Evaluation Innovation:

• About two-thirds of 127 funder respondents do not work for foundations with dedicated evaluation units or departments;
• Less than half of the respondents have advanced degrees in the social sciences or applied research, with only 5% having advanced degrees in evaluation; and
• Many shared challenges with their evaluation practices, including the extent to which their foundations are incorporating evaluation results into the way they approach their work in the future.

Responding to member demand for evaluation training, PNY has developed a three-part program series, Measuring What Matters: Funder Evaluation, which encourages funders to think about how to evaluate their own practices and strategies and build a culture of learning and quality improvement internally.  The series, which launches on May 23^rd, will include perspectives from PNY members both small and large on how to use data that they have already gathered in the course of daily foundation due diligence – like those grant reports!  That’s just one of a number of ideas that we will be exploring as we think about who in our foundations care about this information, why they care, and how they can make different decisions based on what they are learning.

The curriculum also represents a new collaboration with the Equitable Evaluation Initiative.   Participants will learn throughout the three days how to build in “pause” points to explore values, test assumptions embedded in their processes and consider fundamental questions about voice, validity, choice and ownership throughout their evaluation processes.  Throughout the series, we will explore how equity, transparency, grantee engagement and other essential values can manifest in the evaluation processes they design.

As a final step, we hope to also Measure What Matters – asking our members for feedback on how well this particularly intense program supported their learning.  We promise not to kill any trees – Philanthropy New York enjoys the use of a quick Survey Monkey and lots of honest conversation with our members to learn and improve our work!  So, please consider joining us on May 23^rd, June 13^th and June 20^th so that together we can begin to Measure What Matters.