Evaluating the Children and Family Fellowship

A recent­ly released eval­u­a­tion of the Annie E. Casey Foundation’s Chil­dren and Fam­i­ly Fel­low­ship® found that the inten­sive lead­er­ship pro­gram helped social-sec­tor exec­u­tives trans­form the way their orga­ni­za­tions worked on behalf of chil­dren in low-income communities.

Apply­ing tools and skills from the Foundation’s Results Count® frame­work, Fel­lows were able to embed a data-dri­ven approach into orga­ni­za­tion­al strate­gies, work prac­tices and equi­ty ini­tia­tives and to extend that approach to their partners.

ICF, the con­sult­ing firm that con­duct­ed the three-year eval­u­a­tion, observed that the Fel­low­ship was a unique expe­ri­ence for par­tic­i­pants: ​“While most lead­er­ship devel­op­ment pro­grams focus on the indi­vid­ual leader, the Fel­low­ship addressed indi­vid­ual devel­op­ment specif­i­cal­ly in ser­vice of orga­ni­za­tion­al change to achieve pop­u­la­tion-lev­el results.” All Fel­lows who par­tic­i­pat­ed in the pro­gram between 2013 and 2017 were invit­ed to take part in the evaluation.

Among the evaluation’s findings:

1. Results-based facil­i­ta­tion tech­niques and tools helped Fel­lows hold more effec­tive meet­ings that were focused on results.
2. Fel­lows were able to advance dif­fi­cult con­ver­sa­tions about racial equi­ty with­in their orga­ni­za­tions and use data to iden­ti­fy and begin to address gaps.
3. Orga­ni­za­tions and part­ner­ships that were most suc­cess­ful in pur­su­ing a results focus did so by con­cen­trat­ing on shared goals when work­ing through chal­lenges and using action com­mit­ments to encour­age progress.

“The goal of the Chil­dren and Fam­i­ly Fel­low­ship is to devel­op lead­ers who will have pow­er­ful influ­ence with­in their orga­ni­za­tions and the field on behalf of chil­dren and fam­i­lies,” says Bar­bara Squires, the Foundation’s direc­tor of Lead­er­ship Devel­op­ment. ​“It’s affirm­ing to see through this eval­u­a­tion how Fel­lows are imple­ment­ing what they’ve learned and reori­ent­ing the way their non­prof­it and gov­ern­ment agen­cies cen­ter results and equi­ty in their work.”

Data-dri­ven decisions

Estab­lished in 1993, the Fel­low­ship pro­gram cur­rent­ly com­pris­es a series of 10 four- to five-day sem­i­nars, peer con­sul­ta­tions and indi­vid­ual coach­ing over 21 months. All Fel­low­ship activ­i­ties have been based on the con­vic­tion that mean­ing­ful change requires data-dri­ven deci­sion mak­ing, aligned strate­gies with exter­nal part­ners, an under­stand­ing of sys­tems, true col­lab­o­ra­tion and the abil­i­ty to lead through com­plex­i­ty and ambiguity.

Each Fel­low devel­oped a Results Action Plan, which iden­ti­fies a mea­sur­able change in child and fam­i­ly out­comes to be achieved by his or her orga­ni­za­tion by the program’s con­clu­sion. Accord­ing to the ICF eval­u­a­tion, ​“Focus­ing on the end goal and defin­ing goals in terms of how well the orga­ni­za­tion is help­ing chil­dren and fam­i­lies (to what extent are they bet­ter off) instead of how much assis­tance is pro­vid­ed (the quan­ti­ty of peo­ple helped in a giv­en pro­gram) was a crit­i­cal empha­sis for Fel­lows and their organizations.”

The focus on a desired result for tar­get pop­u­la­tions led most orga­ni­za­tions to incor­po­rate mul­ti­ple new prac­tices into their oper­a­tions, work across orga­ni­za­tion­al and depart­men­tal bound­aries and engage com­mu­ni­ties in mean­ing­ful ways. Data walks, which pro­vide a vis­i­ble dis­play of quan­ti­ta­tive infor­ma­tion to track progress toward a desired result, rein­forced the impor­tance of break­ing down data to ana­lyze dis­par­i­ties among racial and eth­nic groups and eco­nom­ic groups, and encour­aged trans­paren­cy when decid­ing how to adjust and refine efforts.

The ICF eval­u­a­tion team found that results-based facil­i­ta­tion tech­niques helped Fel­lows con­vene meet­ings that pri­or­i­tized con­ver­sa­tions — not one-way con­ver­sa­tions — and kept results in the fore­front. Uti­liz­ing results-based meet­ing agen­das helped to dri­ve the work of groups to be more data-dri­ven and ori­ent­ed to action in ser­vice of achiev­ing a desired result.

Mov­ing from talk to action

A key Results Count tool Fel­lows used was the action com­mit­ment, which iden­ti­fies spe­cif­ic ways for part­ners, time­frames and pos­si­ble met­rics to demon­strate ful­fill­ment of a com­mit­ment. These com­mit­ments, as one Fel­low not­ed, ​“have been a good way to keep track of what I need to do and remain account­able to actions I need to take.”

One of the Fellowship’s major goals was to devel­op lead­ers capa­ble of advanc­ing equi­table results for chil­dren and fam­i­lies. Accord­ing to the eval­u­a­tion, the Fel­lows pri­or­i­tized diver­si­ty and equi­ty with­in their orga­ni­za­tions, facil­i­tat­ed dif­fi­cult con­ver­sa­tions about inequities and used data ​“to effec­tive­ly address race equi­ty and inclu­sion gaps.”

Orga­ni­za­tions and part­ner­ships that suc­cess­ful­ly tran­si­tioned to a results focus faced some chal­lenges along the way. They report­ed encoun­ter­ing an ​“us vs. them” men­tal­i­ty among some part­ners, work­ing to achieve buy-in for a new way of doing busi­ness, and need­ing to revis­it deci­sions and plans based on data rev­e­la­tions. Frank dia­logue about var­i­ous par­ties’ needs and restric­tions helped over­come these hurdles.

The bot­tom line of the ICF eval­u­a­tion: ​“Capa­ble lead­ers can make last­ing impacts when a results focus becomes embed­ded in orga­ni­za­tion­al strate­gies and work prac­tices. Orga­ni­za­tions can main­tain an empha­sis on equi­table results in the face of pol­i­cy and envi­ron­men­tal trends that threat­en the well-being of chil­dren and families.”

Learn more about the cur­rent class of Chil­dren and Fam­i­ly Fellows