Lawrence Katz on How Research and Evidence Drive Social Change

Lisa Hamilton:

From the Annie E. Casey Foundation, I'm Lisa Hamilton and this is CaseyCast.

Today, we welcome Lawrence F. Katz, the Elizabeth Allison Professor of Economics at Harvard University. Professor Katz is a distinguished scholar whose work spans labor economics, social issues and the relationship between education and technology.

Along with Amy Finkelstein of MIT, Professor Katz was the co-founder and co-scientific director of the Abdul Latif Jameel Poverty Action Lab, which is often referred to as J-PAL North America. J-PAL is at the forefront of using randomized evaluations to inform policy and reduce poverty. His research on neighborhood poverty, wage inequality and the long-term impacts of fair housing programs align closely with our work at the Casey Foundation.

Professor Katz, welcome to CaseyCast. We are eager to dive into your insights on how research can drive meaningful change in communities across America; a topic I'm sure we could talk about for hours. Start by talking about social research and how research and evaluation helps us know what works.

Lawrence Katz:

So, I think to both have better policy and to bridge large partisan divides, evidence is necessary on what types of policies are effective and evidence on what are really the important problems that we should be addressing. And one thing that I have learned both in my experience in government when I served in 1993/1994 as the chief economist of the U.S. Department of Labor, working in the Clinton administration, for example, and with people in the White House on the National Economic Council, is that evidence from what we think of the gold standard of randomized control trials, which is what's needed if you want to put a drug on the market or for our health, we always use those now, that's what the FDA needs to approve, whether there's a new vaccine that similar evidence for social programs and environmental policies is very helpful and powerful.

And so, it's crucial to create opportunity, reduce poverty, have thriving families, to have our policies be well-informed by credible evidence. And in many cases, a randomized control trial where in the US and many programs are rationed where you essentially use a lottery who gets access. So you clearly can tell the different outcomes for people who get the program versus don't, tell you about what the actual effect of the policy is. Otherwise, one always worries about confounding factors. The type of people who entered a training program or an education program might just be different and would've different outcomes, otherwise. The randomized control trial naturally creates a control group who is the same up to the luck of the lottery or the draw.

Lisa Hamilton:

Although I will note as an organization that is involved in this work, the research rarely tells us yes or no. There's often lots of nuance to it and you have to understand for whom it works and how long does it take for it to work and how long might the effects last, et cetera. So could you talk a bit about these studies aren't necessarily exactly like drug trials where it cured the illness or it didn't, but it gives us lots of information about for whom something might work.

Lawrence Katz:

They are similar to drug trials and that all these same questions also apply to drug trials.

Lisa Hamilton:

True. That's true.

Lawrence Katz:

What dosage we tested at one dosage? Should someone who weighs 280 pounds have the same dosage as someone who weighs 140 pounds? So it worked on adults, will it work on kids? I mean all these questions are also true of drug trials.

When I took over as the chief economist of the U.S. Department of Labor — actually I was the first one there ever was, that we invented the position in 1993 — we did a large review of past government training programs and all the randomized trials and other evidence, and we saw things littered with mediocre, moderate effects that often they had short run impacts, they got people into jobs a little quicker, but their jobs, they could have gotten any.

Lisa Hamilton:

Otherwise.

Lawrence Katz:

But we started also seeing that there were several programs that had much bigger and long-lasting effects, and it showed up in several trials and they seemed to have certain characteristics. They really worked closely, and we came to call them “sectoral employment” training programs. They worked closely with a group of employers in an industry locally to try to figure out what are the jobs that they were looking to hire now that had upward mobility prospects; paid high wages but also were likely to last. They designed training to be broader than a single employer who could then just exploit the workers, but gave you a certification of skills that a whole industry or a whole sector looked at. And they had a bunch of wraparound services.

Once we have gotten you placed in a job, there's a time period we help out. And what we saw is programs that had this characteristic, they weren't for everyone. You had to have a reasonable level of literacy and numeracy to get through the initial screen show up, but they were paying off. And now we have about a dozen randomized control trials with programs — like Year Up for young people that started here in Boston, Per Scholas in the Bronx, the Project QUEST, which works in health care with a lot of single mothers in Texas — that after the training was completed, placing people into much higher wage jobs that were very different than the types of jobs the control group got — because they were breaking down barriers and getting skills — and they were raising earnings 30% or 40% that we see now persisting over a decade in the programs we've been able to follow and we see the characteristics.

You can have all these characteristics but train people for jobs that aren't very high wage and it doesn't do very much. You can have all the screening and placement you want without doing the training to get you up to those jobs. It doesn't matter. And it's very difficult without creating connections with employers to overcome distrust of hiring people who might look different than the people you would normally hire. So having credible evidence from a group of randomized control trials on different populations, you could see which methods seemed to work and which methods didn't in a very scientific way. And I think that's, at some level, what we try at J-PAL North America to encourage a lot of them and just think about synthesizing the evidence. And there are a number of other organizations in the U.S., I think of MDRC, Mathematica, ABT, that similarly have been building up the evidence in this area.

Lisa Hamilton:

Well, that was an eloquent explanation of both how the research happens and what it tells us and how it's in many ways iterative that we learn something, and it might lead us to ask more questions that this continual process of learning really does help make for better social policy, and I would argue better philanthropy, because not just policymakers are following the research results that you have, but also in philanthropy, we are following it and trying to contribute to it because we want to invest in things that work.

Let's dig a little deeper into J-PAL, the organization that you're a part of. What led you to create this research lab? And you're obviously studying lots of issues around low-income, children, families, communities. Why is that the animating question of a lot of the work that you do?

Lawrence Katz:

Yeah. J-PAL existed and was largely working around the world in developing countries doing randomized control trials founded by Esther Duflo and Abhijit Banerjee and Michael Kremer, former students of ours and Nobel Prize winners, Sendhil Mullainathan and Amy Finkelstein, who's was a former undergraduate student of mine at MIT. And I got to thinking, and Amy really pushed this, that there are a lot of organizations like MDRC and Mathematica that do great work using randomized control trials, but it's difficult for academics to get connected with them who also have a lot of ideas and might also be able to connect more to a broader economic research community.

And while academics sometimes did randomize control trials on important policy, when we talked to colleagues who tried to do this, they thought these were amazing projects where they'd learned a ton, but it was just really hard. Everyone was reinventing the wheel every time they tried to do a randomized control trial. They had to figure out how to get through the institutional, the IRB, how to collect administrative data to follow up, how to figure out statistical power.

So there was very little infrastructure for academics to do this. And we had talked to a number of great scholars who said, "Yes, I'd done this was amazing, but I would never do it again because it just took so much time." So we thought, why don't we create both a network of people, so that best practices would be disseminated, but also have a staff that could help deal with this is how you deal with the following issues of institutional review. These are the ethical issues you're going to place. And there were important things that people found hard to do. The wrong way to think about setting up a research program is, you have some idea from up high, you're a scholar, and you just go out and you test it. You actually need to interact with the community and have connections. When we were designing Moving to Opportunity and then Creating Moves to Opportunity, it wasn't that we just said, oh, this is how to help…

Lisa Hamilton:

This would be interesting.

Lawrence Katz:

We actually spent a lot of time talking to people who had been in housing voucher programs about what were the main barriers, where did they want to move when they had a voucher, what were the barriers? Was it financial? Was it information? Was it landlords wouldn't-accept them?

And so we felt that this was the hard part, that having a group that really provided that infrastructure, research management, setting up collaborations with more qualitative researchers, getting you connected to partners, could really facilitate more research using the most credible research methods on social policy and poverty issues in the United States. So that was our motivation back in 2013.

Lisa Hamilton:

So then Dr. Katz, you all have created an infrastructure through J-PAL that allows you to do more academics to really be involved in this research process and to connect them to communities in ways that they might not be able to do themselves. Talk about why poverty is so central to so much of the work that J-PAL does.

Lawrence Katz:

Yeah, I mean I think we were founded and motivated by really issues concerning the persistence of poverty in a high-income country in the U.S. and more broadly in North America. We are J-PAL North America. And, we've really prioritized research that works with more vulnerable communities and how to create both upward mobility for adults, but also intergenerational improvements and outcomes because we really think in the United States and in North America more broadly, inequality and persistence of poverty among plenty is our biggest social program. Divides by race and ethnicity also fit in there. And so our hope is to have some of the both strongest economic researchers focused on these issues, but also part of what J-PAL North America tried to do is expand the network to be much more diverse and not just be people at the top elite universities. And to reach out and to have scholars in every community be involved is another important factor.

But I think ever since I got into this business, out of concerns, my mother was a school psychologist who worked in diverse and low-income communities in Los Angeles when I was growing up. And this has always been what's motivated me, seeing inequality across communities and groups and the tremendous cost both to individual families but to society. Just the research looked at and done in the past more macro economically on how much of our growth comes from reallocating talent, whether it's determined by the neighborhood you grew up in, your race or your gender, not giving opportunities to people are a huge cost to our society.

And a very large part of growth of the US economy has been because of breaking down those barriers. And in some sense, what J-PAL North America does is try to get into the micro economics and the details of how do you create, how do you break down barriers to getting into higher wage jobs? How do you reduce segregation by race, ethnicity and economic status of where people live and what opportunities individuals have? So that's motivated us. And while there are other big issues where RCTs are important, the broader health care system, environmental climate change, we tend to always, when we're looking at proposals, put a very high weight on the impact on poverty and low-income populations.

Lisa Hamilton:

Well, we would agree that is an essential question for our country to try to figure out what works. And so you have lots of insight on research that suggests what effective strategies are for helping struggling families over these decades of research. What are you learning is most effective? Where do we need to be putting more of our bets as a society?

Lawrence Katz:

Yeah, I mean I think we've learned a lot. We still have a lot more to learn. But I think having a safety net with low administrative burdens that provide some reasonable minimal support, especially to households with children, is essential to reduce deprivation. Having stable housing is clearly essential as we've seen in some of the work we've looked at homelessness and housing instability. So you need a reasonable starting safety net, whether it's through a child tax credit or child allowance, and some clear housing support to set a foundation for families. I think what we're learning is while all that is very important, it doesn't create and it reduces deprivation, which is hugely important. You need pathways to higher wage, higher skill jobs are key for durable long run impacts. And some of that is through reducing the segregation of where people grow up and go to school.

We're seeing increasing evidence that giving opportunities for families to move to more integrated neighborhoods and giving children long-term exposure to a more diverse environment, a safer environment has big, long run payoffs, but it also means traditional education on the margin, getting into a selective four-year public university has a very, very high rate of return.

Lisa Hamilton:

One of the projects that I think you are most renowned for is something you've mentioned, the Federal Moving to Opportunity Program. You were the lead researcher on that and many of us have followed it and studied it to inform what we do. Tell us a little bit about what the Moving to Opportunity Program was about and what your key takeaways were.

Lawrence Katz:

Yeah. So Moving To Opportunity is a federal demonstration project that went through the Department of Housing and Urban Development and started in 1994. And what it did is it tried to see, if you help low income families living in public housing projects and high poverty areas who are interested in trying to take their housing support and make it more portable and move to lower poverty, higher opportunity areas, is that both feasible, given what you could rent on a Section 8 or now housing choice voucher. And does that matter for their current economic opportunities and long run outcomes for their kids? We have a lot of evidence historically that families living in more advantaged or safer neighborhoods do better. But one always worries that they're just unobserved differences between families growing up in more advantaged areas, they have greater wealth in many ways, they're more motivated parents.

So, we designed this as a randomized control trials, given we have in the U.S., we ration vouchers. So we essentially, for a brief period following the Rodney King Los Angeles Riot in late 1992, Congress was a bit focused on the issues of segregation, of trying to find strategies to help families in the most high poverty areas in the U.S. And so, an urban bill was passed and there was some demonstration money. And in the early part of the Clinton administration, we had inter agency group and we borrowed some ideas that had been floating around from a lot of housing advocates. What if we could truly test trying to help families with housing vouchers moved to lower poverty areas? And that was the origin. It took place in five large US cities, Boston, Baltimore, Chicago, Los Angeles, and New York City. Eligible families were living in public housing in highest poverty areas of the city, so greater than 40% poverty rates.

Those interested were randomized among getting a regular Section 8 voucher, which in many of these cities had eight year waiting lists to get or getting a voucher plus support to move and a restricted voucher to a low poverty area under 10% or keeping your regular public housing support. And so we've been tracking those families for a long time. There were about 5,000 families, a thousand at each site randomized in the program. And the first thing we found is that about half the families who got the experimental voucher that tried to help you move to low poverty area were successful at moving. That was actually higher than we'd anticipated when we set it up. And so it was possible. And over the next 20 years you see that they are living, they do some moving around, but on average they're living in areas with poverty rates if they move from this program, about 18% points lower than the control groups. So it changed their environment. They're living in safer places and in lower poverty places.

And what we found in the short run from Moving to Opportunity, which a lot of people found disappointing, but I found very interesting, is it looked more like what you might call moving to tranquility than moving to opportunity. That is in the short run, families were much happier, they were safer. You were seeing health benefits for the, these were largely, many were families that were single female headed, but married also. But the adults were reporting less stress, but by itself it didn't change the economic circumstances of the families. And the kids were safer, staying out of violent crime. They were less victimized by crime, but you weren't seeing a lot in traditional test scores from the outcomes. But once we were tracking it over time, what we started seeing is the effects on the adults persisted health improved. There were big reductions in depression and in extreme obesity. Didn't change their economic outcomes that much in family income, but for the kids, once you followed up later, the kids who moved at younger ages and got long to the neighborhood, when we see them as young adults, were earning 30% to 40% more.

They were much more likely to go to college. And when we're looking at the next generation, their kids are growing up in bigger circumstances. And so what it seems to be is that having exposure to safer lower poverty, higher opportunity neighborhoods has very big long run impacts on kids. The way I think of the evidence we've seen there and then a lot of work that my colleagues, Raj Chetty, who's a former student of mine, my colleague at Harvard who works for Opportunity Insight and Nathan Hendren who've worked with me on Moving To Opportunity, we've also the availability of big data be able to link families to their locations and their history of earnings through tax records now and census. We've been able to look at millions of moves of families across different neighborhoods and areas. And what you see is for children, there is really this exposure effect. The longer time you're in a more diverse higher upward neighborhood, the bigger the payoff.

Lisa Hamilton:

So as an organization that does lots of work around neighborhood transformation for just the reason you talked about that where children grow up makes a huge difference in their lives, we do work to try to improve disinvested communities. This project focused on a certain set of families who are getting vouchers who have the possibility of moving to another community. Tell us what your take is on this research in terms of neighborhood effect and what we ought to do. There are certainly families who are going to move, but also families who are going to stay and need stronger neighborhoods. So what is your recommendation you give folks about neighborhoods and how we can make sure the most number of children can grow up in healthy places?

Lawrence Katz:

So, I think of mobility policies that is giving families who have housing support, greater assistance to move where they want to move for the families as a complementary strategy to investing in more disadvantaged neighborhoods and trying to improve them. So if we give more opportunity, and that's our second stage of our research, Creating Moves to Opportunity, that we've been working on, is trying to figure out what the barriers have been to moving and how do we overcome them. What we've seen there is that with a modest investment in mobility counselors relative to the cost of a housing voucher, I can increase the likelihood that a family that wants to move to a higher opportunity area. We did this in Seattle and King County from about 10% to 15% without extra help for maybe $2,500 on average of help on security deposits, counselors and others, we can raise that to 50% to 60% are able to move to higher opportunity areas. So that's a first part, making sure that families who want to live where is affordable and more integrated neighborhoods, making sure the vouchers that then also puts more pressure on landlords in less advantaged areas to try to keep people there by providing better quality housing and improving it. So it creates a market pressure

Lisa Hamilton:

It creates a choice…

Lawrence Katz:

…of choice so that you don't know that you can just keep the place in low rates and get the voucher because they have nowhere to move.

But we also need real investments, these communities. So when I was working in the Clinton administration, we were working both on moving to opportunities as one strategy, but empowerment zones as another strategy of directly not just giving tax rates to claim you made some profits in a community, but to actually invest in the infrastructure and to bubble up, have the community think about what their barriers to mobility were, have some tax credits for employing people from low income communities but also invest, whether that's improving the green spaces, creating a safer environment.

And so it's tougher to do a randomized control trial. I had argued that when we did empowerment zones that we should have randomized among the finalists. I didn't win among that, but we have quasi-random variation, but comparing the winning communities to losing communities. And we saw there that the communities themselves did improve with investments, but what we were unable to do this is work that Pat Klein at Berkeley and some collaborators is what we were unable to do in the past is see, did it actually make these higher opportunity areas or did it just change the mix of people living there? Because we only had repeated cross-section. We could look at the census in 1990. In 2000, we could see people were doing better. But was that changing…

Lisa Hamilton:

…It wasn't the same people.

Lawrence Katz:

Yeah. What we can do now is track people using the links of the census to tax data that we have through census and evaluate policies like empowerment zones or Hope VI revitalization projects. That's what I've been recently working on with my colleague Raj Chetty, some other, Laura Tach who's a sociologist at Cornell, Rebecca Diamond who's at Stanford, a former student of mine and Matt Steger. We've been linking people over time and seeing when you invest in a community through Hope VI to try to create mixed income housing and do investments, if you look at the families who then move in, does it actually create higher upward mobility? We can look before and after, see the families, and what did it do for the families who were there before? Unfortunately, it doesn't seem to do a ton for the families who were there before.

There's such high turnover in such neighborhoods that by the time the neighborhood is revitalized, not a lot of them are there. But for the next cohort that moves in, we're seeing big increases in upward mobility. And in fact, one of the things we're looking at, it looks larger than you would predict from the change in the mix of poverty because we're seeing that creating integrated housing as opposed to a normal neighborhood that has a poverty rate of this level, but one group lives in one set of housing, one group lives in another, creating mixed development seems to create interactions that are very valuable for changing your network. And there are all sorts of interesting new research. One could look at Facebook data and the high schools in these areas and see in fact that you're now seeing kids have more friends from different socioeconomic classes than before the change in the neighborhood and the infrastructure and more than you would predict from a normal neighborhood.

So we're still trying to figure this out, it seems to not have negative effects on the higher SES kids and have very positive effects on the lower SES kids. So that's our hope there is that you can put some market pressure to improve communities by giving more choice to families there, but you can also directly invest in those communities. And I think we need to do both of them. We're never going to do everything through mobility policy, but we certainly should increase the returns on what we already invest in housing vouchers, for example.

Lisa Hamilton:

That's great. And I think the lesson from what you've just talked about so often I have people ask me, what's the one thing we need to do? And there isn't one thing, there are many things we need to do. And I appreciate you talking about how two strategies, empowerment zones and Moving To Opportunity were complimentary and we've got to do multiple things.

At Casey, we are always trying to find ways to connect young people to family-sustaining work, and you have studied many important aspects of this. Could you talk about what we've learned about how to help young people enter and be successful at work?

Lawrence Katz:

What we've seen from a whole series of randomized control trials over decades of, in some sense, much of the work on doing randomized control trials started in the U.S. with both the negative income tax, which today we call guaranteed income, but also a series of training programs in the '60s and '70s. And then building on that for decades with organizations like MDRC, which started working on supported work in the 1970s with the randomized control trial. And so I said what you see is traditional programs that trained for the available jobs that hire workers without college degrees, have some short run success but don't create a lot of pathways to higher wage jobs.

What we're seeing is that when you work in your regional labor market with trying to figure out with a sector, whether it's the financial institutions in New York City as Per Scholas did and what in the early 2000s, what were the need for help desk people in IT today, who's going to maintain your Cisco servers and your Apple devices, but being a nimble organization that trains people for what the current skills needed by a group of employers are, and that also teaches soft skills and plays this intermediary role. So you need real training for stuff that's valuable for jobs that have some upward mobility where companies are finding difficult times hiring people and will be open to people with non-traditional credentials. That's been the sweet spot that is highly effective. And we've seen that in health care. We've seen that in manufacturing in Milwaukee in tech and in business with Year Up.

And so the key questions are: There are a lot of wonderful local nonprofits that have been doing this, but can we also scale this by using the community colleges in the U.S.? I think that's the really where you have a strong education, but creating greater links to employers. And for example, Year Up has been partnering with a lot of community colleges in trying to integrate their connections with employers and curriculum along with traditional community college education. And a lot of states have been growing their joint programs with high school community college and vocational. So I think there are a number of pathways states like Kentucky and Tennessee have been pretty good at doing this. There's a lot of heterogeneity in the U.S., but can we leverage our community college system with local nonprofits that have created connections with employers to create greater pathways to high wage jobs and reduce barriers? And so that's a key point.

The other is, while we've been quite successful with people who have some community college or effectively at least a GED and reasonable levels of literacy and numeracy, trying to create pathways to get there, can you have a program like a YouthBuild which often combines a GED with some other training that may not directly lead to a higher way as a pathway to get into a program like a Per Scholas or a Year up or a community college program. I think trying to get people who had failed the screens initially to get into a sectoral employment program or a well-functioning community college nursing program or something is the other area where we need to be doing research and finding.

So right now, our sweet spot are people who are job ready but lacking those opportunities and providing them the connections and training. But the people who aren't ready for those jobs yet, because they need some further education to get there or they need some experience, that's where trying to build pathways or criminal records is a barrier. We've also been doing a lot of experiments at J-PAL at can you work with employers or placement groups to reduce the screening on criminal records and make employers more comfortable with giving opportunities to workers they might've just screened out initially? That's another area that given the U.S. history is a very important one given the large number of low-income workers who have some blemish on their record who may get screened out. So that's another very important area.

Lisa Hamilton:

That's great. Well, I have to tell you, J-PAL is in some ways invisible but essential infrastructure for social change in this country, and we are so thrilled to have you as a partner and to be able to learn from what you are doing. I'm wondering if you have any last thoughts about how those of us who are trying to make the world a little better for kids and families and communities can learn from and work with your work. What advice or encouragement would you give to those of us in philanthropy or policymakers or even practitioners about how best we can use what you're learning and how we can contribute to additional evidence building in this country?

Lawrence Katz:

Yeah, I mean, I think we have tried at J-PAL North America to also have a very strong policy staff to try to disseminate and summarize, work with our scholars to summarize the research and work with policymakers and funders about what we're learning. And I think there's both, clearly philanthropy has an important role in funding the basic research that is needed to learn these things and helping scholars and fantastic organizations like MDRC, Mathematica and others do the type of work that builds our research infrastructure to do more evidence-based work, and then think about prioritizing resources and things where we have evidence that actually are going to be effective, but also I think play an important role in making sure that researchers are listening to communities and are paying attention. And that feeds in.

One huge advantage that social scientists have over say physicists is, I can't talk to a quark and learn much about its motivation, but I can talk to the people in the communities we're working in and they may not speak in, use the same terms we would do, but I would put much more faith than trying to test a hypothesis that someone says is related to their own motivations that one they said they would never think about in a million years or sounds foreign to them. And so the back and forth between the community, funders and researchers is really important. And one thing we've tried to push at J-PAL North America is quantitative scholars doing RCTs should be working carefully both with their communities and with ethnographic qualitative researchers, both to come up with new hypotheses to test and design interventions, but also to interpret what we're finding.

A lot of, just like in drug trials, maybe you can test the molecular pathway and understand it, but even in drug trials often you need to talk to people to learn where they're feeling better and what we think is different about the community if we move them and can we then test the pathways that people think value. And so I think there's really a need for dialogue and back and forth to make progress. And that researchers need to be very much involved with talking with participants and that I think we all benefit from that rather than scholars in the academy being divided and just using secondary data sets and not listening to what members of the community think are important.

Lisa Hamilton:

Well, that's wonderful. I think you have provided us with so much insight on why evidence building is important for anyone who wants to see good things happen and see society improve in this country. Ways that we can do that work more responsibly and inclusively to make sure we get better results, and to really build on decades of learning that have really helped us understand what does work and what's left still to learn about how to address the big issues of our time such as poverty. We know that is critical work if we are going to help kids and families have productive lives in this country. And we certainly owe gratitude to J-PAL and your many partners in other research institutions for the really important, and as I said, sometimes invisible work you do to help us know how to do that. So thank you so much for joining us today and sharing your insights with our listeners and for all the work that you do.

Lawrence Katz:

Well, thank you. I greatly enjoyed getting to talk about these important issues.

Lisa Hamilton:

Great. And thank you to our listeners as well. If you liked our conversation, please recommend us on your favorite podcasting app. To learn more about Casey and the work of our guest, check out our show notes at AECF.org/podcast. Until next time, I wish all of America's kids and all of you a bright future.