B School for Public Policy
Collecting State & Local Data for Informed Social Policy Making
Dennis Culhane, Professor at the School of Social Policy and Practice at the University of Pennsylvania, joins host Dan Loney to discuss his recent B-School Seminar presented to congressional staffers that focuses on helping staffers better understand how state and local evidence is gathered, which ultimately serves as a basis for forming federal social policy regulations. Dennis is also the Co-Principal Investigator for the Actionable Intelligence for Social Policy, and Director of Research, National Center for Homelessness Among Veterans.
An edited transcript of the conversation follows.
Knowledge@Wharton: There is a concern in this country that some federal level policy decisions does not use the evidence that is collected by people at the state and local level. The impact of these moves can have a wide range of effects on different sectors, including housing, health, and education. In light of this, the Penn Wharton Public Policy Initiative held a conference to try and reinforce the importance of data-centric decision making. Dennis Culhane is a Professor at Penn’s School of Social Policy and Practice, Co-Principal Investigator for the Actionable Intelligence for Social Policy, and Director of Research for the National Center for Homelessness among Veterans. He joins Dan Loney on Knowledge@Wharton radio to discuss.
Dennis Culhane: Good morning. Thank you.
Knowledge@Wharton: So in terms of this lack of efficient data usage, how frequent of a problem do you think it is right now?
Culhane: Well, the government collects reams and tons of data on a daily basis, mostly as a part of transactions from different programs that they fund and operate. But they’re not usually using this data, except for the standard audit and compliance related procedures.
Culhane: So it represents a really undervalued, underused asset on its own, but the power grows exponentially when you are able to combine this data across different agencies, linking them at the individual or a family level, so you can get a much bigger, richer, and deeper picture.
Knowledge@Wharton: As I said, potential impact crosses so many different sectors such as (a) saving lives (b) saving finance – and those are just two of the top areas.
Culhane: Absolutely. Probably a third of the total federal budget is social spending of this sort, and maybe even a little more than that. And so there’s a tremendous amount of opportunity there because of the amount of resources being spent. Furthermore, it’s an opportunity where many efficiencies, better outcomes and better programs can still be established.
Knowledge@Wharton: Let’s look at this more specifically for a second. When you think about health care, a sector with social programs in which we as a country spend a tremendous amount of money, what percentage of that could be saved or moved to other areas according to data studies?
Culhane: So every state now realizes that when it comes to Medicaid, as an example, 65% of the total expenditures are attributable to just 5% of the clients. So a very small proportion of the insured population accounts for two-thirds of the expenditure. And a significant part of that is not really primarily health-related, but it’s social factors-related. These are people who have complex and vulnerable lives such as the homeless. They can be people who are in and out of jail frequently. They can be people who are coming out of long-term incarceration, and these folks have lots of social needs that the health care system ends up having to deal with, even though they could be better and more affordably addressed through other social programs.
Knowledge@Wharton: The idea that you brought forward in the conference involves the idea of an integrated data system. And it’s not necessarily an IT project but rather just the usage of data that is in front of you in the best and most effective ways possible. Correct?
Culhane: That’s right. All of these agencies collect data, but they don’t share it with each other. There are a lot of legal barriers – there’s a lot of perceived legal barriers, I should say – because although there are federal laws that restrict certain sharing of information, it has always been permissible to share data for evaluation purposes, for planning purposes, for what are considered sort of usual business – “routine business practices.”
So what we’re doing is helping states overcome and resolve these perceived legal barriers.
Knowledge@Wharton So you were helping tear down some of those walls and build better trans-state and trans-agency relationships to resolve some of the aforementioned issues.
Culhane: Yes, ultimately this is a social problem, not a data one. The data is there, but it’s the humans, the people, the policies and procedures of the agencies that stand in the way of its proper use. Effectively, if there’s a desire to go forward, and there’s political leadership, you can put the team together and a state can get up and running with the more evidence-informed, integrated data system in about 18 months.
Knowledge@Wharton: Some of the barriers to getting these things done are policies, or perceived policies, that have been in place for probably several decades now, correct?
Culhane: Oh, yes. The technology was also always perceived to be a barrier, as well. It was never as easy as it is now to share data in a secure way. Because no agency wants their data to get redisclosed, we’ve come to a place where there’s actually a lot of security and anonymization of data that protects it in a much stronger way than ever before.
Knowledge@Wharton: But you alluded to the fact that that there has been at least conversation of trying to eliminate some of these barriers at the federal level. Paul Ryan has been involved in a conference, as well – correct?
Culhane: So there was a recent evidence-based policy commission that was formed through federal legislation, sponsored by Paul Ryan and Patty Murray.
Culhane: And that commission – The Evidence-Based Policy Commission – concluded their work in September of last year, and they put out a report with various recommendations. The focus of the report was to essentially figure out how to unlock all of this data so it could be leveraged and used to inform the evidence base for social programs.
And one of the recommendations was the creation of a federal data enclave, a center where federal agency records could be linked together in a secure way and analyzed to inform policy. So that’s one of the key recommendations that’s on the table right now.
Knowledge@Wharton: So being able to have one location where all this data is housed, and every federal organization can be able to get in securely, take what they need, come back out, and probably be able to connect with other federal agencies would cure some of these issues.
Culhane: Well, the data does not come out. The data would go in, and the analysis is done inside the secure enclave. Because what agencies fear is that their losing control of their data. So that is one of the key provisions. And at a state and local level, that’s what we’re also doing. We work with these state governments and country governments to build their own versions of these enclaves.
Knowledge@Wharton: We are joined here in studio by Dennis Culhane of the University of Pennsylvania and the School of Social Policy and Practice.
One state level issue and application you have brought up is Massachusetts homelessness. Could you expand on this?
Culhane: Oh, yes. The homelessness issue and the opioid crisis – they’re pretty common. All across the country, states are struggling with this. And it’s a high-priority issue. Take the opioid crisis for a minute as a public health emergency; although there is a lot of data that states would need to try and be more proactive about identifying solutions, this data is not readily accessible and can’t be sort of deployed for an emergency purpose. But Massachusetts is one of nine states that recently undertook a major effort to link their data to better inform responses to the opioid crisis. And it was one of the motivators for Massachusetts to get involved in building an integrated data system, because it took them nine months just to construct the legal agreement for an emergency project.
Knowledge@Wharton: Which is not that long at all, when you think about the process of putting an agreement like that together.
Culhane: Right, but if they wanted to do it again next year, they’d have to go through the process all over again, because they didn’t create a systematic approach to that. So that’s really what an integrated data system is. It creates a systematic process, where a request can come in and can be processed within 90 days, and the data agreement is executed, and the data put aside and analyzed. So you want to reduce this usual year to year-and-a-half-long time down to 90 days, if you can.
Knowledge@Wharton: How many states have done, or are at least considering this type of an approach right now?
Culhane: There are about eight states that currently have one of these integrated data systems, and we’re in the process of bringing in another thirteen states. So about 50% of the U.S. population will reside in a state where there’s an integrated data system by the end of 2019.
Knowledge@Wharton: Are you able to start and gauge what the impact is in those early states, and what you hope the impact will be when those other states come on board?
Culhane: There have been a couple of states that have been doing this in excess of 30 years.
Culhane: So there is a playbook, if you will – there is a system built to scale that we can look at and say, “This is what it looks like – a mature system that’s 30 years out there.”
Knowledge@Wharton: For example?
Culhane: The state of Washington is one, and the state of South Carolina. Those are the two that have been around the longest, and both of them employ 50 to 60 full-time employees. So they have staff with economists, demographers, social scientists – some with domain knowledge and education, health care, et cetera. And they do literally dozens of projects every year that inform a smarter state government.
They have on their websites many, many documents showing the publication of their work.
Knowledge@Wharton: So then why is it that if you’ve seen a positive impact from Washington and South Carolina over the last few decades, that other states don’t follow suit?
Culhane: Well, first if you’re a lawyer or a general counsel at an agency and someone says, “We have to share data” the easiest thing is to say is “No, we can’t do it. We can’t do it safely.”
Because to actually say “yes”– to commit to having to get involved in the process means having to write an agreement, go through the negotiations. It’s time consuming, and it adds to the workload of people who are already pretty busy.
So unless there’s a big priority put on it by senior leadership, it tends to get put to the side. And one year, two years can pass, and then new people come in, and the priority is lost. So it’s hard to sustain the momentum sometimes.
Knowledge@Wharton: But as you said a little while ago, there are laws in the books – federal laws – about data sharing. And I think the perception of a lot of people is they don’t exists in the first place. But you dispute that with the actual proof.
Culhane: So the two laws most people might have heard of are HIPAA and FERPA. So HIPAA relates to health care information, and FERPA to education data. Both of them have provisions for the sharing of these records for routine business activities. So that can include evaluation and research and planning and things of this nature. However, there is a provison that states data can be shared securely but cannot redisclose any identifying information about individuals.
Culhane: At the end of the day we have to produce aggregate statistical. And so yes, under both FERPA and HIPAA, it is permissible to do this. And there has been clear guidance, by the way. The Department of Education put out a new guidance document just last explaining to states and local governments how they can do this legally and comply with FERPA. And that sort of settled a lot of questions that were out there.
Knowledge@Wharton: Dennis Culhane of the University of Pennsylvania, joining us. We’re talking about the recent conference that they did about evidence-based data being shared by organizations.
So considering advances in technology and the demonstrated empirical success of other states, is the expectation, then, that we will have a majority of the other states on board as well?
Culhane: So the first goal is for these states to be able to just share information within agency to agency – because that doesn’t happen, yet.
Culhane: So that’s the first thing that people have to accomplish and see what they can learn from all of that, because there’s a tremendous amount that’s there. Once you see that process is in motion, there will be more procedures for external researchers to be able to come in and propose projects that are of high value. In the more advanced states we already see that they have relationships to academic researchers, and the faculty from business schools and social science programs to stimulate collaborative research that has the potential to inform the policy-making process.
Knowledge@Wharton: What about state-to-state sharing of data? When you think about it from a regional perspective – something that may be impacting the Pacific Northwest. Obviously you mentioned Washington State. What about the sharing of data with Oregon or Idaho or states in that area?
Culhane: So in law enforcement already, there’s a lot of sharing of data, mostly through the FBI. That’s for a different, immediate, operational purpose.
Culhane: The type of data which is most commonly being shared is in regards to earnings records, because states need to know – just from a tax collections point of view – if residents in one state are working in a bordering state, for example.
Culhane: So there are already agreements in regional areas for states to share data, but we’re increasingly seeing interest in doing comparative studies across these states.
Culhane: And the platforms that are being created for these integrated data systems should enable that.
Knowledge@Wharton: So would it possible to anticipate the impact of these studies by comparing the similarities of one state with its neighbors? Similarities x, y, and z may basically result in the same impact across adjacent state boarders, and thus you may be able to solve the problem between the two states.
Culhane: Sure. And every state, even if they can say, “Oh, Washington State has established that there’s this pattern,” every state still wants to be able to do that with their own data, because they have to still figure out exactly where they’re going to target these programs to change that or to improve upon a situation.
Culhane: So you can’t just accept another state’s analysis as proof of how you’re going to go forward. There still needs to be sort of the planning phase of it, and then sort of an iterative approach between the agencies to try and triangulate these populations that you’re focusing on; figuring out ways to identify them and intervene.
Knowledge@Wharton: We mentioned Massachusetts, but in terms of the issue of low income housing in general, how is that area being impacted by some of these now integrated data decisions?
Culhane: Well, affordable housing is a very big problem, especially on the West Coast. The homelessness numbers around the country have been declining for the last seven or eight years, except on the West Coast. And the problem just seems to be getting worse. There are all these encampments, and it’s polling as the number one issue among the voters, because they just see it as getting entrenched. And it’s a complex mix of populations. About 40% of the people are actually coming out of institutions that the states and the local governments administer – jails, prisons, treatment centers, hospitals, psychiatric hospitals, et cetera. So there are points of contact with the state and county that are actually quite expensive. A lot of taxpayer money is being spent. And then if people are not really being effectively managed or discharged – their discharge is not being managed properly, and they’re not getting into housing placements, then they end up recycling back through that system. And a significant proportion of the net costs of these systems is attributable to people who are not getting adequate discharge services and housing.
Knowledge@Wharton: You mentioned county, and I wanted to touch on that because when you think about the resources available, obviously the federal government has a massive amount of resources – even though there still are mistakes being made and data is not being shared as often as it probably should. However, on the State level there are arguably fewer resources. In many cases, at the county level, they don’t even consider this option, because the resources aren’t there, haven’t been there, and there’s not even an expectation that they would be there. So the potential change at the country level – in many cases at the ground floor seemingly could be very important.
Culhane: Oh, the county level is crucial, because most states in the United States are administered – they are state governed, but they are administered at a county level. And the contracts that are let out, because most of these services are administered under contract to different organizations, and those are all the contracts between counties and the local provider organizations. And believe it or not, the contract represents the major management opportunity to set performance benchmarks, to engage in a conversation about what is the expectation, and how do we go about improving the outcomes from these programs and improving the efficiencies of these programs? So you’re correct. Where the rubber meets the road is where the counties go under contract with these providers. And that’s where the money gets spent and the services delivered.
Knowledge@Wharton: So doesn’t that mean that you have to have greater communication between the state and the county or even, at times, probably the federal and the county – to make sure that the management of said programs is going well, and to be able to get that data so that you can really parse what’s going well and what’s not?
Culhane: Yes, and the counties, believe it or not, actually usually have the data. They collect the data with their providers – often through their providers. And the data go to the state because they’re transmitted by the counties to the state.
Culhane: So actually, most counties don’t need to get the data from the state, although it happens in some cases, in some agencies. Mostly they just have to get permission to engage in this work, which is usually obtainable because ultimately the owner of the data might be the state – even if it’s administered by the county.
Knowledge@Wharton: What is the hope, then, with the work that Paul Ryan and Patty Murray have been doing, in terms of trying to eliminate some of these walls that have been put up there? You mentioned that they have done their report, which was September of last year, I think you said. Obviously, there’s a time to parse through it and actually get it to go. Is it the hope that in the next year or two years, that we’ll be able to see an effective change with a lot of this communication?
Culhane: Well, there is pending legislation that they’re hoping to push through that would create this data enclave. Part of that legislation puts in place an intent that agencies have to share their data, unless they have a very firm legal reason for not doing so. So they’re trying to make the valence, if you will, positive and less risk-averse for sharing data. But the federal process is going to take some time, no matter what.
State and local governments can move much more quickly because they don’t have the same level of bureaucracy. State leadership – governors, commissioners, and secretaries of state agencies have tremendous authority to make things happen. And so there’s more nimble and faster development efforts going on at the state and local level.
Knowledge@Wharton: A lot of people say that about a variety of things, not just these areas where data is concerned, but getting other projects done, as well – that you can get done faster and probably at times more effectively at the state and local level than you can at the federal level.
Culhane: And that’s all true. One important asset that the federal government will have is that they have certain data sets, like the earnings data, which is probably the most important data set that many of these integrated data systems want access to. Because in all these programs, ultimately you want to see that families and young people are achieving – they’re getting jobs, they’re paying taxes. And so virtually all of the social programs have, as part of their goal, self-sufficiency and people achieving. Well, you can’t measure that unless you can link it up with earnings data. And state-specific earnings data have several limitations. One which we were just discussing is that people who work in an adjacent state – you might not see – that would not show up, necessarily.
Knowledge@Wharton: Sure. Yes.
Culhane: But the other two important ones are federal employees and military. They are not in the state earnings records. And the only way you would pick those up is if you went to the federal government – a version of the IRS data, essentially.
Culhane: So if states and counties could have a standard procedure for porting over data sets that they’re working on to evaluate and be linked to the federal earnings data in a federal data enclave, such as the one inside the Census Bureau then that would be hugely important and significant, because that would give people a much fuller picture of what the employment outcomes are.
The other big federal – it’s not federal, it’s privately held, but it’s national in reach – is the National Student Clearinghouse, which is all college attendance and achievement.
Knowledge@Wharton: Yes, right.
Culhane: So these are the kinds of natioanl data sets that could be of tremendous use in evaluating these programs.
Knowledge@Wharton: Great seeing you again.
Culhane: Thank you for having me.
Knowledge@Wharton: Congratulations on the work.
Knowledge@Wharton: Thank you.