Using Linked Survey and Administrative Data to Better Measure Income: Implications for Poverty, Program Effectiveness and Holes in the Safety Net
October 2015
Working Paper Number:
CES-15-35
Abstract
We examine the consequences of underreporting of transfer programs in household survey data for several prototypical analyses of low-income populations. We focus on the Current Population Survey (CPS), the source of official poverty and inequality statistics, but provide evidence that our qualitative conclusions are likely to apply to other surveys. We link administrative data for food stamps, TANF, General Assistance, and subsidized housing from New York State to the CPS at the individual level. Program receipt in the CPS is missed for over one-third of housing assistance recipients, 40 percent of food stamp recipients and 60 percent of TANF and General Assistance recipients. Dollars of benefits are also undercounted for reporting recipients, particularly for TANF, General Assistance and housing assistance. We find that the survey data sharply understate the income of poor households, as conjectured in past work by one of the authors. Underreporting in the survey data also greatly understates the effects of anti-poverty programs and changes our understanding of program targeting, often making it seem that welfare programs are less targeted to both the very poorest and middle income households than they are. Using the combined data rather than survey data alone, the poverty reducing effect of all programs together is nearly doubled while the effect of housing assistance is tripled. We also re-examine the coverage of the safety net, specifically the share of people without work or program receipt. Using the administrative measures of program receipt rather than the survey ones often reduces the share of single mothers falling through the safety net by one-half or more.
Document Tags and Keywords
Keywords
Keywords are automatically generated using KeyBERT, a powerful and innovative
keyword extraction tool that utilizes BERT embeddings to ensure high-quality and contextually relevant
keywords.
By analyzing the content of working papers, KeyBERT identifies terms and phrases that capture the essence of the
text, highlighting the most significant topics and trends. This approach not only enhances searchability but
provides connections that go beyond potentially domain-specific author-defined keywords.
:
survey,
recession,
subsidy,
disadvantaged,
population,
welfare,
housing,
poor,
poverty,
medicaid,
family,
income households,
subsidized
Tags
Tags are automatically generated using a pretrained language model from spaCy, which excels at
several tasks, including entity tagging.
The model is able to label words and phrases by part-of-speech,
including "organizations." By filtering for frequent words and phrases labeled as "organizations", papers are
identified to contain references to specific institutions, datasets, and other organizations.
:
Federal Reserve Bank,
University of Chicago,
Current Population Survey,
Chicago Census Research Data Center,
Housing and Urban Development,
Social Security,
Department of Housing and Urban Development,
American Community Survey,
Protected Identification Key,
National Opinion Research Center,
Earned Income Tax Credit,
American Housing Survey,
Special Sworn Status,
Temporary Assistance for Needy Families,
Social and Economic Supplement,
Person Validation System,
Person Identification Validation System
Similar Working Papers
Similarity between working papers are determined by an unsupervised neural
network model
know as Doc2Vec.
Doc2Vec is a model that represents entire documents as fixed-length vectors, allowing for the
capture of semantic meaning in a way that relates to the context of words within the document. The model learns to
associate a unique vector with each document while simultaneously learning word vectors, enabling tasks such as
document classification, clustering, and similarity detection by preserving the order and structure of words. The
document vectors are compared using cosine similarity/distance to determine the most similar working papers.
Papers identified with 🔥 are in the top 20% of similarity.
The 10 most similar working papers to the working paper 'Using Linked Survey and Administrative Data to Better Measure Income: Implications for Poverty, Program Effectiveness and Holes in the Safety Net' are listed below in order of similarity.
-
Working PaperErrors in Survey Reporting and Imputation and Their Effects on Estimates of Food Stamp Program Participation🔥
April 2011
Working Paper Number:
CES-11-14
Benefit receipt in major household surveys is often underreported. This misreporting leads to biased estimates of the economic circumstances of disadvantaged populations, program takeup, and the distributional effects of government programs, and other program effects. We use administrative data on Food Stamp Program (FSP) participation matched to American Community Survey (ACS) and Current Population Survey (CPS) household data. We show that nearly thirty-five percent of true recipient households do not report receipt in the ACS and fifty percent do not report receipt in the CPS. Misreporting, both false negatives and false positives, varies with individual characteristics, leading to complicated biases in FSP analyses. We then directly examine the determinants of program receipt using our combined administrative and survey data. The combined data allow us to examine accurate participation using individual characteristics missing in administrative data. Our results differ from conventional estimates using only survey data, as such estimates understate participation by single parents, non-whites, low income households, and other groups. To evaluate the use of Census Bureau imputed ACS and CPS data, we also examine whether our estimates using survey data alone are closer to those using the accurate combined data when imputed survey observations are excluded. Interestingly, excluding the imputed observations leads to worse ACS estimates, but has less effect on the CPS estimates.View Full Paper PDF
-
Working PaperThe Antipoverty Impact of the EITC: New Estimates from Survey and Administrative Tax Records🔥
April 2019
Working Paper Number:
CES-19-14R
We reassess the antipoverty effects of the EITC using unique data linking the CPS Annual Social and Economic Supplement to IRS data for the same individuals spanning years 2005-2016. We compare EITC benefits from standard simulators to administrative EITC payments and find that significantly more actual EITC payments flow to childless tax units than predicted, and to those whose family income places them above official poverty thresholds. However, actual EITC payments appear to be target efficient at the tax unit level. In 2016, about 3.1 million persons were lifted out of poverty by the EITC, substantially less than prior estimates.View Full Paper PDF
-
Working PaperA METHOD OF CORRECTING FOR MISREPORTING APPLIED TO THE FOOD STAMP PROGRAM🔥
May 2013
Working Paper Number:
CES-13-28
Survey misreporting is known to be pervasive and bias common statistical analyses. In this paper, I first use administrative data on SNAP receipt and amounts linked to American Community Survey data from New York State to show that survey data can misrepresent the program in important ways. For example, more than 1.4 billion dollars received are not reported in New York State alone. 46 percent of dollars received by house- holds with annual income above the poverty line are not reported in the survey data, while only 19 percent are missing below the poverty line. Standard corrections for measurement error cannot remove these biases. I then develop a method to obtain consistent estimates by combining parameter estimates from the linked data with publicly available data. This conditional density method recovers the correct estimates using public use data only, which solves the problem that access to linked administrative data is usually restricted. I examine the degree to which this approach can be used to extrapolate across time and geography, in order to solve the problem that validation data is often based on a convenience sample. I present evidence from within New York State that the extent of heterogeneity is small enough to make extrapolation work well across both time and geography. Extrapolation to the entire U.S. yields substantive differences to survey data and reduces deviations from official aggregates by a factor of 4 to 9 compared to survey aggregates.View Full Paper PDF
-
Working PaperMeasuring Income of the Aged in Household Surveys: Evidence from Linked Administrative Records🔥
June 2024
Working Paper Number:
CES-24-32
Research has shown that household survey estimates of retirement income (defined benefit pensions and defined contribution account withdrawals) suffer from substantial underreporting which biases downward measures of financial well-being among the aged. Using data from both the redesigned 2016 Current Population Survey Annual Social and Economic Supplement (CPS ASEC) and the Health and Retirement Study (HRS), each matched with administrative records, we examine to what extent underreporting of retirement income affects key statistics such as reliance on Social Security benefits and poverty among the aged. We find that underreporting of retirement income is still prevalent in the CPS ASEC. While the HRS does a better job than the CPS ASEC in terms of capturing retirement income, it still falls considerably short compared to administrative records. Consequently, the relative importance of Social Security income remains overstated in household surveys'53 percent of elderly beneficiaries in the CPS ASEC and 49 percent in the HRS rely on Social Security for the majority of their incomes compared to 42 percent in the linked administrative data. The poverty rate for those aged 65 and over is also overstated'8.8 percent in the CPS ASEC and 7.4 percent in the HRS compared to 6.4 percent in the linked administrative data. Our results illustrate the effects of using alternative data sources in producing key statistics from the Social Security Administration's Income of the Aged publication.View Full Paper PDF
-
Working PaperAlternative Measures of Income Poverty and the Anti-Poverty Effects of Taxes and Transfers
June 2005
Working Paper Number:
CES-05-08
The Census Bureau prepared a number of alternative income-based measures of poverty to illustrate the distributional impacts of several alternatives to the official measure. The paper examines five income variants for two different units of analysis (families and households) for two different assumptions about inflation (the historical Consumer Price Index and a 'Research Series' alternative that uses current methods) for two different sets of thresholds (official and a formula-based alternative base on three parameters). The poverty rate effects are analyzed for the total population, the distributional effects are analyzed using poverty shares, and the anti-poverty effects of taxes and transfers are analyzed using a percentage reduction in poverty rates. Suggestions for future research are included.View Full Paper PDF
-
Working PaperMeasuring School Economic Disadvantage
November 2022
Working Paper Number:
CES-22-50R
Many educational policies hinge on the valid measurement of student economic disadvantage at the school level. Measures based on free and reduced-price lunch enrollment are used widely. However, recent research raises questions about their reliability, particularly following the introduction of universal free lunch in certain schools and districts. Using unique data linking the universe of students in Oregon public schools to IRS tax records and other data housed at the U.S. Census Bureau, we provide the first examination of how well different measures capture school economic disadvantage. We find that, in Oregon, direct certification provides the best widely-available measure, both over time and across the distribution of school economic disadvantage. By contrast, neighborhood-based measures consistently perform relatively poorly.View Full Paper PDF
-
Working PaperEarnings Through the Stages: Using Tax Data to Test for Sources of Error in CPS ASEC Earnings and Inequality Measures
September 2024
Working Paper Number:
CES-24-52
In this paper, I explore the impact of generalized coverage error, item non-response bias, and measurement error on measures of earnings and earnings inequality in the CPS ASEC. I match addresses selected for the CPS ASEC to administrative data from 1040 tax returns. I then compare earnings statistics in the tax data for wage and salary earnings in samples corresponding to seven stages of the CPS ASEC survey production process. I also compare the statistics using the actual survey responses. The statistics I examine include mean earnings, the Gini coefficient, percentile earnings shares, and shares of the survey weight for a range of percentiles. I examine how the accuracy of the statistics calculated using the survey data is affected by including imputed responses for both those who did not respond to the full CPS ASEC and those who did not respond to the earnings question. I find that generalized coverage error and item nonresponse bias are dominated by measurement error, and that an important aspect of measurement error is households reporting no wage and salary earnings in the CPS ASEC when there are such earnings in the tax data. I find that the CPS ASEC sample misses earnings at the high end of the distribution from the initial selection stage and that the final survey weights exacerbate this.View Full Paper PDF
-
Working PaperTrends in the Relative Household Income of Working-Age Men with Work Limitations: Correcting the Record Using Internal Current Population Survey Data
March 2008
Working Paper Number:
CES-08-05
Previous research measuring the economic well-being of working-age men with work limitations relative to such men without work limitations in the public use March Current Population Survey (CPS) systematically understates the mean household income of both groups; overstates the relative household income of those with work limitations; and understates the decline in their relative household income over time. Using the internal March CPS, we demonstrate this by creating a cell mean series beginning in 1975 that provides the mean reported income of all topcoded persons for each source of income in the public use March CPS data. Using our cell mean series with the public use March CPS, we closely match the yearly mean income of working-age men with and without work limitations over the period 1987-2004 in the internal data and show that this match is superior to ones using alternative methods of correcting for topcoding currently used in the disability literature. We then provide levels and trends in the relative income of working-age men with work limitations from 1980-2006, the earliest year in the March CPS that such comparisons can be made.View Full Paper PDF
-
Working PaperIncorporating Administrative Data in Survey Weights for the 2018-2022 Survey of Income and Program Participation
October 2024
Working Paper Number:
CES-24-58
Response rates to the Survey of Income and Program Participation (SIPP) have declined over time, raising the potential for nonresponse bias in survey estimates. A potential solution is to leverage administrative data from government agencies and third-party data providers when constructing survey weights. In this paper, we modify various parts of the SIPP weighting algorithm to incorporate such data. We create these new weights for the 2018 through 2022 SIPP panels and examine how the new weights affect survey estimates. Our results show that before weighting adjustments, SIPP respondents in these panels have higher socioeconomic status than the general population. Existing weighting procedures reduce many of these differences. Comparing SIPP estimates between the production weights and the administrative data-based weights yields changes that are not uniform across the joint income and program participation distribution. Unlike other Census Bureau household surveys, there is no large increase in nonresponse bias in SIPP due to the COVID-19 Pandemic. In summary, the magnitude and sign of nonresponse bias in SIPP is complicated, and the existing weighting procedures may change the sign of nonresponse bias for households with certain incomes and program benefit statuses.View Full Paper PDF
-
Working PaperThe Measurement of Medicaid Coverage in the SIPP: Evidence from California, 1990-1996
September 2002
Working Paper Number:
CES-02-21
This paper studies the accuracy of reported Medicaid coverage in the Survey of Income and Program Participation (SIPP) using a unique data set formed by matching SIPP survey responses to administrative records from the State of California. Overall, we estimate that the SIPP underestimates Medicaid coverage in the California populaton by about 10 percent. Among SIPP respondents who can be matched to administrative records, we estimate that the probability someone reports Medicaid coverage in a month when they are actually covered is around 85 percent. The corresponding probability for low-income children is even higher ' at least 90 percent. These estimates suggest that the SIPP provides reasonably accurate coverage reports for those who are actually in the Medicaid system. On the other hand, our estimate of the false positive rate (the rate of reported coverage for those who are not covered in the administrative records) is relatively high: 2.5 percent for the sample as a whole, and up to 20 percent for poor children. Some of this is due to errors in the recording of Social Security numbers in the administrative system, rather than to problems in the SIPP.View Full Paper PDF
