The Icing on the Cake: The Effects of Monetary Incentives on Income Data Quality in the SIPP
January 2024
Working Paper Number:
CES-24-03
Abstract
Accurate measurement of key income variables plays a crucial role in economic research and policy decision-making. However, the presence of item nonresponse and measurement error in survey data can cause biased estimates. These biases can subsequently lead to sub-optimal policy decisions and inefficient allocation of resources. While there have been various studies documenting item nonresponse and measurement error in economic data, there have not been many studies investigating interventions that could reduce item nonresponse and measurement error. In our research, we investigate the impact of monetary incentives on reducing item nonresponse and measurement error for labor and investment income in the Survey of Income and Program Participation (SIPP). Our study utilizes a randomized incentive experiment in Waves 1 and 2 of the 2014 SIPP, which allows us to assess the effectiveness of incentives in reducing item nonresponse and measurement error. We find that households receiving incentives had item nonresponse rates that are 1.3 percentage points lower for earnings and 1.5 percentage points lower for Social Security income. Measurement error was 6.31 percentage points lower at the intensive margin for interest income, and 16.48 percentage points lower for dividend income compared to non-incentive recipient households. These findings provide valuable insights for data producers and users and highlight the importance of implementing strategies to improve data quality in economic research.
Document Tags and Keywords
Keywords
Keywords are automatically generated using KeyBERT, a powerful and innovative
keyword extraction tool that utilizes BERT embeddings to ensure high-quality and contextually relevant
keywords.
By analyzing the content of working papers, KeyBERT identifies terms and phrases that capture the essence of the
text, highlighting the most significant topics and trends. This approach not only enhances searchability but
provides connections that go beyond potentially domain-specific author-defined keywords.
:
exogeneity,
endogeneity,
estimating,
survey,
respondent,
earnings,
incentive,
consumption,
bias,
indicator,
policy,
irs,
income survey,
survey income,
assessing,
taxpayer
Tags
Tags are automatically generated using a pretrained language model from spaCy, which excels at
several tasks, including entity tagging.
The model is able to label words and phrases by part-of-speech,
including "organizations." By filtering for frequent words and phrases labeled as "organizations", papers are
identified to contain references to specific institutions, datasets, and other organizations.
:
Internal Revenue Service,
Social Security Administration,
Current Population Survey,
Survey of Income and Program Participation,
Social Security,
Detailed Earnings Records,
Master Beneficiary Record,
Disability Insurance,
Protected Identification Key,
Master Earnings File,
Census Bureau Disclosure Review Board,
ASEC,
Person Validation System,
Supplemental Nutrition Assistance Program,
Census Bureau Person Identification Validation System
Similar Working Papers
Similarity between working papers are determined by an unsupervised neural
network model
know as Doc2Vec.
Doc2Vec is a model that represents entire documents as fixed-length vectors, allowing for the
capture of semantic meaning in a way that relates to the context of words within the document. The model learns to
associate a unique vector with each document while simultaneously learning word vectors, enabling tasks such as
document classification, clustering, and similarity detection by preserving the order and structure of words. The
document vectors are compared using cosine similarity/distance to determine the most similar working papers.
Papers identified with 🔥 are in the top 20% of similarity.
The 10 most similar working papers to the working paper 'The Icing on the Cake: The Effects of Monetary Incentives on Income Data Quality in the SIPP' are listed below in order of similarity.
-
Working PaperSelf-Employment Income Reporting on Surveys
April 2023
Working Paper Number:
CES-23-19
We examine the relation between administrative income data and survey reports for self-employed and wage-earning respondents from 2000 - 2015. The self-employed report 40 percent more wages and self-employment income in the survey than in tax administrative records; this estimate nets out differences between these two sources that are also shared by wage-earners. We provide evidence that differential reporting incentives are an important explanation of the larger self-employed gap by exploiting a well-known artifact ' self-employed respondents exhibit substantial bunching at the first EITC kink in their administrative records. We do not observe the same behavior in their survey responses even after accounting for survey measurement concerns.View Full Paper PDF
-
Working PaperCriminal court fees, earnings, and expenditures: A multi-state RD analysis of survey and administrative data
February 2023
Working Paper Number:
CES-23-06
Millions of people in the United States face fines and fees in the criminal court system each year, totaling over $27 billion in overall criminal debt to-date. In this study, we leverage five distinct natural experiments in Florida, Michigan, North Carolina, Texas, and Wisconsin using regression discontinuity designs to evaluate the causal impact of such financial sanctions and user fees. We consider a range of long-term outcomes including employment, recidivism, household expenditures, and other self-reported measures of well-being, which we measure through a combination of administrative records on earnings and employment, the Criminal Justice Administrative Records System, and household surveys. We find consistent evidence across the range of natural experiments and subgroup analyses of precise null effects on the population, ruling out long-run impacts larger than +/-3.6% on total earnings and +/-4.7% on total recidivism. Failure to find changes in outcomes undermines popular narratives of poverty traps arising from criminal debt but argues against the use of fines and fees as a source of local revenue and as a crime control tool.View Full Paper PDF
-
Working PaperAn Analysis of Sample Selection and the Reliability of Using Short-term Earnings Averages in SIPP-SSA Matched Data
December 2011
Working Paper Number:
CES-11-39
In this paper, we document the extent to which the sample of the Survey of Income and Program Participation that is matched to the Social Security Administration's administrative earnings records is nationally representative. We conclude that the match bias is small, so selection is not a serious concern. The matched sample over-represents individuals who are wealthy, who have financial assets or who have received a government-transfer and under-represents individuals who attrited from the SIPP. We use this matched sample to examine the relationship between short-term averages of earnings from the SIPP earnings and average lifetime earnings from the administrative records. Our estimates suggest that using short averages of earnings may understate the effects of permanent income on particular outcomes of interest.View Full Paper PDF
-
Working PaperThe Measurement of Medicaid Coverage in the SIPP: Evidence from California, 1990-1996
September 2002
Working Paper Number:
CES-02-21
This paper studies the accuracy of reported Medicaid coverage in the Survey of Income and Program Participation (SIPP) using a unique data set formed by matching SIPP survey responses to administrative records from the State of California. Overall, we estimate that the SIPP underestimates Medicaid coverage in the California populaton by about 10 percent. Among SIPP respondents who can be matched to administrative records, we estimate that the probability someone reports Medicaid coverage in a month when they are actually covered is around 85 percent. The corresponding probability for low-income children is even higher ' at least 90 percent. These estimates suggest that the SIPP provides reasonably accurate coverage reports for those who are actually in the Medicaid system. On the other hand, our estimate of the false positive rate (the rate of reported coverage for those who are not covered in the administrative records) is relatively high: 2.5 percent for the sample as a whole, and up to 20 percent for poor children. Some of this is due to errors in the recording of Social Security numbers in the administrative system, rather than to problems in the SIPP.View Full Paper PDF
-
Working PaperUsing Linked Survey and Administrative Data to Better Measure Income: Implications for Poverty, Program Effectiveness and Holes in the Safety Net
October 2015
Working Paper Number:
CES-15-35
We examine the consequences of underreporting of transfer programs in household survey data for several prototypical analyses of low-income populations. We focus on the Current Population Survey (CPS), the source of official poverty and inequality statistics, but provide evidence that our qualitative conclusions are likely to apply to other surveys. We link administrative data for food stamps, TANF, General Assistance, and subsidized housing from New York State to the CPS at the individual level. Program receipt in the CPS is missed for over one-third of housing assistance recipients, 40 percent of food stamp recipients and 60 percent of TANF and General Assistance recipients. Dollars of benefits are also undercounted for reporting recipients, particularly for TANF, General Assistance and housing assistance. We find that the survey data sharply understate the income of poor households, as conjectured in past work by one of the authors. Underreporting in the survey data also greatly understates the effects of anti-poverty programs and changes our understanding of program targeting, often making it seem that welfare programs are less targeted to both the very poorest and middle income households than they are. Using the combined data rather than survey data alone, the poverty reducing effect of all programs together is nearly doubled while the effect of housing assistance is tripled. We also re-examine the coverage of the safety net, specifically the share of people without work or program receipt. Using the administrative measures of program receipt rather than the survey ones often reduces the share of single mothers falling through the safety net by one-half or more.View Full Paper PDF
-
Working PaperSpillovers From Costly Credit
March 2013
Working Paper Number:
CES-13-11
Recent research on the effects of credit access among low- and moderate-income households finds that high-cost payday loans exacerbate, rather than alleviate, financial distress for a subset of borrowers (Melzer 2011; Skiba and Tobacman 2011). In this study I find that others, outside the borrowing household, bear a portion of these costs too: households with payday loan access are 20% more likely to use food assistance benefits and 10% less likely to make child support payments required of non-resident parents. These findings suggest that as borrowers accommodate interest and principal payments on payday loan debt, they prioritize loan payments over other liabilities like child support payments and they turn to transfer programs like food stamps to supplement the household's resources. To establish this finding, the analysis uses a measure of payday loan access that is robust to the concern that lender location decisions and state policies governing payday lending are endogenous relative to household financial condition. The analysis also confirms that the effect is absent in the mid-1990s, prior to the spread of payday lending, and that the effect grows over time, in parallel with the growth of payday lending.View Full Paper PDF
-
Working PaperDifferences in Disability Insurance Allowance Rates
August 2025
Working Paper Number:
CES-25-54
Allowance rates for disability insurance applications vary by race and ethnicity, but it is unclear to what extent these differences are artifacts of other differing socio-economic and health characteristics, or selection issues in SSA's race and ethnicity data. This paper uses the 2015 American Community Survey linked to 2015-2019 SSA administrative data to investigate DI application allowance rates among non-Hispanic White, non-Hispanic Black, non-Hispanic Asian, non-Hispanic American Indian/Alaska Native, and Hispanic applicants aged 25-65. The analysis uses regression, propensity score matching, and inverse probability weighting to estimate differences in allowance rates among applicants who are similar on observable characteristics. Relative to raw comparisons, differences by race and ethnicity in multivariate analyses are substantially smaller in magnitude and are generally not statistically significant.View Full Paper PDF
-
Working PaperThe Antipoverty Impact of the EITC: New Estimates from Survey and Administrative Tax Records
April 2019
Working Paper Number:
CES-19-14R
We reassess the antipoverty effects of the EITC using unique data linking the CPS Annual Social and Economic Supplement to IRS data for the same individuals spanning years 2005-2016. We compare EITC benefits from standard simulators to administrative EITC payments and find that significantly more actual EITC payments flow to childless tax units than predicted, and to those whose family income places them above official poverty thresholds. However, actual EITC payments appear to be target efficient at the tax unit level. In 2016, about 3.1 million persons were lifted out of poverty by the EITC, substantially less than prior estimates.View Full Paper PDF
-
Working PaperIncorporating Administrative Data in Survey Weights for the 2018-2022 Survey of Income and Program Participation
October 2024
Working Paper Number:
CES-24-58
Response rates to the Survey of Income and Program Participation (SIPP) have declined over time, raising the potential for nonresponse bias in survey estimates. A potential solution is to leverage administrative data from government agencies and third-party data providers when constructing survey weights. In this paper, we modify various parts of the SIPP weighting algorithm to incorporate such data. We create these new weights for the 2018 through 2022 SIPP panels and examine how the new weights affect survey estimates. Our results show that before weighting adjustments, SIPP respondents in these panels have higher socioeconomic status than the general population. Existing weighting procedures reduce many of these differences. Comparing SIPP estimates between the production weights and the administrative data-based weights yields changes that are not uniform across the joint income and program participation distribution. Unlike other Census Bureau household surveys, there is no large increase in nonresponse bias in SIPP due to the COVID-19 Pandemic. In summary, the magnitude and sign of nonresponse bias in SIPP is complicated, and the existing weighting procedures may change the sign of nonresponse bias for households with certain incomes and program benefit statuses.View Full Paper PDF
-
Working PaperNon-Random Assignment of Individual Identifiers and Selection into Linked Data: Implications for Research
January 2026
Working Paper Number:
CES-26-06
The U.S. Census Bureau's Person Identification Validation System facilitates anonymous linkages between survey and administrative records by assigning Protected Identification Keys (PIKs) to person records. While PIK assignment is generally accurate, some person records are not successfully assigned a PIK, which can lead to sample selection bias in analyses of linked data. Using the American Community Survey (ACS) and the Current Population Survey Annual Social and Economic Supplement (CPS ASEC) between 2005 and 2022, we corroborate and extend existing findings on the drivers of PIK assignment, showing that the rate of PIK assignment varies widely across socio-demographic subgroups. Using earnings as a test case, we then show that limiting a survey sample of wage earners to person records with PIKs or successful linkages to W-2 wage records tends to overestimate self-reported wage earnings, on average, indicative of linkage-induced selection bias. In a validation exercise, we demonstrate that reweighting methods, such as inverse probability weighting or entropy balancing, can mitigate this bias.View Full Paper PDF
