The Icing on the Cake: The Effects of Monetary Incentives on Income Data Quality in the SIPP
January 2024
Working Paper Number:
CES-24-03
Abstract
Accurate measurement of key income variables plays a crucial role in economic research and policy decision-making. However, the presence of item nonresponse and measurement error in survey data can cause biased estimates. These biases can subsequently lead to sub-optimal policy decisions and inefficient allocation of resources. While there have been various studies documenting item nonresponse and measurement error in economic data, there have not been many studies investigating interventions that could reduce item nonresponse and measurement error. In our research, we investigate the impact of monetary incentives on reducing item nonresponse and measurement error for labor and investment income in the Survey of Income and Program Participation (SIPP). Our study utilizes a randomized incentive experiment in Waves 1 and 2 of the 2014 SIPP, which allows us to assess the effectiveness of incentives in reducing item nonresponse and measurement error. We find that households receiving incentives had item nonresponse rates that are 1.3 percentage points lower for earnings and 1.5 percentage points lower for Social Security income. Measurement error was 6.31 percentage points lower at the intensive margin for interest income, and 16.48 percentage points lower for dividend income compared to non-incentive recipient households. These findings provide valuable insights for data producers and users and highlight the importance of implementing strategies to improve data quality in economic research.
Document Tags and Keywords
Keywords
Keywords are automatically generated using KeyBERT, a powerful and innovative
keyword extraction tool that utilizes BERT embeddings to ensure high-quality and contextually relevant
keywords.
By analyzing the content of working papers, KeyBERT identifies terms and phrases that capture the essence of the
text, highlighting the most significant topics and trends. This approach not only enhances searchability but
provides connections that go beyond potentially domain-specific author-defined keywords.
:
exogeneity,
endogeneity,
estimating,
survey,
respondent,
earnings,
incentive,
consumption,
bias,
indicator,
policy,
irs,
income survey,
survey income,
assessing,
taxpayer
Tags
Tags are automatically generated using a pretrained language model from spaCy, which excels at
several tasks, including entity tagging.
The model is able to label words and phrases by part-of-speech,
including "organizations." By filtering for frequent words and phrases labeled as "organizations", papers are
identified to contain references to specific institutions, datasets, and other organizations.
:
Internal Revenue Service,
Social Security Administration,
Current Population Survey,
Survey of Income and Program Participation,
Social Security,
Detailed Earnings Records,
Master Beneficiary Record,
Disability Insurance,
Protected Identification Key,
Master Earnings File,
Census Bureau Disclosure Review Board,
ASEC,
Person Validation System,
Supplemental Nutrition Assistance Program,
Census Bureau Person Identification Validation System
Similar Working Papers
Similarity between working papers are determined by an unsupervised neural
network model
know as Doc2Vec.
Doc2Vec is a model that represents entire documents as fixed-length vectors, allowing for the
capture of semantic meaning in a way that relates to the context of words within the document. The model learns to
associate a unique vector with each document while simultaneously learning word vectors, enabling tasks such as
document classification, clustering, and similarity detection by preserving the order and structure of words. The
document vectors are compared using cosine similarity/distance to determine the most similar working papers.
Papers identified with 🔥 are in the top 20% of similarity.
The 10 most similar working papers to the working paper 'The Icing on the Cake: The Effects of Monetary Incentives on Income Data Quality in the SIPP' are listed below in order of similarity.
-
Working PaperAn Analysis of Sample Selection and the Reliability of Using Short-term Earnings Averages in SIPP-SSA Matched Data
December 2011
Working Paper Number:
CES-11-39
In this paper, we document the extent to which the sample of the Survey of Income and Program Participation that is matched to the Social Security Administration's administrative earnings records is nationally representative. We conclude that the match bias is small, so selection is not a serious concern. The matched sample over-represents individuals who are wealthy, who have financial assets or who have received a government-transfer and under-represents individuals who attrited from the SIPP. We use this matched sample to examine the relationship between short-term averages of earnings from the SIPP earnings and average lifetime earnings from the administrative records. Our estimates suggest that using short averages of earnings may understate the effects of permanent income on particular outcomes of interest.View Full Paper PDF
-
Working PaperSchool Discipline and Racial Disparities in Early Adulthood
June 2021
Working Paper Number:
CES-21-14
Despite interest in the role of school discipline in the creation of racial inequality, previous research has been unable to identify how students who receive suspensions in school differ from unsuspended classmates on key young adult outcomes. We utilize novel data to document the links between high school discipline and important young adult outcomes related to criminal justice contact, social safety net program participation, post-secondary education, and the labor market. We show that the link between school discipline and young adult outcomes tends to be stronger for Black students than for White students, and that inequality in exposure to school discipline accounts for approximately 30 percent of the Black-White disparities in young adult criminal justice outcomes and SNAP receipt.View Full Paper PDF
-
Working PaperChildhood Housing and Adult Earnings: A Between-Siblings Analysis of Housing Vouchers and Public Housing
January 2013
Working Paper Number:
CES-13-48RR
To date, research on the long-term effects of childhood participation in voucher-assisted and public housing has been limited by the lack of data and suitable identification strategies. We create a national level longitudinal data set that enables us to analyze how children's housing experiences affect adult earnings and incarceration rates. While naive estimates suggest there are substantial negative consequences to childhood participation in voucher assisted and public housing, this result appears to be driven largely by selection of households into housing assistance programs. To mitigate this source of bias, we employ household fixed-effects specifications that use only within-household (across-sibling) variation for identification. Compared to naive specifications, household fixed-effects estimates for earnings are universally more positive, and they suggest that there are positive and statistically significant benefits from childhood residence in assisted housing on young adult earnings for nearly all demographic groups. Childhood participation in assisted housing also reduces the likelihood of incarceration across all household race/ethnicity groups. Time spent in voucher-assisted or public housing is especially beneficial for females from non-Hispanic Black households, who experience substantial increases in expected earnings and lower incarceration rates.View Full Paper PDF
-
Working PaperSelf-Employment Income Reporting on Surveys
April 2023
Working Paper Number:
CES-23-19
We examine the relation between administrative income data and survey reports for self-employed and wage-earning respondents from 2000 - 2015. The self-employed report 40 percent more wages and self-employment income in the survey than in tax administrative records; this estimate nets out differences between these two sources that are also shared by wage-earners. We provide evidence that differential reporting incentives are an important explanation of the larger self-employed gap by exploiting a well-known artifact ' self-employed respondents exhibit substantial bunching at the first EITC kink in their administrative records. We do not observe the same behavior in their survey responses even after accounting for survey measurement concerns.View Full Paper PDF
-
Working PaperParental Earnings and Children's Well-Being and Future Success: An Analysis of the SIPP Matched to SSA Earnings Data
April 2011
Working Paper Number:
CES-11-12
We estimate the association between parental earnings and a wide variety of indicators of child well-being using data from the Survey of Income and Program Participation (SIPP) matched to administrative earnings records from the Social Security Administration. We find that the use of longer time averages of parent earnings leads to substantially higher estimated effects compared to using only a single year of parent earnings. This suggests that previous studies may have understated the potential efficacy of income support programs to improve child well-being. Further, policy makers should take into account the attenuation bias when comparing studies that use different time spans to measure parental income. Using 7 year time averages of parent earnings, we show for example, that a doubling of parent earnings reduces the probability of a teenager reporting being in poor health by close to 50 percent and a child having insufficient food by 75 percent.View Full Paper PDF
-
Working PaperErrors in Survey Reporting and Imputation and Their Effects on Estimates of Food Stamp Program Participation
April 2011
Working Paper Number:
CES-11-14
Benefit receipt in major household surveys is often underreported. This misreporting leads to biased estimates of the economic circumstances of disadvantaged populations, program takeup, and the distributional effects of government programs, and other program effects. We use administrative data on Food Stamp Program (FSP) participation matched to American Community Survey (ACS) and Current Population Survey (CPS) household data. We show that nearly thirty-five percent of true recipient households do not report receipt in the ACS and fifty percent do not report receipt in the CPS. Misreporting, both false negatives and false positives, varies with individual characteristics, leading to complicated biases in FSP analyses. We then directly examine the determinants of program receipt using our combined administrative and survey data. The combined data allow us to examine accurate participation using individual characteristics missing in administrative data. Our results differ from conventional estimates using only survey data, as such estimates understate participation by single parents, non-whites, low income households, and other groups. To evaluate the use of Census Bureau imputed ACS and CPS data, we also examine whether our estimates using survey data alone are closer to those using the accurate combined data when imputed survey observations are excluded. Interestingly, excluding the imputed observations leads to worse ACS estimates, but has less effect on the CPS estimates.View Full Paper PDF
-
Working PaperCriminal court fees, earnings, and expenditures: A multi-state RD analysis of survey and administrative data
February 2023
Working Paper Number:
CES-23-06
Millions of people in the United States face fines and fees in the criminal court system each year, totaling over $27 billion in overall criminal debt to-date. In this study, we leverage five distinct natural experiments in Florida, Michigan, North Carolina, Texas, and Wisconsin using regression discontinuity designs to evaluate the causal impact of such financial sanctions and user fees. We consider a range of long-term outcomes including employment, recidivism, household expenditures, and other self-reported measures of well-being, which we measure through a combination of administrative records on earnings and employment, the Criminal Justice Administrative Records System, and household surveys. We find consistent evidence across the range of natural experiments and subgroup analyses of precise null effects on the population, ruling out long-run impacts larger than +/-3.6% on total earnings and +/-4.7% on total recidivism. Failure to find changes in outcomes undermines popular narratives of poverty traps arising from criminal debt but argues against the use of fines and fees as a source of local revenue and as a crime control tool.View Full Paper PDF
-
Working PaperWhen and Why Does Nonresponse Occur? Comparing the Determinants of Initial Unit Nonresponse and Panel Attrition
September 2023
Working Paper Number:
CES-23-44
Though unit nonresponse threatens data quality in both cross-sectional and panel surveys, little is understood about how initial nonresponse and later panel attrition may be theoretically or empirically distinct phenomena. This study advances current knowledge of the determinants of both unit nonresponse and panel attrition within the context of the U.S. Census Bureau's Survey of Income and Program Participation (SIPP) panel survey, which I link with high-quality federal administrative records, paradata, and geographic data. By exploiting the SIPP's interpenetrated sampling design and relying on cross-classified random effects modeling, this study quantifies the relative effects of sample household, interviewer, and place characteristics on baseline nonresponse and later attrition, addressing a critical gap in the literature. Given the reliance on successful record linkages between survey sample households and federal administrative data in the nonresponse research, this study also undertakes an explicitly spatial analysis of the place-based characteristics associated with successful record linkages in the U.S.View Full Paper PDF
-
Working PaperIncorporating Administrative Data in Survey Weights for the 2018-2022 Survey of Income and Program Participation
October 2024
Working Paper Number:
CES-24-58
Response rates to the Survey of Income and Program Participation (SIPP) have declined over time, raising the potential for nonresponse bias in survey estimates. A potential solution is to leverage administrative data from government agencies and third-party data providers when constructing survey weights. In this paper, we modify various parts of the SIPP weighting algorithm to incorporate such data. We create these new weights for the 2018 through 2022 SIPP panels and examine how the new weights affect survey estimates. Our results show that before weighting adjustments, SIPP respondents in these panels have higher socioeconomic status than the general population. Existing weighting procedures reduce many of these differences. Comparing SIPP estimates between the production weights and the administrative data-based weights yields changes that are not uniform across the joint income and program participation distribution. Unlike other Census Bureau household surveys, there is no large increase in nonresponse bias in SIPP due to the COVID-19 Pandemic. In summary, the magnitude and sign of nonresponse bias in SIPP is complicated, and the existing weighting procedures may change the sign of nonresponse bias for households with certain incomes and program benefit statuses.View Full Paper PDF
-
Working PaperInvestigating the Use of Administrative Records in the Consumer Expenditure Survey
March 2018
Working Paper Number:
carra-2018-01
In this paper, we investigate the potential of applying administrative records income data to the Consumer Expenditure (CE) survey to inform measurement error properties of CE estimates, supplement respondent-collected data, and estimate the representativeness of the CE survey by income level. We match individual responses to Consumer Expenditure Quarterly Interview Survey data collected from July 2013 through December 2014 to IRS administrative data in order to analyze CE questions on wages, social security payroll deductions, self-employment income receipt and retirement income. We find that while wage amounts are largely in alignment between the CE and administrative records in the middle of the wage distribution, there is evidence that wages are over-reported to the CE at the bottom of the wage distribution and under-reported at the top of the wage distribution. We find mixed evidence for alignment between the CE and administrative records on questions covering payroll deductions and self-employment income receipt, but find substantial divergence between CE responses and administrative records when examining retirement income. In addition to the analysis using person-based linkages, we also match responding and non-responding CE sample units to the universe of IRS 1040 tax returns by address to examine non-response bias. We find that non-responding households are substantially richer than responding households, and that very high income households are less likely to respond to the CE.View Full Paper PDF
