-
Nonresponse and Coverage Bias in the Household Pulse Survey: Evidence from Administrative Data
October 2024
Working Paper Number:
CES-24-60
The Household Pulse Survey (HPS) conducted by the U.S. Census Bureau is a unique survey that provided timely data on the effects of the COVID-19 Pandemic on American households and continues to provide data on other emergent social and economic issues. Because the survey has a response rate in the single digits and only has an online response mode, there are concerns about nonresponse and coverage bias. In this paper, we match administrative data from government agencies and third-party data to HPS respondents to examine how representative they are of the U.S. population. For comparison, we create a benchmark of American Community Survey (ACS) respondents and nonrespondents and include the ACS respondents as another point of reference. Overall, we find that the HPS is less representative of the U.S. population than the ACS. However, performance varies across administrative variables, and the existing weighting adjustments appear to greatly improve the representativeness of the HPS. Additionally, we look at household characteristics by their email domain to examine the effects on coverage from limiting email messages in 2023 to addresses from the contact frame with at least 90% deliverability rates, finding no clear change in the representativeness of the HPS afterwards.
View Full
Paper PDF
-
Incorporating Administrative Data in Survey Weights for the 2018-2022 Survey of Income and Program Participation
October 2024
Working Paper Number:
CES-24-58
Response rates to the Survey of Income and Program Participation (SIPP) have declined over time, raising the potential for nonresponse bias in survey estimates. A potential solution is to leverage administrative data from government agencies and third-party data providers when constructing survey weights. In this paper, we modify various parts of the SIPP weighting algorithm to incorporate such data. We create these new weights for the 2018 through 2022 SIPP panels and examine how the new weights affect survey estimates. Our results show that before weighting adjustments, SIPP respondents in these panels have higher socioeconomic status than the general population. Existing weighting procedures reduce many of these differences. Comparing SIPP estimates between the production weights and the administrative data-based weights yields changes that are not uniform across the joint income and program participation distribution. Unlike other Census Bureau household surveys, there is no large increase in nonresponse bias in SIPP due to the COVID-19 Pandemic. In summary, the magnitude and sign of nonresponse bias in SIPP is complicated, and the existing weighting procedures may change the sign of nonresponse bias for households with certain incomes and program benefit statuses.
View Full
Paper PDF
-
The Icing on the Cake: The Effects of Monetary Incentives on Income Data Quality in the SIPP
January 2024
Working Paper Number:
CES-24-03
Accurate measurement of key income variables plays a crucial role in economic research and policy decision-making. However, the presence of item nonresponse and measurement error in survey data can cause biased estimates. These biases can subsequently lead to sub-optimal policy decisions and inefficient allocation of resources. While there have been various studies documenting item nonresponse and measurement error in economic data, there have not been many studies investigating interventions that could reduce item nonresponse and measurement error. In our research, we investigate the impact of monetary incentives on reducing item nonresponse and measurement error for labor and investment income in the Survey of Income and Program Participation (SIPP). Our study utilizes a randomized incentive experiment in Waves 1 and 2 of the 2014 SIPP, which allows us to assess the effectiveness of incentives in reducing item nonresponse and measurement error. We find that households receiving incentives had item nonresponse rates that are 1.3 percentage points lower for earnings and 1.5 percentage points lower for Social Security income. Measurement error was 6.31 percentage points lower at the intensive margin for interest income, and 16.48 percentage points lower for dividend income compared to non-incentive recipient households. These findings provide valuable insights for data producers and users and highlight the importance of implementing strategies to improve data quality in economic research.
View Full
Paper PDF
-
Incorporating Administrative Data in Survey Weights for the Basic Monthly Current Population Survey
January 2024
Working Paper Number:
CES-24-02
Response rates to the Current Population Survey (CPS) have declined over time, raising the potential for nonresponse bias in key population statistics. A potential solution is to leverage administrative data from government agencies and third-party data providers when constructing survey weights. In this paper, we take two approaches. First, we use administrative data to build a non-parametric nonresponse adjustment step while leaving the calibration to population estimates unchanged. Second, we use administratively linked data in the calibration process, matching income data from the Internal Return Service and state agencies, demographic data from the Social Security Administration and the decennial census, and industry data from the Census Bureau's Business Register to both responding and nonresponding households. We use the matched data in the household nonresponse adjustment of the CPS weighting algorithm, which changes the weights of respondents to account for differential nonresponse rates among subpopulations.
After running the experimental weighting algorithm, we compare estimates of the unemployment rate and labor force participation rate between the experimental weights and the production weights. Before March 2020, estimates of the labor force participation rates using the experimental weights are 0.2 percentage points higher than the original estimates, with minimal effect on unemployment rate. After March 2020, the new labor force participation rates are similar, but the unemployment rate is about 0.2 percentage points higher in some months during the height of COVID-related interviewing restrictions. These results are suggestive that if there is any nonresponse bias present in the CPS, the magnitude is comparable to the typical margin of error of the unemployment rate estimate. Additionally, the results are overall similar across demographic groups and states, as well as using alternative weighting methodology. Finally, we discuss how our estimates compare to those from earlier papers that calculate estimates of bias in key CPS labor force statistics.
This paper is for research purposes only. No changes to production are being implemented at this time.
View Full
Paper PDF
-
Producing U.S. Population Statistics Using Multiple Administrative Sources
November 2023
Working Paper Number:
CES-23-58
We identify several challenges encountered when constructing U.S. administrative record-based (AR-based) population estimates for 2020. Though the AR estimates are higher than the 2020 Census at the national level, they are over 15 percent lower in 5 percent of counties, suggesting that locational accuracy can be improved. Other challenges include how to achieve comprehensive coverage, maintain consistent coverage across time, filter out nonresidents and people not alive on the reference date, uncover missing links across person and address records, and predict demographic characteristics when multiple ones are reported or when they are missing. We discuss several ways of addressing these issues, e.g., building in redundancy with more sources, linking children to their parents' addresses, and conducting additional record linkage for people without Social Security Numbers and for addresses not initially linked to the Census Bureau's Master Address File. We discuss modeling to predict lower levels of geography for people lacking those geocodes, the probability that a person is a U.S. resident on the reference date, the probability that an address is the person's residence on the reference date, and the probability a person is in each demographic characteristic category. Regression results illustrate how many of these challenges and solutions affect the AR county population estimates.
View Full
Paper PDF
-
The Demographics of the Recipients of the First Economic Impact Payment
May 2023
Working Paper Number:
CES-23-24
Starting in April 2020, the federal government began to distribute Economic Impact Payments (EIPs) in response to the health and economic crisis caused by COVID-19. More than 160 million payments were disbursed. We produce statistics concerning the receipt of EIPs by individuals and households across key demographic subgroups. We find that payments went out particularly quickly to households with children and lower-income households, and the rate of receipt was quite high for individuals over age 60, likely due to a coordinated effort to issue payments automatically to Social Security recipients. We disaggregate statistics by race/ethnicity to document whether racial disparities arose in EIP disbursement. Receipt rates were high overall, with limited differences across racial/ethnic subgroups. We provide a set of detailed counts in tables for use by the public.
View Full
Paper PDF
-
Estimating the U.S. Citizen Voting-Age Population (CVAP) Using Blended Survey Data, Administrative Record Data, and Modeling: Technical Report
April 2023
Authors:
J. David Brown,
Danielle H. Sandler,
Lawrence Warren,
Moises Yi,
Misty L. Heggeness,
Joseph L. Schafer,
Matthew Spence,
Marta Murray-Close,
Carl Lieberman,
Genevieve Denoeux,
Lauren Medina
Working Paper Number:
CES-23-21
This report develops a method using administrative records (AR) to fill in responses for nonresponding American Community Survey (ACS) housing units rather than adjusting survey weights to account for selection of a subset of nonresponding housing units for follow-up interviews and for nonresponse bias. The method also inserts AR and modeling in place of edits and imputations for ACS survey citizenship item nonresponses. We produce Citizen Voting-Age Population (CVAP) tabulations using this enhanced CVAP method and compare them to published estimates. The enhanced CVAP method produces a 0.74 percentage point lower citizen share, and it is 3.05 percentage points lower for voting-age Hispanics. The latter result can be partly explained by omissions of voting-age Hispanic noncitizens with unknown legal status from ACS household responses. Weight adjustments may be less effective at addressing nonresponse bias under those conditions.
View Full
Paper PDF
-
Maternal and Infant Health Inequality: New Evidence from Linked Administrative Data
November 2022
Working Paper Number:
CES-22-55
We use linked administrative data that combines the universe of California birth records, hospitalizations, and death records with parental income from Internal Revenue Service tax records and the Longitudinal Employer-Household Dynamics file to provide novel evidence on economic inequality in infant and maternal health. We find that birth outcomes vary nonmonotonically with parental income, and that children of parents in the top ventile of the income distribution have higher rates of low birth weight and preterm birth than those in the bottom ventile. However, unlike birth outcomes, infant mortality varies monotonically with income, and infants of parents in the top ventile of the income distribution---who have the worst birth outcomes---have a death rate that is half that of infants of parents in the bottom ventile. When studying maternal health, we find a similar pattern of non-monotonicity between income and severe maternal morbidity, and a monotonic and decreasing relationship between income and maternal mortality. At the same time, these disparities by parental income are small when compared to racial disparities, and we observe virtually no convergence in health outcomes across racial and ethnic groups as income rises. Indeed, infant and maternal health in Black families at the top of the income distribution is markedly worse than that of white families at the bottom of the income distribution. Lastly, we benchmark the health gradients in California to those in Sweden, finding that infant and maternal health is worse in California than in Sweden for most outcomes throughout the entire income distribution.
View Full
Paper PDF
-
EARNINGS ADJUSTMENT FRICTIONS: EVIDENCE FROM SOCIAL SECURITY EARNINGS TEST
September 2013
Working Paper Number:
CES-13-50
We study frictions in adjusting earnings to changes in the Social Security Annual Earnings Test (AET) using a panel of Social Security Administration microdata on one percent of the U.S. population from 1961 to 2006. Individuals continue to "bunch" at the convex kink the AET creates even when they are no longer subject to the AET, consistent with the existence of earnings adjustment frictions in the U.S. We develop a novel framework for estimating an earnings elasticity and an adjustment cost using information on the amount of bunching at kinks before and after policy changes in earnings incentives around the kinks. We apply this method in settings in which individuals face changes in the AET bene.t reduction rate, and we estimate in a baseline case that the earnings elasticity with respect to the implicit net-of-tax share is 0.23, and the .xed cost of adjustment is $152.08.
View Full
Paper PDF
-
Are the Lasting Effects of Employee-Employer Separations induced by Layoff and Disability Similar? Exploring Job Displacement using Survey and Administrative Data
October 2005
Working Paper Number:
tp-2005-03
This paper integrates the existing literatures on displacement and health by examining the enduring
effects of job dislocations that are induced by firm and individual shocks to employment. A joint estimation of
hourly wage rates and weekly hours illuminates the disparities in these economic outcomes
that exist between those who have reestablished themselves in the workplace subsequent to a layoff and
those who have returned to work following the onset of a disability relative to those with uninterrupted
job histories. As an extension of these ideas, employment transitions and workplace adjustments are
modeled to capture spousal reactions to these shocks. Multiple indicators of health from the Survey of
Income and Program Participation and Social Security Administrative benefits records are incorporated
into the analyses of those with impairments that prompted job loss. These measures allow knowledge
to be gleaned regarding the qualitative di'erences in the lasting impacts of job cessation resulting from
medically diagnosed illnesses as compared to estimates uncovered using survey data sources alone. By
considering time durations following these periods of separation in light of these indicators of well-being,
a more comprehensive understanding of the long-run repercussions of employee-employer separation is
acquired.
View Full
Paper PDF
