Incorporating Administrative Data in Survey Weights for the Basic Monthly Current Population Survey
January 2024
Working Paper Number:
CES-24-02
Abstract
Response rates to the Current Population Survey (CPS) have declined over time, raising the potential for nonresponse bias in key population statistics. A potential solution is to leverage administrative data from government agencies and third-party data providers when constructing survey weights. In this paper, we take two approaches. First, we use administrative data to build a non-parametric nonresponse adjustment step while leaving the calibration to population estimates unchanged. Second, we use administratively linked data in the calibration process, matching income data from the Internal Return Service and state agencies, demographic data from the Social Security Administration and the decennial census, and industry data from the Census Bureau's Business Register to both responding and nonresponding households. We use the matched data in the household nonresponse adjustment of the CPS weighting algorithm, which changes the weights of respondents to account for differential nonresponse rates among subpopulations.
After running the experimental weighting algorithm, we compare estimates of the unemployment rate and labor force participation rate between the experimental weights and the production weights. Before March 2020, estimates of the labor force participation rates using the experimental weights are 0.2 percentage points higher than the original estimates, with minimal effect on unemployment rate. After March 2020, the new labor force participation rates are similar, but the unemployment rate is about 0.2 percentage points higher in some months during the height of COVID-related interviewing restrictions. These results are suggestive that if there is any nonresponse bias present in the CPS, the magnitude is comparable to the typical margin of error of the unemployment rate estimate. Additionally, the results are overall similar across demographic groups and states, as well as using alternative weighting methodology. Finally, we discuss how our estimates compare to those from earlier papers that calculate estimates of bias in key CPS labor force statistics.
This paper is for research purposes only. No changes to production are being implemented at this time.
Document Tags and Keywords
Keywords
Keywords are automatically generated using KeyBERT, a powerful and innovative
keyword extraction tool that utilizes BERT embeddings to ensure high-quality and contextually relevant
keywords.
By analyzing the content of working papers, KeyBERT identifies terms and phrases that capture the essence of the
text, highlighting the most significant topics and trends. This approach not only enhances searchability but
provides connections that go beyond potentially domain-specific author-defined keywords.
:
information census,
statistical,
data census,
census data,
survey,
agency,
respondent,
estimator,
population,
socioeconomic,
census bureau,
sampling,
use census,
census employment,
unemployed,
survey income,
assessed,
income data,
population survey,
propensity
Tags
Tags are automatically generated using a pretrained language model from spaCy, which excels at
several tasks, including entity tagging.
The model is able to label words and phrases by part-of-speech,
including "organizations." By filtering for frequent words and phrases labeled as "organizations", papers are
identified to contain references to specific institutions, datasets, and other organizations.
:
Metropolitan Statistical Area,
Standard Statistical Establishment List,
Internal Revenue Service,
Bureau of Labor Statistics,
Social Security Administration,
Service Annual Survey,
New England County Metropolitan,
Current Population Survey,
Decennial Census,
Housing and Urban Development,
Employer Identification Numbers,
Social Security,
Department of Housing and Urban Development,
American Community Survey,
Longitudinal Employer Household Dynamics,
Census Bureau Business Register,
Master Beneficiary Record,
Disability Insurance,
Protected Identification Key,
Computer Assisted Personal Interview,
W-2,
Quarterly Census of Employment and Wages,
Master Address File,
Census Bureau Disclosure Review Board,
2010 Census,
Person Validation System,
Supplemental Nutrition Assistance Program,
Census Bureau Person Identification Validation System,
Social Science Research Institute,
Adjusted Gross Income,
MAF-ARF
Similar Working Papers
Similarity between working papers are determined by an unsupervised neural
network model
know as Doc2Vec.
Doc2Vec is a model that represents entire documents as fixed-length vectors, allowing for the
capture of semantic meaning in a way that relates to the context of words within the document. The model learns to
associate a unique vector with each document while simultaneously learning word vectors, enabling tasks such as
document classification, clustering, and similarity detection by preserving the order and structure of words. The
document vectors are compared using cosine similarity/distance to determine the most similar working papers.
Papers identified with 🔥 are in the top 20% of similarity.
The 10 most similar working papers to the working paper 'Incorporating Administrative Data in Survey Weights for the Basic Monthly Current Population Survey' are listed below in order of similarity.
-
Working PaperIncorporating Administrative Data in Survey Weights for the 2018-2022 Survey of Income and Program Participation🔥
October 2024
Working Paper Number:
CES-24-58
Response rates to the Survey of Income and Program Participation (SIPP) have declined over time, raising the potential for nonresponse bias in survey estimates. A potential solution is to leverage administrative data from government agencies and third-party data providers when constructing survey weights. In this paper, we modify various parts of the SIPP weighting algorithm to incorporate such data. We create these new weights for the 2018 through 2022 SIPP panels and examine how the new weights affect survey estimates. Our results show that before weighting adjustments, SIPP respondents in these panels have higher socioeconomic status than the general population. Existing weighting procedures reduce many of these differences. Comparing SIPP estimates between the production weights and the administrative data-based weights yields changes that are not uniform across the joint income and program participation distribution. Unlike other Census Bureau household surveys, there is no large increase in nonresponse bias in SIPP due to the COVID-19 Pandemic. In summary, the magnitude and sign of nonresponse bias in SIPP is complicated, and the existing weighting procedures may change the sign of nonresponse bias for households with certain incomes and program benefit statuses.View Full Paper PDF
-
Working PaperNonresponse and Coverage Bias in the Household Pulse Survey: Evidence from Administrative Data🔥
October 2024
Working Paper Number:
CES-24-60
The Household Pulse Survey (HPS) conducted by the U.S. Census Bureau is a unique survey that provided timely data on the effects of the COVID-19 Pandemic on American households and continues to provide data on other emergent social and economic issues. Because the survey has a response rate in the single digits and only has an online response mode, there are concerns about nonresponse and coverage bias. In this paper, we match administrative data from government agencies and third-party data to HPS respondents to examine how representative they are of the U.S. population. For comparison, we create a benchmark of American Community Survey (ACS) respondents and nonrespondents and include the ACS respondents as another point of reference. Overall, we find that the HPS is less representative of the U.S. population than the ACS. However, performance varies across administrative variables, and the existing weighting adjustments appear to greatly improve the representativeness of the HPS. Additionally, we look at household characteristics by their email domain to examine the effects on coverage from limiting email messages in 2023 to addresses from the contact frame with at least 90% deliverability rates, finding no clear change in the representativeness of the HPS afterwards.View Full Paper PDF
-
Working PaperThe Impact of Household Surveys on 2020 Census Self-Response🔥
July 2022
Working Paper Number:
CES-22-24
Households who were sampled in 2019 for the American Community Survey (ACS) had lower self-response rates to the 2020 Census. The magnitude varied from -1.5 percentage point for household sampled in January 2019 to -15.1 percent point for households sampled in December 2019. Similar effects are found for the Current Population Survey (CPS) as well.View Full Paper PDF
-
Working PaperConnected and Uncooperative: The Effects of Homogenous and Exclusive Social Networks on Survey Response Rates and Nonresponse Bias
January 2024
Working Paper Number:
CES-24-01
Social capital, the strength of people's friendship networks and community ties, has been hypothesized as an important determinant of survey participation. Investigating this hypothesis has been difficult given data constraints. In this paper, we provide insights by investigating how response rates and nonresponse bias in the American Community Survey are correlated with county-level social network data from Facebook. We find that areas of the United States where people have more exclusive and homogenous social networks have higher nonresponse bias and lower response rates. These results provide further evidence that the effects of social capital may not be simply a matter of whether people are socially isolated or not, but also what types of social connections people have and the sociodemographic heterogeneity of their social networks.View Full Paper PDF
-
Working PaperNational Experimental Wellbeing Statistics - Version 1
February 2023
Working Paper Number:
CES-23-04
This is the U.S. Census Bureau's first release of the National Experimental Wellbeing Statistics (NEWS) project. The NEWS project aims to produce the best possible estimates of income and poverty given all available survey and administrative data. We link survey, decennial census, administrative, and third-party data to address measurement error in income and poverty statistics. We estimate improved (pre-tax money) income and poverty statistics for 2018 by addressing several possible sources of bias documented in prior research. We address biases from 1) unit nonresponse through improved weights, 2) missing income information in both survey and administrative data through improved imputation, and 3) misreporting by combining or replacing survey responses with administrative information. Reducing survey error substantially affects key measures of well-being: We estimate median household income is 6.3 percent higher than in survey estimates, and poverty is 1.1 percentage points lower. These changes are driven by subpopulations for which survey error is particularly relevant. For house holders aged 65 and over, median household income is 27.3 percent higher and poverty is 3.3 percentage points lower than in survey estimates. We do not find a significant impact on median household income for householders under 65 or on child poverty. Finally, we discuss plans for future releases: addressing other potential sources of bias, releasing additional years of statistics, extending the income concepts measured, and including smaller geographies such as state and county.View Full Paper PDF
-
Working PaperUnderstanding Earnings Instability: How Important are Employment Fluctuations and Job Changes?
August 2009
Working Paper Number:
CES-09-20
Using three panel datasets (the matched CPS, the SIPP, and the newly available Longitudinal Employment and Household Dynamics (LEHD) data), we examine trends in male earnings instability in recent decades. In contrast to several papers that find a recent upward trend in earnings instability using the PSID data, we find that earnings instability has been remarkably stable in the 1990s and the 2000s. We find that job changing rates remained relatively constant casting doubt on the importance of labor market 'churning.' We find some evidence that earnings instability increased among job stayers which lends credence to the view that greater reliance on incentive pay increased instability of worker pay. We also find an offsetting decrease in earnings instability among job changers due largely to declining unemployment associated with job changes. One caveat to our findings is that we focus on men who have positive earnings in two adjacent years and thus ignore men who exit the labor force or re-enter after an extended period. Preliminary investigation suggests that ignoring these transitions understates the rise in earnings instability over the past two decades.View Full Paper PDF
-
Working PaperReceipt of Public and Private Food Assistance Across the Rural-Urban Continuum Before and During the COVID-19 Pandemic: Analysis of Current Population Survey Data
August 2025
Working Paper Number:
CES-25-51
Background: The nutrition safety net in the United States is critical to supporting food security among households in need. Food assistance in the United States includes both government-funded food programs and private community-based providers who distribute food to in need households. The COVID-19 pandemic impacted experiences of food security and use of private and public food assistance resources. However, this may have differed for households residing in urban versus rural areas. We explored receipt of Supplemental Nutrition Assistance Program (SNAP) benefits or food from community-based emergency food providers across a detailed measure of the rural-urban continuum before and during the COVID-19 pandemic. Methods: We linked restricted use Current Population Survey Food Security Supplement data to census-tract level United States Department of Agriculture Rural-Urban Commuting Area codes to estimate prevalence of self-reported SNAP participation and receipt of emergency food support across temporal (2015-2019 versus 2020-2021) and socio-spatial (urban, large rural city/town, small rural town, or isolated rural town/area) dimensions. We report prevalences as point estimates with 95% confidence intervals, all weighted for national representation. Results: The weighted prevalence of self-reported SNAP participation was 8.9% (8.7-9.2%) in 2015-2019 and 9.1% (8.5-9.5%) in 2020-2021 in urban areas, 11.4% (10.8-12.2%) in 2015-2019 and 11.6% (10.5-12.9%) in 2020-2021 in large rural towns/cities, 13.4% (12.3-14.6%) in 2015-2019 and 12.3% (10.5-14.5%) in 2020-2021 in small rural towns, and 9.7% (8.6-10.9%) in 2015-2019 and 10.9% (8.8-13.4% )in 2020-2021 isolated rural towns. The weighted prevalence of self-reported receipt of emergency food was 4.9% (4.8-5.1%) in 2015-2019 and 6.2% (5.8-6.5%) in 2020-2021 in urban areas, 6.8% (6.2-7.4%) in 2015-2019 and 7.6% (6.6-8.6%) in 2020-2021 in large rural towns/cities, 8.1% (7.3-9.1%) in 2015-2019 and 7.1% (5.7-8.8%) in 2020-2021 in small rural towns, and 6.8% (5.9-7.7%) in 2015-2019 and 8.5% (6.7-10.6%) in 2020-2021 isolated rural towns. Conclusion: Households in rural communities use public and private food assistance at higher rates than urban areas, but there is variation across communities depending on the level of rurality.View Full Paper PDF
-
Working PaperInvestigating the Use of Administrative Records in the Consumer Expenditure Survey
March 2018
Working Paper Number:
carra-2018-01
In this paper, we investigate the potential of applying administrative records income data to the Consumer Expenditure (CE) survey to inform measurement error properties of CE estimates, supplement respondent-collected data, and estimate the representativeness of the CE survey by income level. We match individual responses to Consumer Expenditure Quarterly Interview Survey data collected from July 2013 through December 2014 to IRS administrative data in order to analyze CE questions on wages, social security payroll deductions, self-employment income receipt and retirement income. We find that while wage amounts are largely in alignment between the CE and administrative records in the middle of the wage distribution, there is evidence that wages are over-reported to the CE at the bottom of the wage distribution and under-reported at the top of the wage distribution. We find mixed evidence for alignment between the CE and administrative records on questions covering payroll deductions and self-employment income receipt, but find substantial divergence between CE responses and administrative records when examining retirement income. In addition to the analysis using person-based linkages, we also match responding and non-responding CE sample units to the universe of IRS 1040 tax returns by address to examine non-response bias. We find that non-responding households are substantially richer than responding households, and that very high income households are less likely to respond to the CE.View Full Paper PDF
-
Working PaperMeasuring the Impact of COVID-19 on Businesses and People: Lessons from the Census Bureau's Experience
January 2021
Working Paper Number:
CES-21-02
We provide an overview of Census Bureau activities to enhance the consistency, timeliness, and relevance of our data products in response to the COVID-19 pandemic. We highlight new data products designed to provide timely and granular information on the pandemic's impact: the Small Business Pulse Survey, weekly Business Formation Statistics, the Household Pulse Survey, and Community Resilience Estimates. We describe pandemic-related content introduced to existing surveys such as the Annual Business Survey and the Current Population Survey. We discuss adaptations to ensure the continuity and consistency of existing data products such as principal economic indicators and the American Community Survey.View Full Paper PDF
-
Working PaperThe Design of Sampling Strata for the National Household Food Acquisition and Purchase Survey
February 2025
Working Paper Number:
CES-25-13
The National Household Food Acquisition and Purchase Survey (FoodAPS), sponsored by the United States Department of Agriculture's (USDA) Economic Research Service (ERS) and Food and Nutrition Service (FNS), examines the food purchasing behavior of various subgroups of the U.S. population. These subgroups include participants in the Supplemental Nutrition Assistance Program (SNAP) and the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), as well as households who are eligible for but don't participate in these programs. Participants in these social protection programs constitute small proportions of the U.S. population; obtaining an adequate number of such participants in a survey would be challenging absent stratified sampling to target SNAP and WIC participating households. This document describes how the U.S. Census Bureau (which is planning to conduct future versions of the FoodAPS survey on behalf of USDA) created sampling strata to flag the FoodAPS targeted subpopulations using machine learning applications in linked survey and administrative data. We describe the data, modeling techniques, and how well the sampling flags target low-income households and households receiving WIC and SNAP benefits. We additionally situate these efforts in the nascent literature on the use of big data and machine learning for the improvement of survey efficiency.View Full Paper PDF
