This paper gives an insider's perspective on the management approaches used to manage the 2010 Census during its operational phase. The approaches used, the challenges faced (in particular, difficulties faced in automating data collection), and the solutions applied to meet those challenges are described. Finally, six management lessons learned are presented.
-
Lessons for Targeted Program Evaluation: A Personal and Professional History of the Survey of Program Dynamics
August 2007
Working Paper Number:
CES-07-24
The Survey of Program Dynamics (SPD) was created by the 1996 welfare reform legislation to facilitate its evaluation. This paper describes the evolution of that survey, discusses its implementation, and draws lessons for future evaluation. Large-scale surveys can be an important part of a portfolio of evaluation methods, but sufficient time must be given to data collection agencies if a high-quality longitudinal survey is expected. Such a survey must have both internal (agency) and external (policy analyst) buy-in. Investments in data analysis by agency staff, downplayed in favor of larger sample sizes given a fixed budget, could have contributed to more external acceptance. More attention up-front to reducing the potentially deleterious effects of attrition in longitudinal surveys, such as through the use of monetary incentives, might have been worthwhile. Given the problems encountered by the Census Bureau in producing the SPD, I argue that ongoing multi-purpose longitudinal surveys like the Survey of Income and Program Participation are potentially more valuable than episodic special-purpose surveys.
View Full
Paper PDF
-
The Management and Organizational Practices Survey (MOPS): Cognitive Testing*
January 2016
Working Paper Number:
CES-16-53
All Census Bureau surveys must meet quality standards before they can be sent to the public for data collection. This paper outlines the pretesting process that was used to ensure that the Management and Organizational Practices Survey (MOPS) met those standards. The MOPS is the first large survey of management practices at U.S. manufacturing establishments. The first wave of the MOPS, issued for reference year 2010, was subject to internal expert review and two rounds of cognitive interviews. The results of this pretesting were used to make significant changes to the MOPS instrument and ensure that quality data was collected. The second wave of the MOPS, featuring new questions on data in decision making (DDD) and uncertainty and issued for reference year 2015, was subject to two rounds of cognitive interviews and a round of usability testing. This paper illustrates the effort undertaken by the Census Bureau to ensure that all surveys released into the field are of high quality and provides insight into how respondents interpret the MOPS questionnaire for those looking to utilize the MOPS data.
View Full
Paper PDF
-
Some Open Questions on Multiple-Source Extensions of Adaptive-Survey Design Concepts and Methods
February 2023
Working Paper Number:
CES-23-03
Adaptive survey design is a framework for making data-driven decisions about survey data collection operations. This paper discusses open questions related to the extension of adaptive principles and capabilities when capturing data from multiple data sources. Here, the concept of 'design' encompasses the focused allocation of resources required for the production of high-quality statistical information in a sustainable and cost-effective way. This conceptual framework leads to a discussion of six groups of issues including: (i) the goals for improvement through adaptation; (ii) the design features that are available for adaptation; (iii) the auxiliary data that may be available for informing adaptation; (iv) the decision rules that could guide adaptation; (v) the necessary systems to operationalize adaptation; and (vi) the quality, cost, and risk profiles of the proposed adaptations (and how to evaluate them). A multiple data source environment creates significant opportunities, but also introduces complexities that are a challenge in the production of high-quality statistical information.
View Full
Paper PDF
-
Measuring the Impact of COVID-19 on Businesses and People: Lessons from the Census Bureau's Experience
January 2021
Working Paper Number:
CES-21-02
We provide an overview of Census Bureau activities to enhance the consistency, timeliness, and relevance of our data products in response to the COVID-19 pandemic. We highlight new data products designed to provide timely and granular information on the pandemic's impact: the Small Business Pulse Survey, weekly Business Formation Statistics, the Household Pulse Survey, and Community Resilience Estimates. We describe pandemic-related content introduced to existing surveys such as the Annual Business Survey and the Current Population Survey. We discuss adaptations to ensure the continuity and consistency of existing data products such as principal economic indicators and the American Community Survey.
View Full
Paper PDF
-
2010 American Community Survey Match Study
July 2014
Working Paper Number:
carra-2014-03
Using administrative records data from federal government agencies and commercial sources, the 2010 ACS Match Study measures administrative records coverage of 2010 ACS addresses, persons, and persons at addresses at different levels of geography as well as by demographic characteristics and response mode. The 2010 ACS Match Study represents a continuation of the research undertaken in the 2010 Census Match Study, the first national-level evaluation of administrative records data coverage. Preliminary results indicate that administrative records provide substantial coverage for addresses and persons in the 2010 ACS (92.7 and 92.1 percent respectively), and less extensive though substantial coverage, for person-address pairs (74.3 percent). In addition, some variation in address, person and/or person-address coverage is found across demographic and response mode groups. This research informs future uses of administrative records in survey and decennial census operations to address the increasing costs of data collection and declining response rates.
View Full
Paper PDF
-
The Impact of Household Surveys on 2020 Census Self-Response
July 2022
Working Paper Number:
CES-22-24
Households who were sampled in 2019 for the American Community Survey (ACS) had lower self-response rates to the 2020 Census. The magnitude varied from -1.5 percentage point for household sampled in January 2019 to -15.1 percent point for households sampled in December 2019. Similar effects are found for the Current Population Survey (CPS) as well.
View Full
Paper PDF
-
Developing Content for the
Management and Organizational Practices Survey-Hospitals (MOPS-HP)
September 2021
Working Paper Number:
CES-21-25
Nationally representative U.S. hospital data does not exist on management practices, which have been shown to be related to both clinical and financial performance using past data collected in the World Management Survey (WMS). This paper describes the U.S. Census Bureau's development of content for the Management and Organizational Practices Survey Hospitals (MOPS-HP) that is similar to data collected in the MOPS conducted for the manufacturing sector in 2010 and 2015 and the 2009 WMS. Findings from cognitive testing interviews with 18 chief nursing officers and 13 chief financial officers at 30 different hospitals across 7 states and the District of Columbia led to using industry-tested terminology, to confirming chief nursing officers as MOPS-HP respondents and their ability to provide recall data, and to eliminating questions that tested poorly. Hospital data collected in the MOPS-HP would be the first nationally representative data on management practices with queries on clinical key performance indicators, financial and hospital-wide patient care goals, addressing patient care problems, clinical team interactions and staffing, standardized clinical protocols, and incentives for medical record documentation. The MOPS-HP's purpose is not to collect COVID-19 pandemic information; however, data measuring hospital management practices prior to and during the COVID-19 pandemic are a byproduct of the survey's one-year recall period (2019 and 2020).
View Full
Paper PDF
-
Disclosure Avoidance Techniques Used for the 1970 through 2010 Decennial Censuses of Population and Housing
November 2018
Working Paper Number:
CES-18-47
The U.S. Census Bureau conducts the decennial censuses under Title 13 of the U. S. Code with the Section 9 mandate to not 'use the information furnished under the provisions of this title for any purpose other than the statistical purposes for which it is supplied; or make any publication whereby the data furnished by any particular establishment or individual under this title can be identified; or permit anyone other than the sworn officers and employees of the Department or bureau or agency thereof to examine the individual reports (13 U.S.C. ' 9 (2007)).' The Census Bureau applies disclosure avoidance techniques to its publicly released statistical products in order to protect the confidentiality of its respondents and their data.
View Full
Paper PDF
-
Effects of a Government-Academic Partnership: Has the NSF-Census Bureau Research Network Helped Improve the U.S. Statistical System?
January 2017
Authors:
Lars Vilhuber,
John M. Abowd,
Daniel Weinberg,
Jerome P. Reiter,
Matthew D. Shapiro,
Robert F. Belli,
Noel Cressie,
David C. Folch,
Scott H. Holan,
Margaret C. Levenstein,
Kristen M. Olson,
Jolene Smyth,
Leen-Kiat Soh,
Bruce D. Spencer,
Seth E. Spielman,
Christopher K. Wikle
Working Paper Number:
CES-17-59R
The National Science Foundation-Census Bureau Research Network (NCRN) was established in 2011 to create interdisciplinary research nodes on methodological questions of interest and significance to the broader research community and to the Federal Statistical System (FSS), particularly the Census Bureau. The activities to date have covered both fundamental and applied statistical research and have focused at least in part on the training of current and future generations of researchers in skills of relevance to surveys and alternative measurement of economic units, households, and persons. This paper discusses some of the key research findings of the eight nodes, organized into six topics: (1) Improving census and survey data collection methods; (2) Using alternative sources of data; (3) Protecting privacy and confidentiality by improving disclosure avoidance; (4) Using spatial and spatio-temporal statistical modeling to improve estimates; (5) Assessing data cost and quality tradeoffs; and (6) Combining information from multiple sources. It also reports on collaborations across nodes and with federal agencies, new software developed, and educational activities and outcomes. The paper concludes with an evaluation of the ability of the FSS to apply the NCRN's research outcomes and suggests some next steps, as well as the implications of this research-network model for future federal government renewal initiatives.
View Full
Paper PDF
-
Predictive Analytics and Organizational Architecture:
Plant-Level Evidence from Census Data
January 2019
Working Paper Number:
CES-19-02
We examine trends in the use of predictive analytics for a sample of more than 25,000 manufacturing plants using proprietary data from the US Census Bureau. Comparing 2010 and 2015, we find that use of predictive analytics has increased markedly, with the greatest use in younger plants, professionally-managed firms, more educated workforces, and stable industries. Decisions on data to be gathered originate from headquarters and are associated with less delegation of decision-making and more widespread awareness of quantitative targets among plant employees. Performance targets become more accurate, long-term oriented, and linked to company-wide performance, and management incentives strengthen, both in terms of monetary bonuses and career outcomes. Plants increasing predictive analytics become more efficient, with lower inventory, increased volume of shipments, narrower product mix, reduced management payroll and increased use of flexible and temporary employees. Results are robust to a specification based on increased government demand for data.
View Full
Paper PDF