This paper gives an insider's perspective on the management approaches used to manage the 2010 Census during its operational phase. The approaches used, the challenges faced (in particular, difficulties faced in automating data collection), and the solutions applied to meet those challenges are described. Finally, six management lessons learned are presented.
-
Lessons for Targeted Program Evaluation: A Personal and Professional History of the Survey of Program Dynamics
August 2007
Working Paper Number:
CES-07-24
The Survey of Program Dynamics (SPD) was created by the 1996 welfare reform legislation to facilitate its evaluation. This paper describes the evolution of that survey, discusses its implementation, and draws lessons for future evaluation. Large-scale surveys can be an important part of a portfolio of evaluation methods, but sufficient time must be given to data collection agencies if a high-quality longitudinal survey is expected. Such a survey must have both internal (agency) and external (policy analyst) buy-in. Investments in data analysis by agency staff, downplayed in favor of larger sample sizes given a fixed budget, could have contributed to more external acceptance. More attention up-front to reducing the potentially deleterious effects of attrition in longitudinal surveys, such as through the use of monetary incentives, might have been worthwhile. Given the problems encountered by the Census Bureau in producing the SPD, I argue that ongoing multi-purpose longitudinal surveys like the Survey of Income and Program Participation are potentially more valuable than episodic special-purpose surveys.
View Full
Paper PDF
-
The Management and Organizational Practices Survey (MOPS): Cognitive Testing*
January 2016
Working Paper Number:
CES-16-53
All Census Bureau surveys must meet quality standards before they can be sent to the public for data collection. This paper outlines the pretesting process that was used to ensure that the Management and Organizational Practices Survey (MOPS) met those standards. The MOPS is the first large survey of management practices at U.S. manufacturing establishments. The first wave of the MOPS, issued for reference year 2010, was subject to internal expert review and two rounds of cognitive interviews. The results of this pretesting were used to make significant changes to the MOPS instrument and ensure that quality data was collected. The second wave of the MOPS, featuring new questions on data in decision making (DDD) and uncertainty and issued for reference year 2015, was subject to two rounds of cognitive interviews and a round of usability testing. This paper illustrates the effort undertaken by the Census Bureau to ensure that all surveys released into the field are of high quality and provides insight into how respondents interpret the MOPS questionnaire for those looking to utilize the MOPS data.
View Full
Paper PDF
-
Some Open Questions on Multiple-Source Extensions of Adaptive-Survey Design Concepts and Methods
February 2023
Working Paper Number:
CES-23-03
Adaptive survey design is a framework for making data-driven decisions about survey data collection operations. This paper discusses open questions related to the extension of adaptive principles and capabilities when capturing data from multiple data sources. Here, the concept of 'design' encompasses the focused allocation of resources required for the production of high-quality statistical information in a sustainable and cost-effective way. This conceptual framework leads to a discussion of six groups of issues including: (i) the goals for improvement through adaptation; (ii) the design features that are available for adaptation; (iii) the auxiliary data that may be available for informing adaptation; (iv) the decision rules that could guide adaptation; (v) the necessary systems to operationalize adaptation; and (vi) the quality, cost, and risk profiles of the proposed adaptations (and how to evaluate them). A multiple data source environment creates significant opportunities, but also introduces complexities that are a challenge in the production of high-quality statistical information.
View Full
Paper PDF
-
2010 American Community Survey Match Study
July 2014
Working Paper Number:
carra-2014-03
Using administrative records data from federal government agencies and commercial sources, the 2010 ACS Match Study measures administrative records coverage of 2010 ACS addresses, persons, and persons at addresses at different levels of geography as well as by demographic characteristics and response mode. The 2010 ACS Match Study represents a continuation of the research undertaken in the 2010 Census Match Study, the first national-level evaluation of administrative records data coverage. Preliminary results indicate that administrative records provide substantial coverage for addresses and persons in the 2010 ACS (92.7 and 92.1 percent respectively), and less extensive though substantial coverage, for person-address pairs (74.3 percent). In addition, some variation in address, person and/or person-address coverage is found across demographic and response mode groups. This research informs future uses of administrative records in survey and decennial census operations to address the increasing costs of data collection and declining response rates.
View Full
Paper PDF
-
The Impact of Household Surveys on 2020 Census Self-Response
July 2022
Working Paper Number:
CES-22-24
Households who were sampled in 2019 for the American Community Survey (ACS) had lower self-response rates to the 2020 Census. The magnitude varied from -1.5 percentage point for household sampled in January 2019 to -15.1 percent point for households sampled in December 2019. Similar effects are found for the Current Population Survey (CPS) as well.
View Full
Paper PDF
-
Developing Content for the
Management and Organizational Practices Survey-Hospitals (MOPS-HP)
September 2021
Working Paper Number:
CES-21-25
Nationally representative U.S. hospital data does not exist on management practices, which have been shown to be related to both clinical and financial performance using past data collected in the World Management Survey (WMS). This paper describes the U.S. Census Bureau's development of content for the Management and Organizational Practices Survey Hospitals (MOPS-HP) that is similar to data collected in the MOPS conducted for the manufacturing sector in 2010 and 2015 and the 2009 WMS. Findings from cognitive testing interviews with 18 chief nursing officers and 13 chief financial officers at 30 different hospitals across 7 states and the District of Columbia led to using industry-tested terminology, to confirming chief nursing officers as MOPS-HP respondents and their ability to provide recall data, and to eliminating questions that tested poorly. Hospital data collected in the MOPS-HP would be the first nationally representative data on management practices with queries on clinical key performance indicators, financial and hospital-wide patient care goals, addressing patient care problems, clinical team interactions and staffing, standardized clinical protocols, and incentives for medical record documentation. The MOPS-HP's purpose is not to collect COVID-19 pandemic information; however, data measuring hospital management practices prior to and during the COVID-19 pandemic are a byproduct of the survey's one-year recall period (2019 and 2020).
View Full
Paper PDF
-
Measuring the Impact of COVID-19 on Businesses and People: Lessons from the Census Bureau's Experience
January 2021
Working Paper Number:
CES-21-02
We provide an overview of Census Bureau activities to enhance the consistency, timeliness, and relevance of our data products in response to the COVID-19 pandemic. We highlight new data products designed to provide timely and granular information on the pandemic's impact: the Small Business Pulse Survey, weekly Business Formation Statistics, the Household Pulse Survey, and Community Resilience Estimates. We describe pandemic-related content introduced to existing surveys such as the Annual Business Survey and the Current Population Survey. We discuss adaptations to ensure the continuity and consistency of existing data products such as principal economic indicators and the American Community Survey.
View Full
Paper PDF
-
Evaluating the Impact of MEP Services on Establishment Performance: A Preliminary Empirical Investigation
July 2012
Working Paper Number:
CES-12-15
This work examines the impact of manufacturing extension services on establishment productivity. It builds on an earlier study conducted by Jarmin in the 1990s, by matching the Census of Manufacturers (CMF) with the Manufacturing Extension Partnership (MEP) customer and activity datasets to generate treatment and comparison groups for analysis. The scope of the study is the period 1997 to 2002, which was a period of economic downturn in the manufacturing sector and budgetary challenges for the MEP. The paper presents some preliminary findings from this analysis. Both lagged dependent variable (LDV) and difference in difference (DiD) models are employed to estimate the relationship between manufacturing extension and labor productivity. The results presented are inconclusive and paint a mixed picture as they demonstrate the benefits and limitations of using Census microdata in program evaluation. They also point to the need to conduct analyses that could help to better understand the dynamic impact of MEP services.
View Full
Paper PDF
-
The Rise of Industrial AI in America: Microfoundations of the Productivity J-curve(s)
April 2025
Working Paper Number:
CES-25-27
We examine the prevalence and productivity dynamics of artificial intelligence (AI) in American manufacturing. Working with the Census Bureau to collect detailed large-scale data for 2017 and 2021, we focus on AI-related technologies with industrial applications. We find causal evidence of J-curve-shaped returns, where short-term performance losses precede longer-term gains. Consistent with costly adjustment taking place within core production processes, industrial AI use increases work-in-progress inventory, investment in industrial robots, and labor shedding, while harming productivity and profitability in the short run. These losses are unevenly distributed, concentrating among older businesses while being mitigated by growth-oriented business strategies and within-firm spillovers. Dynamics, however, matter: earlier (pre-2017) adopters exhibit stronger growth over time, conditional on survival. Notably, among older establishments, abandonment of structured production-management practices accounts for roughly one-third of these losses, revealing a specific channel through which intangible factors shape AI's impact. Taken together, these results provide novel evidence on the microfoundations of technology J-curves, identifying mechanisms and illuminating how and why they differ across firm types. These findings extend our understanding of modern General Purpose Technologies, explaining why their economic impact'exemplified here by AI'may initially disappoint, particularly in contexts dominated by older, established firms.
View Full
Paper PDF
-
Predictive Analytics and Organizational Architecture:
Plant-Level Evidence from Census Data
January 2019
Working Paper Number:
CES-19-02
We examine trends in the use of predictive analytics for a sample of more than 25,000 manufacturing plants using proprietary data from the US Census Bureau. Comparing 2010 and 2015, we find that use of predictive analytics has increased markedly, with the greatest use in younger plants, professionally-managed firms, more educated workforces, and stable industries. Decisions on data to be gathered originate from headquarters and are associated with less delegation of decision-making and more widespread awareness of quantitative targets among plant employees. Performance targets become more accurate, long-term oriented, and linked to company-wide performance, and management incentives strengthen, both in terms of monetary bonuses and career outcomes. Plants increasing predictive analytics become more efficient, with lower inventory, increased volume of shipments, narrower product mix, reduced management payroll and increased use of flexible and temporary employees. Results are robust to a specification based on increased government demand for data.
View Full
Paper PDF