The Management and Organizational Practices Survey (MOPS): Cognitive Testing*
January 2016
Working Paper Number:
CES-16-53
Abstract
All Census Bureau surveys must meet quality standards before they can be sent to the public for data collection. This paper outlines the pretesting process that was used to ensure that the Management and Organizational Practices Survey (MOPS) met those standards. The MOPS is the first large survey of management practices at U.S. manufacturing establishments. The first wave of the MOPS, issued for reference year 2010, was subject to internal expert review and two rounds of cognitive interviews. The results of this pretesting were used to make significant changes to the MOPS instrument and ensure that quality data was collected. The second wave of the MOPS, featuring new questions on data in decision making (DDD) and uncertainty and issued for reference year 2015, was subject to two rounds of cognitive interviews and a round of usability testing. This paper illustrates the effort undertaken by the Census Bureau to ensure that all surveys released into the field are of high quality and provides insight into how respondents interpret the MOPS questionnaire for those looking to utilize the MOPS data.
Document Tags and Keywords
Keywords
Keywords are automatically generated using KeyBERT, a powerful and innovative
keyword extraction tool that utilizes BERT embeddings to ensure high-quality and contextually relevant
keywords.
By analyzing the content of working papers, KeyBERT identifies terms and phrases that capture the essence of the
text, highlighting the most significant topics and trends. This approach not only enhances searchability but
provides connections that go beyond potentially domain-specific author-defined keywords.
:
report,
data,
manufacturing,
respondent,
survey,
organizational,
reporting,
management,
performance,
sample,
census survey
Tags
Tags are automatically generated using a pretrained language model from spaCy, which excels at
several tasks, including entity tagging.
The model is able to label words and phrases by part-of-speech,
including "organizations." By filtering for frequent words and phrases labeled as "organizations", papers are
identified to contain references to specific institutions, datasets, and other organizations.
:
National Science Foundation,
Annual Survey of Manufactures,
Center for Economic Studies,
National Bureau of Economic Research,
Federal Reserve Bank,
University of Chicago,
Census Bureau Center for Economic Studies,
Michigan Institute for Teaching and Research in Economics,
North American Industry Classification System,
Business Register,
Sloan Foundation,
University of Toronto,
Kauffman Foundation,
Probability Density Function,
Stanford University,
Management and Organizational Practices Survey
Similar Working Papers
Similarity between working papers are determined by an unsupervised neural
network model
know as Doc2Vec.
Doc2Vec is a model that represents entire documents as fixed-length vectors, allowing for the
capture of semantic meaning in a way that relates to the context of words within the document. The model learns to
associate a unique vector with each document while simultaneously learning word vectors, enabling tasks such as
document classification, clustering, and similarity detection by preserving the order and structure of words. The
document vectors are compared using cosine similarity/distance to determine the most similar working papers.
Papers identified with 🔥 are in the top 20% of similarity.
The 10 most similar working papers to the working paper 'The Management and Organizational Practices Survey (MOPS): Cognitive Testing*' are listed below in order of similarity.
-
Working PaperDeveloping Content for the Management and Organizational Practices Survey-Hospitals (MOPS-HP)🔥
September 2021
Working Paper Number:
CES-21-25
Nationally representative U.S. hospital data does not exist on management practices, which have been shown to be related to both clinical and financial performance using past data collected in the World Management Survey (WMS). This paper describes the U.S. Census Bureau's development of content for the Management and Organizational Practices Survey Hospitals (MOPS-HP) that is similar to data collected in the MOPS conducted for the manufacturing sector in 2010 and 2015 and the 2009 WMS. Findings from cognitive testing interviews with 18 chief nursing officers and 13 chief financial officers at 30 different hospitals across 7 states and the District of Columbia led to using industry-tested terminology, to confirming chief nursing officers as MOPS-HP respondents and their ability to provide recall data, and to eliminating questions that tested poorly. Hospital data collected in the MOPS-HP would be the first nationally representative data on management practices with queries on clinical key performance indicators, financial and hospital-wide patient care goals, addressing patient care problems, clinical team interactions and staffing, standardized clinical protocols, and incentives for medical record documentation. The MOPS-HP's purpose is not to collect COVID-19 pandemic information; however, data measuring hospital management practices prior to and during the COVID-19 pandemic are a byproduct of the survey's one-year recall period (2019 and 2020).View Full Paper PDF
-
Working PaperThe Management and Organizational Practices Survey (MOPS): Collection and Processing🔥
December 2018
Working Paper Number:
CES-18-51
The U.S. Census Bureau partnered with a team of external researchers to conduct the first-ever large-scale survey of management practices in the United States, the Management and Organizational Practices Survey (MOPS), for reference year 2010. With the help of the research team, the Census Bureau expanded and improved the survey for a second wave for reference year 2015. The MOPS is a supplement to the Annual Survey of Manufacturing (ASM), and so the collection and processing strategy for the MOPS built on the methodology for the ASM, while differing on key dimensions to address the unique nature of management relative to other business data. This paper provides detail on the mail strategy pursued for the MOPS, the collection methods for paper and electronic responses, the processing and estimation procedures, and the official Census Bureau data releases. This detail is useful for all those who have interest in using the MOPS for research purposes, those wishing to understand the MOPS data more deeply, and those with an interest in survey methodology.View Full Paper PDF
-
Working PaperThe Management and Organizational Practices Survey (MOPS): An Overview*🔥
January 2016
Working Paper Number:
CES-16-28
Understanding productivity and business dynamics requires measuring production outputs and inputs. Through its surveys and use of administrative data, the Census Bureau collects information on production outputs and inputs including labor, capital, energy, and materials. With the introduction of the Management and Organizational Practices Survey (MOPS), the Census Bureau added information on another component of production: management. It has long been hypothesized that management is an important component of firm success, but until recently the study of management was confined to hypotheses, anecdotes, and case studies. Building upon the work of Bloom and Van Reenen (2007), the first-ever large scale survey of management practices in the United States, the MOPS, was conducted by the Census Bureau for 2010. A second, enhanced version of the MOPS is being conducted for 2015. The enhancement includes two new topics related to management: data and decision making (DDD) and uncertainty. As information technology has expanded plants are increasingly able to utilize data in their decision making. Structured management practices have been found to be complementary to DDD in earlier studies. Uncertainty has policy implications because uncertainty is found to be associated with reduced investment and employment. Uncertainty also plays a role in the targeting component of management.View Full Paper PDF
-
Working PaperDevelopment of Survey Questions on Robotics Expenditures and Use in U.S. Manufacturing Establishments🔥
October 2018
Working Paper Number:
CES-18-44
The U.S. Census Bureau in partnership with a team of external researchers developed a series of questions on the use of robotics in U.S. manufacturing establishments. The questions include: (1) capital expenditures for new and used industrial robotic equipment in 2018, (2) number of industrial robots in operation in 2018, and (3) number of industrial robots purchased in 2018. These questions are to be included in the 2018 Annual Survey of Manufactures. This paper documents the background and cognitive testing process used for the development of these questions.View Full Paper PDF
-
Working PaperData in Action: Data-Driven Decision Making in U.S. Manufacturing
January 2016
Working Paper Number:
CES-16-06
Manufacturing in America has become significantly more data-intensive. We investigate the adoption, performance effects and organizational complementarities of data-driven decision making (DDD) in the U.S. Using data collected by the Census Bureau for 2005 and 2010, we observe the extent to which manufacturing firms track and use data to guide decision making, as well as their investments in information technology (IT) and the use of other structured management practices. Examining a representative sample of over 18,000 plans, we find that adoption of DDD is earlier and more prevalent among larger, older plants belonging to multi-unit firms. Smaller single-establishment firms adopt later but have a higher correlation with performance than similar non-adopters. Using a fixed-effects estimator, we find the average value-added for later DDD adopters to be 3% greater than non-adopters, controlling for other inputs to production. This effect is distinct from that associated with IT and other structured management practices and is concentrated among single-unit firms. Performance improves after plants adopt DDD, but not before ' consistent with a causal relationship. However, DDD-related performance differentials decrease over time for early and late adopters, consistent with firm learning and development of organizational complementarities. Formal complementarity tests suggest that DDD and high levels of IT capital reinforce each other, as do DDD and skilled workers. For some industries, the benefits of DDD adoption appear to be greater for plants that delegate some decision making to frontline workers.View Full Paper PDF
-
Working PaperManagement in America
January 2013
Working Paper Number:
CES-13-01
The Census Bureau recently conducted a survey of management practices in over 30,000 plants across the US, the first large-scale survey of management in America. Analyzing these data reveals several striking results. First, more structured management practices are tightly linked to better performance: establishments adopting more structured practices for performance monitoring, target setting and incentives enjoy greater productivity and profitability, higher rates of innovation and faster employment growth. Second, there is a substantial dispersion of management practices across the establishments. We find that 18% of establishments have adopted at least 75% of these more structured management practices, while 27% of establishments adopted less than 50% of these. Third, more structured management practices are more likely to be found in establishments that export, who are larger (or are part of bigger firms), and have more educated employees. Establishments in the South and Midwest have more structured practices on average than those in the Northeast and West. Finally, we find adoption of structured management practices has increased between 2005 and 2010 for surviving establishments, particularly for those practices involving data collection and analysis.View Full Paper PDF
-
Working PaperBusiness-Level Expectations and Uncertainty
December 2020
Working Paper Number:
CES-20-41
The Census Bureau's 2015 Management and Organizational Practices Survey (MOPS) utilized innovative methodology to collect five-point forecast distributions over own future shipments, employment, and capital and materials expenditures for 35,000 U.S. manufacturing plants. First and second moments of these plant-level forecast distributions covary strongly with first and second moments, respectively, of historical outcomes. The first moment of the distribution provides a measure of business' expectations for future outcomes, while the second moment provides a measure of business' subjective uncertainty over those outcomes. This subjective uncertainty measure correlates positively with financial risk measures. Drawing on the Annual Survey of Manufactures and the Census of Manufactures for the corresponding realizations, we find that subjective expectations are highly predictive of actual outcomes and, in fact, more predictive than statistical models fit to historical data. When respondents express greater subjective uncertainty about future outcomes at their plants, their forecasts are less accurate. However, managers supply overly precise forecast distributions in that implied confidence intervals for sales growth rates are much narrower than the distribution of actual outcomes. Finally, we develop evidence that greater use of predictive computing and structured management practices at the plant and a more decentralized decision-making process (across plants in the same firm) are associated with better forecast accuracy.View Full Paper PDF
-
Working PaperAddressing Data Gaps: Four New Lines of Inquiry in the 2017 Economic Census
September 2019
Working Paper Number:
CES-19-28
We describe four new lines of inquiry added to the 2017 Economic Census regarding (i) retail health clinics, (ii) management practices in health care services, (iii) self-service in retail and service industries, and (iv) water use in manufacturing and mining industries. These were proposed by economists from the U.S. Census Bureau's Center for Economic Studies in order to fill data gaps in current Census Bureau products concerning the U.S. economy. The new content addresses such issues as the rise in importance of health care and its complexity, the adoption of automation technologies, and the importance of measuring water, a critical input to many manufacturing and mining industries.View Full Paper PDF
-
Working PaperLessons for Targeted Program Evaluation: A Personal and Professional History of the Survey of Program Dynamics
August 2007
Working Paper Number:
CES-07-24
The Survey of Program Dynamics (SPD) was created by the 1996 welfare reform legislation to facilitate its evaluation. This paper describes the evolution of that survey, discusses its implementation, and draws lessons for future evaluation. Large-scale surveys can be an important part of a portfolio of evaluation methods, but sufficient time must be given to data collection agencies if a high-quality longitudinal survey is expected. Such a survey must have both internal (agency) and external (policy analyst) buy-in. Investments in data analysis by agency staff, downplayed in favor of larger sample sizes given a fixed budget, could have contributed to more external acceptance. More attention up-front to reducing the potentially deleterious effects of attrition in longitudinal surveys, such as through the use of monetary incentives, might have been worthwhile. Given the problems encountered by the Census Bureau in producing the SPD, I argue that ongoing multi-purpose longitudinal surveys like the Survey of Income and Program Participation are potentially more valuable than episodic special-purpose surveys.View Full Paper PDF
-
Working PaperEVIDENCE OF AN 'ENERGY-MANAGEMENT GAP' IN U.S. MANUFACTURING: SPILLOVERS FROM FIRM MANAGEMENT PRACTICES TO ENERGY EFFICIENCY
April 2013
Working Paper Number:
CES-13-25
In this paper we merge a well-cited survey of firm management practices into confidential U.S. Census microdata to examine whether generic, i.e. non-energy specific, firm management practices, 'spillover' to enhance energy efficiency in the United States. We find the relationship in U.S. plants to be more nuanced than past research on UK plants has suggested. Most management techniques have beneficial spillovers to energy efficiency, but an emphasis on generic targets, conditional on other management practices, results in spillovers that increase energy intensity. Our specification controls for industry specific effects at a detailed 6-digit NAICS level and shows that this result is stronger for firms in energy intensive industries. We interpret the empirical result that generic management practices do not necessarily spillover to improved energy performance as evidence of an 'energy management gap.'View Full Paper PDF
