The Need to Account for Complex Sampling Features when Analyzing Establishment Survey Data: An Illustration using the 2013 Business Research and Development and Innovation Survey (BRDIS)
January 2017
Working Paper Number:
CES-17-62
Abstract
The importance of correctly accounting for complex sampling features when generating finite population inferences based on complex sample survey data sets has now been clearly established in a variety of fields, including those in both statistical and non statistical domains. Unfortunately, recent studies of analytic error have suggested that many secondary analysts of survey data do not ultimately account for these sampling features when analyzing their data, for a variety of possible reasons (e.g., poor documentation, or a data producer may not provide the information in a publicuse data set). The research in this area has focused exclusively on analyses of household survey data, and individual respondents. No research to date has considered how analysts are approaching the data collected in establishment surveys, and whether published articles advancing science based on analyses of establishment behaviors and outcomes are correctly accounting for complex sampling features. This article presents alternative analyses of real data from the 2013 Business Research and Development and Innovation Survey (BRDIS), and shows that a failure to account for the complex design features of the sample underlying these data can lead to substantial differences in inferences about the target population of establishments for the BRDIS.
Document Tags and Keywords
Keywords
Keywords are automatically generated using KeyBERT, a powerful and innovative
keyword extraction tool that utilizes BERT embeddings to ensure high-quality and contextually relevant
keywords.
By analyzing the content of working papers, KeyBERT identifies terms and phrases that capture the essence of the
text, highlighting the most significant topics and trends. This approach not only enhances searchability but
provides connections that go beyond potentially domain-specific author-defined keywords.
:
analysis,
estimating,
data,
statistical,
survey data,
survey,
study,
respondent,
research,
establishment,
innovation,
population,
sampling,
sample,
inference,
housing survey
Tags
Tags are automatically generated using a pretrained language model from spaCy, which excels at
several tasks, including entity tagging.
The model is able to label words and phrases by part-of-speech,
including "organizations." By filtering for frequent words and phrases labeled as "organizations", papers are
identified to contain references to specific institutions, datasets, and other organizations.
:
Service Annual Survey,
National Science Foundation,
Alfred P Sloan Foundation,
International Trade Research Report,
Business Research and Development and Innovation Survey,
National Center for Science and Engineering Statistics
Similar Working Papers
Similarity between working papers are determined by an unsupervised neural
network model
know as Doc2Vec.
Doc2Vec is a model that represents entire documents as fixed-length vectors, allowing for the
capture of semantic meaning in a way that relates to the context of words within the document. The model learns to
associate a unique vector with each document while simultaneously learning word vectors, enabling tasks such as
document classification, clustering, and similarity detection by preserving the order and structure of words. The
document vectors are compared using cosine similarity/distance to determine the most similar working papers.
Papers identified with 🔥 are in the top 20% of similarity.
The 10 most similar working papers to the working paper 'The Need to Account for Complex Sampling Features when Analyzing Establishment Survey Data: An Illustration using the 2013 Business Research and Development and Innovation Survey (BRDIS)' are listed below in order of similarity.
-
Working PaperAn In-Depth Examination of Requirements for Disclosure Risk Assessment
October 2023
Working Paper Number:
CES-23-49
The use of formal privacy to protect the confidentiality of responses in the 2020 Decennial Census of Population and Housing has triggered renewed interest and debate over how to measure the disclosure risks and societal benefits of the published data products. Following long-established precedent in economics and statistics, we argue that any proposal for quantifying disclosure risk should be based on pre-specified, objective criteria. Such criteria should be used to compare methodologies to identify those with the most desirable properties. We illustrate this approach, using simple desiderata, to evaluate the absolute disclosure risk framework, the counterfactual framework underlying differential privacy, and prior-to-posterior comparisons. We conclude that satisfying all the desiderata is impossible, but counterfactual comparisons satisfy the most while absolute disclosure risk satisfies the fewest. Furthermore, we explain that many of the criticisms levied against differential privacy would be levied against any technology that is not equivalent to direct, unrestricted access to confidential data. Thus, more research is needed, but in the near-term, the counterfactual approach appears best-suited for privacy-utility analysis.View Full Paper PDF
-
Working PaperSYNTHETIC DATA FOR SMALL AREA ESTIMATION IN THE AMERICAN COMMUNITY SURVEY
April 2013
Working Paper Number:
CES-13-19
Small area estimates provide a critical source of information used to study local populations. Statistical agencies regularly collect data from small areas but are prevented from releasing detailed geographical identifiers in public-use data sets due to disclosure concerns. Alternative data dissemination methods used in practice include releasing summary/aggregate tables, suppressing detailed geographic information in public-use data sets, and accessing restricted data via Research Data Centers. This research examines an alternative method for disseminating microdata that contains more geographical details than are currently being released in public-use data files. Specifically, the method replaces the observed survey values with imputed, or synthetic, values simulated from a hierarchical Bayesian model. Confidentiality protection is enhanced because no actual values are released. The method is demonstrated using restricted data from the 2005-2009 American Community Survey. The analytic validity of the synthetic data is assessed by comparing small area estimates obtained from the synthetic data with those obtained from the observed data.View Full Paper PDF
-
Working PaperR&D, Attrition and Multiple Imputation in BRDIS
January 2017
Working Paper Number:
CES-17-13
Multiple imputation in business establishment surveys like BRDIS, an annual business survey in which some companies are sampled every year or multiple years, may enhance the estimates of total R&D in addition to helping researchers estimate models with subpopulations of small sample size. Considering a panel of BRDIS companies throughout the years 2008 to 2013 linked to LBD data, this paper uses the conclusions obtained with missing data visualization and other explorations to come up with a strategy to conduct multiple imputation appropriate to address the item nonresponse in R&D expenditures. Because survey design characteristics are behind much of the item and unit nonresponse, multiple imputation of missing data in BRDIS changes the estimates of total R&D significantly and alters the conclusions reached by models of the determinants of R&D investment obtained with complete case analysis.View Full Paper PDF
-
Working PaperGrassroots Design Meets Grassroots Innovation: Rural Design Orientation and Firm Performance
March 2024
Working Paper Number:
CES-24-17
The study of grassroots design'applying structured, creative processes to the usability or aesthetics of a product without input from professional design consultancies'remains under investigated. If design comprises a mediation between people and technology whereby technologies are made more accessible or more likely to delight, then the process by which new grassroots inventions are transformed into innovations valued in markets cannot be fully understood. This paper uses U.S. data on the design orientation of respondents in the 2014 Rural Establishment Innovation Survey linked to longitudinal data on the same firms to examine the association between design, innovation, and employment and payroll growth. Findings from the research will inform questions to be investigated in the recently collected 2022 Annual Business Survey (ABS) that for the first time contains a Design module.View Full Paper PDF
-
Working PaperInvestigating the Effect of Innovation Activities of Firms on Innovation Performance: Does Firm Size Matter?
January 2025
Working Paper Number:
CES-25-04
Understanding the relationship between a firm's innovation activities and its performance has been of great interest to management scholars. While the literature on innovation activities is vast, there is a dearth of studies investigating the effect of key innovation activities of the firm on innovation outcomes in a single study, and whether their effects are dependent on the nature of firms, specifically firm size. Drawing from a longitudinal dataset from the Business Research & Development and Innovation Survey (BRDIS), and informed by contingency theory and resource orchestration theory, we examine the relationship between a firm's innovation activities - including its Research & Development (R&D) investment, securing patents, collaborative R&D, R&D toward new business areas, and grants for R&D - and its product innovation and process innovation. We also investigate whether these relationships are contingent on firm size. Consistent with contingency theory, we find a significant difference between large firms and small firms regarding how they enhance product innovation and process innovation. Large firms can improve product innovation by securing patents through applications and issuances, coupled with active participation in collaborative R&D efforts. Conversely, smaller firms concentrate their efforts on the number of patents applied for, directing R&D efforts toward new business areas, and often leveraging grants for R&D efforts. To achieve process innovation, a similar dichotomy emerges. Larger firms demonstrate a commitment to securing patents, engage in R&D efforts tailored to new business areas, and actively collaborate with external entities on R&D efforts. In contrast, smaller firms primarily focus on securing patents and channel their R&D efforts toward new business pursuits. This nuanced exploration highlights the varied strategies employed by large and small firms in navigating the intricate landscape of both product and process innovation. The results shed light on specific innovation activities as antecedents of innovation outcomes and demonstrate how the effectiveness of such assets is contingent upon firm size.View Full Paper PDF
-
Working PaperAn Economist's Primer on Survey Samples
September 2000
Working Paper Number:
CES-00-15
Survey data underlie most empirical work in economics, yet economists typically have little familiarity with survey sample design and its effects on inference. This paper describes how sample designs depart from the simple random sampling model implicit in most econometrics textbooks, points out where the effects of this departure are likely to be greatest, and describes the relationship between design-based estimators developed by survey statisticians and related econometric methods for regression. Its intent is to provide empirical economists with enough background in survey methods to make informed use of design-based estimators. It emphasizes surveys of households (the source of most public-use files), but also considers how surveys of businesses differ. Examples from the National Longitudinal Survey of Youth of 1979 and the Current Population Survey illustrate practical aspects of design-based estimation.View Full Paper PDF
-
Working PaperDo Environmental Regulations Disproportionately Affect Small Businesses? Evidence from the Pollution Abatement Costs and Expenditures Survey
September 2012
Working Paper Number:
CES-12-25R
It remains an open question whether the impact of environmental regulations differs by the size of the business. Such differences might be expected because of statutory, enforcement, and/or compliance asymmetries. Here, we consider the net effect of these three asymmetries, by estimating the relationship between plant size and pollution abatement expenditures, using establishment-level data on U.S. manufacturers from the Census Bureau's Pollution Abatement Costs and Expenditures (PACE) surveys of 1974-1982, 1984-1986, 1988-1994, 1999, and 2005, combined with data from the Annual Survey of Manufactures and Census of Manufactures. We model establishments' PAOC intensity - that is, their pollution abatement operating costs per unit of economic activity - as a function of establishment size, industry, and year. Our results show that PAOC intensity increases with establishment size. We also find that larger firms spend more per unit of output than do smaller firms.View Full Paper PDF
-
Working PaperUsing Small-Area Estimation (SAE) to Estimate Prevalence of Child Health Outcomes at the Census Regional-, State-, and County-Levels
November 2022
Working Paper Number:
CES-22-48
In this study, we implement small-area estimation to assess the prevalence of child health outcomes at the county, state, and regional levels, using national survey data.View Full Paper PDF
-
Working PaperIssues and Challenges in Measuring Environmental Expenditures by U.S. Manufacturing: The Redevelopment of the PACE Survey
July 2007
Working Paper Number:
CES-07-20
The Pollution Abatement Costs and Expenditures (PACE) survey is the most comprehensive source of information on U.S. manufacturing's capital expenditures and operating costs associated with pollution abatement. In 2003, the U.S. Environmental Protection Agency began a significant initiative to redevelop the survey, guided by the advice of a multi-disciplinary workgroup consisting of economists, engineers, survey design experts, and experienced data users, in addition to incorporating feedback from key manufacturing industries. This paper describes some of these redevelopment efforts. Issues discussed include the approach to developing the new survey instrument, methods used to evaluate (and improve) its performance, innovations in sampling, and the special development and role of outside expertise. The completely redesigned PACE survey was first administered in early 2006.View Full Paper PDF
-
Working PaperCharacteristics of the Top R&D Performing Firms in the U.S.: Evidence from the Survey of Industrial R&D
September 2010
Working Paper Number:
CES-10-33
Innovation drives economic growth and productivity growth, and as such, indicators of innovative activity such as research and development (R&D) expenditures are of paramount importance. We combine Census confidential microdata from two sources in order to examine the characteristics of the top R&D performing firms in the U.S. economy. We use the Survey of Industrial Research and Development (SIRD) to identify the top 200 R&D performing firms in 2003 and, to the extent possible, to trace the evolution of these firms from 1957 to 2007. The Longitudinal Business Database (LBD) further extends our knowledge about these firms and enables us to make comparisons to the U.S. economy. By linking the SIRD and the LBD we are able to create a detailed portrait of the evolution of the top R&D performing firms in the U.S.View Full Paper PDF
