Lessons for Targeted Program Evaluation: A Personal and Professional History of the Survey of Program Dynamics
August 2007
Working Paper Number:
CES-07-24
Abstract
The Survey of Program Dynamics (SPD) was created by the 1996 welfare reform legislation to facilitate its evaluation. This paper describes the evolution of that survey, discusses its implementation, and draws lessons for future evaluation. Large-scale surveys can be an important part of a portfolio of evaluation methods, but sufficient time must be given to data collection agencies if a high-quality longitudinal survey is expected. Such a survey must have both internal (agency) and external (policy analyst) buy-in. Investments in data analysis by agency staff, downplayed in favor of larger sample sizes given a fixed budget, could have contributed to more external acceptance. More attention up-front to reducing the potentially deleterious effects of attrition in longitudinal surveys, such as through the use of monetary incentives, might have been worthwhile. Given the problems encountered by the Census Bureau in producing the SPD, I argue that ongoing multi-purpose longitudinal surveys like the Survey of Income and Program Participation are potentially more valuable than episodic special-purpose surveys.
Document Tags and Keywords
Keywords
Keywords are automatically generated using KeyBERT, a powerful and innovative
keyword extraction tool that utilizes BERT embeddings to ensure high-quality and contextually relevant
keywords.
By analyzing the content of working papers, KeyBERT identifies terms and phrases that capture the essence of the
text, highlighting the most significant topics and trends. This approach not only enhances searchability but
provides connections that go beyond potentially domain-specific author-defined keywords.
:
analysis,
economist,
survey data,
survey,
agency,
respondent,
longitudinal,
valuation,
fiscal,
survey income,
assessing
Tags
Tags are automatically generated using a pretrained language model from spaCy, which excels at
several tasks, including entity tagging.
The model is able to label words and phrases by part-of-speech,
including "organizations." By filtering for frequent words and phrases labeled as "organizations", papers are
identified to contain references to specific institutions, datasets, and other organizations.
:
Social Security Administration,
Center for Economic Studies,
Office of Management and Budget,
Urban Institute,
Current Population Survey,
1940 Census,
Survey of Income and Program Participation,
General Accounting Office,
Social Security,
Department of Health and Human Services,
National Academy of Sciences
Similar Working Papers
Similarity between working papers are determined by an unsupervised neural
network model
know as Doc2Vec.
Doc2Vec is a model that represents entire documents as fixed-length vectors, allowing for the
capture of semantic meaning in a way that relates to the context of words within the document. The model learns to
associate a unique vector with each document while simultaneously learning word vectors, enabling tasks such as
document classification, clustering, and similarity detection by preserving the order and structure of words. The
document vectors are compared using cosine similarity/distance to determine the most similar working papers.
Papers identified with 🔥 are in the top 20% of similarity.
The 10 most similar working papers to the working paper 'Lessons for Targeted Program Evaluation: A Personal and Professional History of the Survey of Program Dynamics' are listed below in order of similarity.
-
Working PaperManagement Challenges of the 2010 U.S. Census🔥
August 2011
Working Paper Number:
CES-11-22
This paper gives an insider's perspective on the management approaches used to manage the 2010 Census during its operational phase. The approaches used, the challenges faced (in particular, difficulties faced in automating data collection), and the solutions applied to meet those challenges are described. Finally, six management lessons learned are presented.View Full Paper PDF
-
Working PaperMeasuring Poverty in the United States: History and Current Issues
April 2006
Working Paper Number:
CES-06-11
Formal measurement of poverty in the United States is now about 40 years old. This paper first briefly describes the origins and basis of the official poverty thresholds adopted by the federal government in the late 1960s. Then, it discusses in some detail some of the more current issues that observers suggest must be addressed if changes are to be made. The final sections discuss recent efforts to propose alternates to the current official approach.View Full Paper PDF
-
Working PaperIncome Packaging and Economic Disconnection: Do Sources of Support Differ from Other Low-Income Women?
December 2013
Working Paper Number:
CES-13-61
Income packaging, or piecing together cash and non-cash resources from a variety of sources, is a common financial survival strategy among low-income women. This strategy is particularly important for economically disconnected women, who lack both employment income and public cash assistance receipt. Using data from the confidential Census Bureau versions of the Survey of Income and Program Participation, this study compares the use of public and private supports between disconnected and connected low-income women, controlling for differences in state welfare rules and county unemployment rates. Findings from bivariate comparisons and multilevel logistic regressions indicate that disconnected women utilize public non-cash supports at similar rates to connected women, but rely more heavily on private sources. Conclusions focus on the policy implications for outreach and program development.View Full Paper PDF
-
Working PaperThe Impact of Welfare Waivers on Female Headship Decisions
February 2003
Working Paper Number:
CES-03-03
While much of the focus of recent welfare reforms has been on moving recipients from welfare to work, many reforms were also directed at affecting decisions about living arrangements, pregnancy, marriage and cohabitation. This paper focuses on women's decisions to become or remain unmarried mothers, that is, female heads of families. We assess the impact of welfare reform waivers on those decisions while controlling for confounding local economic and social contextual conditions. We pool the 1990, 1992, and 1993 panels of the Survey of Income and Program Participation (SIPP) which span the calendar time when many states began adopting welfare waivers. For its descriptors of local labor market conditions, the project uses skill specific measures of wages and employment opportunities for counties. We estimate models for levels of female headship and proportional hazard models for entry and exit from female headship. In the hazards, we employ stratified Cox partial likelihood methods and investigate the use of state fixed effects or state stratified hazard models to control for unmeasured state influences. Based on data through 1995, we find limited evidence that workencouraging waivers had a beneficial effect by reducing female headship of families. We find little evidence that family caps, teenage coresidence requirements or termination limits will reduce the number of single-parent families.View Full Paper PDF
-
Working PaperThe Impact of Household Surveys on 2020 Census Self-Response
July 2022
Working Paper Number:
CES-22-24
Households who were sampled in 2019 for the American Community Survey (ACS) had lower self-response rates to the 2020 Census. The magnitude varied from -1.5 percentage point for household sampled in January 2019 to -15.1 percent point for households sampled in December 2019. Similar effects are found for the Current Population Survey (CPS) as well.View Full Paper PDF
-
Working PaperEstimating the Immediate Impact of the COVID-19 Shock on Parental Attachment to the Labor Market and the Double Bind of Mothers
July 2020
Working Paper Number:
CES-20-22R
I examine the impact of the COVID-19 shock on parents' labor supply during the initial stages of the pandemic. Using difference-in-difference approaches and monthly panel data from the Current Population Survey (CPS), I compare labor market attachment, non-work activity, hours worked, and earnings and wages of those in areas with early school closures and stay-in-place orders with those in areas with delayed or no pandemic closures. While there was no immediate impact on detachment or unemployment, mothers with jobs in early closure states were 53.2 percent more likely than mothers in late closure states to have a job but not be working as a result of early shutdowns. There was no effect on working fathers or working women without school age children. Of mothers who continued working, those in early closure states worked more weekly hours than mothers in late closure states; fathers reduced their hours. Overall, the pandemic appears to have induced a unique immediate juggling act for working mothers of school age children.View Full Paper PDF
-
Working PaperAlternative Measures of Income Poverty and the Anti-Poverty Effects of Taxes and Transfers
June 2005
Working Paper Number:
CES-05-08
The Census Bureau prepared a number of alternative income-based measures of poverty to illustrate the distributional impacts of several alternatives to the official measure. The paper examines five income variants for two different units of analysis (families and households) for two different assumptions about inflation (the historical Consumer Price Index and a 'Research Series' alternative that uses current methods) for two different sets of thresholds (official and a formula-based alternative base on three parameters). The poverty rate effects are analyzed for the total population, the distributional effects are analyzed using poverty shares, and the anti-poverty effects of taxes and transfers are analyzed using a percentage reduction in poverty rates. Suggestions for future research are included.View Full Paper PDF
-
Working PaperThe Management and Organizational Practices Survey (MOPS): Cognitive Testing*
January 2016
Working Paper Number:
CES-16-53
All Census Bureau surveys must meet quality standards before they can be sent to the public for data collection. This paper outlines the pretesting process that was used to ensure that the Management and Organizational Practices Survey (MOPS) met those standards. The MOPS is the first large survey of management practices at U.S. manufacturing establishments. The first wave of the MOPS, issued for reference year 2010, was subject to internal expert review and two rounds of cognitive interviews. The results of this pretesting were used to make significant changes to the MOPS instrument and ensure that quality data was collected. The second wave of the MOPS, featuring new questions on data in decision making (DDD) and uncertainty and issued for reference year 2015, was subject to two rounds of cognitive interviews and a round of usability testing. This paper illustrates the effort undertaken by the Census Bureau to ensure that all surveys released into the field are of high quality and provides insight into how respondents interpret the MOPS questionnaire for those looking to utilize the MOPS data.View Full Paper PDF
-
Working PaperWhen and Why Does Nonresponse Occur? Comparing the Determinants of Initial Unit Nonresponse and Panel Attrition
September 2023
Working Paper Number:
CES-23-44
Though unit nonresponse threatens data quality in both cross-sectional and panel surveys, little is understood about how initial nonresponse and later panel attrition may be theoretically or empirically distinct phenomena. This study advances current knowledge of the determinants of both unit nonresponse and panel attrition within the context of the U.S. Census Bureau's Survey of Income and Program Participation (SIPP) panel survey, which I link with high-quality federal administrative records, paradata, and geographic data. By exploiting the SIPP's interpenetrated sampling design and relying on cross-classified random effects modeling, this study quantifies the relative effects of sample household, interviewer, and place characteristics on baseline nonresponse and later attrition, addressing a critical gap in the literature. Given the reliance on successful record linkages between survey sample households and federal administrative data in the nonresponse research, this study also undertakes an explicitly spatial analysis of the place-based characteristics associated with successful record linkages in the U.S.View Full Paper PDF
-
Working PaperThe EITC over the business cycle: Who benefits?
December 2014
Working Paper Number:
carra-2014-15
In this paper, I examine the impact of the Great Recession on Earned Income Tax Credit (EITC) eligibility. Because the EITC is structurally tied to earnings, the direction of this impact is not immediately obvious. Families who experience complete job loss for an entire tax year lose eligibility, while those experiencing underemployment (part-year employment, a reduction in hours, or spousal unemployment in married households) may become eligible. Determining the direction and magnitude of the impact is important for a number of reasons. The EITC has become the largest cash-transfer program in the U.S., and many low-earning families rely on it as a means of support in tough times. The program has largely been viewed as a replacement for welfare, enticing former welfare recipients into the labor force. However, the effectiveness of the EITC during a period of very high unemployment has not been assessed. To answer these questions, I first use the Current Population Survey (CPS) matched to Internal Revenue Service data from tax years 2005 to 2010 to assess patterns of employment and eligibility over the Great Recession for different labor-force groups. Results indicate that overall, EITC eligibility increased over the recession, but only among groups that were cushioned from total household earnings loss by marriage. I also use the 2006 CPS matched to tax data from 2005 through 2011 to examine changes in eligibility experienced by individuals over time. In assessing three competing causes of eligibility loss, I find that less-educated, unmarried women experienced a greater hazard of eligibility loss due a yearlong lack of earnings compared with other labor-market groups. I discuss the implications of these findings on the view of the EITC as a safety-net program.View Full Paper PDF
