Results from two meta-analyses of research into the efficacy of after-school programs ______.

* References marked with an asterisk indicate studies included in the meta-analysis

Afterschool Alliance. America after 3PM: Afterschool programs in demand. 2014 http://afterschoolalliance.org/documents/AA3PM-2014/AA3PM_National_Report.pdf

* Apsler R. After-school programs for adolescents: A review of evaluation research. Adolescence. 2009;44:1–19. [PubMed] [Google Scholar]

Arcaira E, Vile JD, Reisner ER. Achieving high school graduation: Citizen Schools’ youth outcomes in Boston. Washington, DC: Policy Studies Associates Inc; 2010. [Google Scholar]

* Biggart A, Kerr K, O’Hare L, Connolly P. A randomised control trial evaluation of a literacy after-school programme for struggling beginning readers. International Journal of Educational Research. 2013;62:129–140. [Google Scholar]

Biglan A, Metzler CW, Wirt R, Ary D, Noell J, Ochs L, et al. Social and behavioral factors associated with high-risk sexual behaviors among adolescents. Journal of Behavioral Medicine. 1990;13:245–261. [PubMed] [Google Scholar]

* Blumer MLC, Werner-Wilson RJ. Leaving no girl behind: Clinical intervention effects on adolescent female academic “high-risk” behaviors. Journal of Feminist Family Therapy. 2010;22(1):22–42. [Google Scholar]

Bohnert A, Fredricks J, Randall E. Capturing unique dimensions of youth organized activity involvement theoretical and methodological considerations. Review of Educational Research. 2010;80(4):576–610. [Google Scholar]

Bohnert AM, Richards M, Kohl K, Randall E. Relationships between discretionary time activities, emotional experiences, delinquency and depressive symptoms among urban African American adolescents. Journal of Youth and Adolescence. 2009;38:587–601. [PubMed] [Google Scholar]

Bradshaw CP, Waasdorp TE, Goldweber A, Johnson SL. Bullies, gangs, drugs, and school: Understanding the overlap and the role of ethnicity and urbanicity. Journal of Youth and Adolescence. 2013;42(2):220–234. [PubMed] [Google Scholar]

Buscemi N, Hartling L, Vandemeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. Journal of Clinical Epidemiology. 2006;59:697–703. [PubMed] [Google Scholar]

Bushman BJ, Wells GL. Narrative impressions of literature: The availability bias and the corrective properties of meta-analytic approaches. Personality and Social Psychology Bulletin. 2001;27(9):1123–1130. [Google Scholar]

Campbell Collaboration. Campbell Collaboration systematic reviews: Policies and guidelines. 2014 http://www.campbellcollaboration.org/lib/download/3308/C2_Policies_Guidelines_Version_1_0.pdf.

Cooper H, DeNeve K, Charlton K. Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychological Methods. 1997;2(4):447. [Google Scholar]

Cooper H, Hedges LV. The handbook of research synthesis. New York: Russell Sage Foundation; 1994. [Google Scholar]

Cross AB, Gottfredson DC, Wilson DM, Rorie M, Connell N. The impact of after-school programs on the routine activities of middle-school students: Results from a randomized, controlled trial. Criminology and Public Policy. 2009;8(2):391–412. [Google Scholar]

DuBois DL, Portillo N, Rhodes JE, Silverthorn N, Valentine JC. How effective are mentoring programs for youth? A systematic assessment of the evidence. Psychological Science in the Public Interest. 2011;12:57–91. [PubMed] [Google Scholar]

Durlak JA, Weissberg RP, Pachan M. A meta-analysis of after-school programs that seek to promote personal social skills in children and adolescents. American Journal of Community Psychology. 2010;45:294–309. [PubMed] [Google Scholar]

Dynarski M, James-Burdumy S, Moore M, Rosenberg L, Deke J, Mansfield W, et al. When schools stay open late: The national evaluation of the 21st Century Community Learning Centers program: New findings. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance; 2004. [Google Scholar]

Fashola OS. Review of extended-day and after-school programs and their effectiveness. Washington, D.C: Center for Research on the Education of Students Placed at Risk; 1998. [Google Scholar]

Fisher Z, Tipton E. Robumeta (Version 1.1) (Software) 2014 http://cran.r-project.org/web/packages/robumeta/robumeta.pdf

* Foley EM, Eddins G. Preliminary analysis of Virtual Y after-school program participants’ patterns of school attendance and academic performance. New York, NY: Fordham University; 2001. [Google Scholar]

Fox JA, Newman SA. After-school crime or after-school programs: Tuning into the prime time for violent juvenile crime and implications for national policy. A report to the United States Attorney General. Washington, D.C: Fight Crime: Invest in Kids; 1997. [Google Scholar]

* Frazier S, Mehta T, Atkins M, Hur K, Rusch D. Not just a walk in the park: Efficacy to effectiveness for after school programs in communities of concentrated urban poverty. Administration and Policy in Mental Health and Mental Health Services Research. 2013;40(5):406–418. [PubMed] [Google Scholar]

Glass GV, McGaw B, Smith ML. Meta-analysis in social research. Thousand Oaks, CA: Sage Publications; 1981. [Google Scholar]

* Gottfredson D, Cross A, Soulé D. Distinguishing characteristics of effective and ineffective after-school programs to prevent delinquency and victimization. Criminology and Public Policy. 2007;6(2):289–318. [Google Scholar]

Gottfredson D, Cross A, Wilson D, Connell N, Rorie M. Final report submitted to the US Department of Education Institute for Educational Sciences. 2010a. A randomized trial of the effects of an enhanced after-school program for middle-school students. [Google Scholar]

Gottfredson D, Cross A, Wilson D, Rorie M, Connell N. An experimental evaluation of the All Stars prevention curriculum in a community after school setting. Prevention Science. 2010b;11(2):142–154. [PubMed] [Google Scholar]

Gottfredson D, Cross A, Wilson D, Rorie M, Connell N. Effects of participation in after-school programs for middle school students: A randomized trial. Journal of Research on Educational Effectiveness. 2010c;3(3):282–313. [Google Scholar]

* Gottfredson D, Gerstenblith SA, Soulé D, Womer SC, Lu S. Do after school programs reduce delinquency? Prevention Science. 2004;5(4):253–266. [PubMed] [Google Scholar]

Gottfredson DC, Gottfredson GD, Weisman SA. The timing of delinquent behavior and its implications for after-school programs. Criminology & Public Policy. 2001;1:61–86. [Google Scholar]

Gozsche PC, Hrogjartsson A, Maric K, Tendal B. Data extraction errors in meta-analyses that use standardized mean differences. Journal of the American Medical Association. 2007;298:430–437. [PubMed] [Google Scholar]

Halpern R. After-school programs for low-income children: Promises and challenges. The Future of Children. 1999;9(2):81–95. [PubMed] [Google Scholar]

Halpern R. A different kind of child development institution: The history of after-school programs for low-income children. Teachers College Record. 2002;104:178–211. [Google Scholar]

Hedges LV, Tipton E, Johnson MC. Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods. 2010;1(1):164–165. [PubMed] [Google Scholar]

Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. British Medical Journal. 2011;343:d5928. doi: 10.1136/bmj.d5928. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 2002;21(11):1539–1558. [PubMed] [Google Scholar]

* Hirsch B, Hedges LV, Stawicki J, Mekinda MA. After-school programs for high school students: An evaluation of After School Matters. Chicago, IL: Northwestern University; 2011. [Google Scholar]

Hollister R. The growth in after-school programs and their impact. Washington, DC: Brookings Institution; 2003. [Google Scholar]

Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. The Cochrane Database of Systematic Reviews. 2009;1(1):1–26. [PMC free article] [PubMed] [Google Scholar]

Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. The Cochrane Database of Systematic Reviews. 2007;2(2) [PMC free article] [PubMed] [Google Scholar]

Huang D, Silver D, Cheung M, Duong N, Gualpa A, Hodson C, Rivera G. CRESST Report 789. Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing; 2011. Independent statewide evaluation of after school programs: ASES and 21st CCLC year 2 annual report. [Google Scholar]

* James-Burdumy S, Dynarski M, Deke J. When elementary schools stay open late: Results from the national evaluation of the 21st Century Community Learning Centers program. Educational Evaluation and Policy Analysis. 2007;29(4):296–318. [Google Scholar]

* James-Burdumy S, Dynarski M, Deke J. After-school program effects on behavior: Results from the 21st Century Community Learning Centers program national evaluation. Economic Inquiry. 2008;46(1):13–18. [Google Scholar]

James-Burdumy S, Dynarski M, Moore M, Deke J, Mansfield W, Pistorino C, et al. When schools stay open late: The national evaluation of the 21st Century Community Learning Centers program: Final report. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance; 2005. [Google Scholar]

Kane TJ. Working paper. New York, NY: W.T. Grant Foundation; 2004. The impact of after-school programs: Interpreting the results of four recent evaluations. [Google Scholar]

* LaFrance S, Twersky F, Latham N, Foley E, Bott C, Lee L. A safe place for healthy youth development: A comprehensive evaluation of the Bayview Safe Haven. San Francisco, CA: BTW Consultants & LaFrance Associates; 2001. [Google Scholar]

* Langberg J, Smith BH, Bogle KE, Schmidt JD, Cole WR, Pender CAS. A pilot evaluation of small group challenging horizons program (CHP): A randomized trial. Journal of Applied School Psychology. 2007;23(1):31–58. [Google Scholar]

Lauer PA, Akiba M, Wilkerson SB, Apthorp HS, Snow D, Martin-Glenn ML. Out-of-school-time programs: A meta-analysis of effects for at-risk students. Review of Educational Research. 2006;76:275–313. [Google Scholar]

* Le T, Arifuku I, Vuong L, Tran G, Lustig D, Zimring F. Community mobilization and community-based participatory research to prevent youth violence among Asian and immigrant populations. American Journal of Community Psychology. 2011;48(1/2):77–88. [PubMed] [Google Scholar]

Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks, CA: Sage; 2001. [Google Scholar]

Littell JH, Corcoran J, Pillai V. Systematic reviews and meta-analysis. Oxford: Oxford University Press; 2008. [Google Scholar]

Mahoney JL, Parente ME, Zigler EF. Afterschool programs in America: Origins, growth, popularity, and politics. Journal of Youth Development. 2009;4(3):25–44. [Google Scholar]

Maynard BR, Peters KE, Vaughn MG, Sarteschi CM. Fidelity in after-school program intervention research: A systematic review. Research on Social Work Practice. 2013;23(6):613–623. [Google Scholar]

Maynard BR, Salas-Wright CP, Vaughn MG, Peters KE. Who are truant youth? Examining distinctive profiles of truant youth using latent profile analysis. Journal of Youth and Adolescence. 2012;41(12):1671–1684. [PMC free article] [PubMed] [Google Scholar]

McKinsey & Company. The economic impact of the achievement gap in America’s schools. Washington, DC: Social Sector Office; 2009. [Google Scholar]

McLeod BD, Weisz JR. Using dissertations to examine potential bias in child and adolescent clinical trials. Journal of Consulting and Clinical Psychology. 2004;72(2):235–251. doi: 10.1037/0022-006x.72.2.235. [PubMed] [CrossRef] [Google Scholar]

* Molina BS, Flory K, Bukstein OG, Greiner AR, Baker JL, Krug V, et al. Feasibility and preliminary efficacy of an after-school program for middle schoolers with ADHD: A randomized trial in a large public middle school. Journal of Attention Disorders. 2008;12(3):207–217. [PubMed] [Google Scholar]

Moyer A, Schneider S, Knapp-Oliver SK, Sohl SJ. Published versus unpublished dissertations in psycho-oncology intervention research. Psycho-Oncology. 2010;19(3):313–317. [PMC free article] [PubMed] [Google Scholar]

Newman SA, Fox JA, Flynn EA, Christeson W. America’s after-school choice: The prime time for juvenile crime or youth enrichment and achievement. Washington, DC: Fight Crime: Invest in Kids; 2000. [Google Scholar]

* Nguyen D. A statewide impact study of 21st Century Community Learning Center programs in Florida. Tallahassee, FL: Department of Educational Leadership and Policy Studies, Florida State University. Doctor of Philosophy; 2007. [Google Scholar]

* Oyserman D, Terry K, Bybee D. A possible selves intervention to enhance school involvement. Journal of Adolescence. 2002;25(3):313. [PubMed] [Google Scholar]

Parsad B, Lewis L. After-school programs in public elementary schools (NCES 2009-043) Washington, DC: National Center for Education Statistics; 2009. [Google Scholar]

* Pastchal-Temple A. The effect of regular participation in an after-school program on student achievement, attendance, and behavior. Educational Administration, Mississippi State University. Doctor of Philosophy; 2012. [Google Scholar]

Pigott TD. Methods for handling missing data in research synthesis. In: Cooper H, Hedges LV, editors. The handbook of research synthesis. New York: Russell Sage Foundation; 1994. pp. 163–175. [Google Scholar]

Pigott TD. Advances in meta-analysis. New York, NY: Springer; 2012. [Google Scholar]

Pigott TD, Valentine JC, Polanin JR, Williams RT, Canada DD. Outcome-reporting bias in education research. Educational Researcher. 2013;42(8):424–434. doi: 10.3102/0013189X13507104. [CrossRef] [Google Scholar]

Polanin JR, Pigott TD. The use of meta-analytic statistical significance testing. Research Synthesis Methods. 2014 doi: 10.1002/jrsm.1124. Advanced online publication. [PubMed] [CrossRef] [Google Scholar]

* Prenovost JKE. A first-year evaluation of after-school learning programs in four urban middle schools in the Santa Ana Unified School District. Irvine, CA: Educational Administration, University of California—Irvine. Doctor of Education; 2001. [Google Scholar]

Redd Z, Cochran S, Hair E, Moore K. Academic achievement programs and youth development: A synthesis. 2002 www.childtrends.org/what_works/clarkwww/acadach/acadachrpt.pdf.

Richards MH, Miller BV, O’Donnell PC, Wasserman MS, Colder C. Parental monitoring mediates the effects of age and sex on problem behaviors among African American urban young adolescents. Journal of Youth and Adolescence. 2004;33:221–233. [Google Scholar]

Richardson JL, Dwyer K, McGuigan K, Hansen WB, Dent C, Johnson CA, et al. Substance use among eighth-grade students who take care of themselves after school. Pediatrics. 1989;84:556–566. [PubMed] [Google Scholar]

Ritter GW, Barnett JH, Denny GS, Albin GR. The effectiveness of volunteer tutoring programs for elementary and middle school students: A meta-analysis. Review of Educational Research. 2009;79:3–38. [Google Scholar]

Roth JL, Malone LM, Brooks-Gunn J. Does the amount of participation in afterschool programs relate to developmental outcomes? A review of the literature. American Journal of Community Psychology. 2010;45:310–324. [PubMed] [Google Scholar]

* Schinke S, Cole KC, Poulin SR. Enhancing the educational achievement of at-risk youth. Prevention Science. 2000;1(1):51–60. [PubMed] [Google Scholar]

Scott-Little C, Hamann MS, Jurs SG. Evaluations of after-school programs: A meta-evaluation of methodologies and narrative synthesis of findings. American Journal of Evaluation. 2002;23:387–419. [Google Scholar]

Shadish WR, Cook TD, Campbell TD. Experimental and quasi-experimental design for generalized causal inference. Belmont, CA: Wadsworth Cengage Learning; 2002. [Google Scholar]

Shernoff DJ, Vandell DL. Engagement in after-school program activities: Quality of experience from the perspective of participants. Journal of Youth and Adolescence. 2007;36:891–903. [Google Scholar]

* Sibley-Butler O. The impact of a 21st Century Community Learning Center after-school program on school-age children in grades 4 & 5. Hattiesburg, MS: University of Southern Mississippi. Doctor of Philosophy; 2004. [Google Scholar]

Simpkins SC, Little PMD, Weiss HB. Understanding and measuring attendance in out-of-school time programs (Number 7) 2004 http://www.gse.harvard.edu/hfrp/content/projects/afterschool/resources/issuebrief7.pdf

* Smeallie JE. An evaluation of an after-school tutorial and study skills program for middle school students at risk of academic failure. Maryland: Department of Curriculum and Instruction, University of Maryland at College Park. Doctor of Education; 1997. [Google Scholar]

Tanner-Smith EE, Tipton E. Robust variance estimation with dependent effect sizes: Practical considerations including a software tutorial in Stata and SPSS. Published advance online. Research Synthesis Methods. 2013 doi: 10.1002/jrsm.1091. [PubMed] [CrossRef] [Google Scholar]

* Tebes JK, Feinn R, Vanderploeg JJ, Chinman MJ, Shepard J, Brabham T, et al. Impact of a positive youth development program in urban after-school settings on the prevention of adolescent substance use. Journal of Adolescent Health. 2007;41(3):239–247. [PubMed] [Google Scholar]

Tolan P, Henry D, Schoeny M, Bass A, Lovegrove P, Nichols E. Mentoring interventions to affect juvenile delinquency and associated problems: A systematic review. Campbell Collaboration Reviews. 2013;2013:10. [Google Scholar]

U.S. Department of Education. Elementary and secondary education: Part B—21st Century Community Learning Centers. 2011 http://www2.ed.gov/policy/elsec/leg/esea02/pg55.html.

U.S. Department of Education. Programs: 21st Century Community Learning Centers. 2014 http://www2.ed.gov/programs/21stcclc/index.html.

Vandell DL, Shernoff DJ, Pierce KM, Bolt DM, Dadisman K, Brown BB. Activities, engagement, and emotion in after-school programs (and elsewhere) New Directions for Youth Development. 2005;105:121–129. [PubMed] [Google Scholar]

* Weisman SA, Womer SC, Kellstrong M, Bryner S, Kahler A, Slocum LA, et al. Maryland After-School Community Grant Program Part 1: Report on the 2002–2003 shool year evaluation of the phase 3 after-school programs. College Park, MD: University of Maryland; 2003. [Google Scholar]

* Welsh ME, Russell CA, Williams I, Reisner ER, White RN. Promoting learning and school attendance through after school programs: Student level changes in education across TASC’s first three years. Washington, D. C: Policy Studies Associates Inc; 2002. [Google Scholar]

Zief SG, Lauver S, Maynard RA. Campbell Systematic Reviews. 2006. Impacts of after-school programs on student outcomes. [Google Scholar]

Page 2

Summary of included studies