The effects of ProVaka and Papilio interventions on children’s behavior and educator feedback: A cluster-randomized controlled trial in early childhood education and care
DOI:
https://doi.org/10.58955/jecer.132008Keywords:
social-emotional learning, intervention effectiveness, children's behavior, educator feedbackAbstract
Evidence-based interventions can develop social and emotional competencies and foster social relationships during early childhood. In this paper, we explored the efficacy, implementation fidelity, and social validity of two systemic, preventive social-emotional learning interventions in Finnish early childhood education and care (ECEC)—ProVaka and Papilio—after their initial nine weeks of implementation. ProVaka is an application of Positive Behavior Intervention and Support that provides general guidelines for the development of social-emotional skills. Alongside general guidelines, the Papilio intervention also utilizes ready-made lesson plans and play-based measures. In this study, 80 ECEC classes from 39 ECEC centers participated in a cluster-randomized controlled trial. We measured educators’ feedback, children’s behavior, social validity, and implementation fidelity. The results indicate effects particularly on educator-reported child behavior and educators’ feedback on certain observed areas when the systemic interventions were implemented with sufficient fidelity. Further, both interventions seem well suitable for Finnish ECEC context.
References
Barkley, R. A., & Edelbrock, C. (1987). Assessing situational variation in children’s problem behaviors: The home and school situations questionnaires. Advances in Behavioral Assessment of Children and Families, 3, 157–176.
Blewitt, C., Fuller-Tyzkiewich, M., Nolan, A., Bergmeier, H., Vicary, D., Huang, T., McCabe, P., McKay, T., & Skouteris, H. (2018). Social and emotional learning associated with universal curriculum-based interventions in early childhood education and care centers. A systematic review and meta-analysis. JAMA Network Open, 1(8), 1–19. https://doi.org/10.1001/jamanetworkopen.2018.5727
Bradshaw, C., Waasdorp, T., & Leaf, P. (2015). Examining variation in the impact of school-wide positive behavioral interventions and supports: Findings from a randomized controlled effectiveness trial. Journal of Educational Psychology, 107(2), 546–557. https://doi.org/10.1037/a0037630
Carr, E., & Horner, R. (2007). The expanding vision of positive behavior support: Research perspectives on happiness, helpfulness, hopefulness. Journal of Positive Behavior Interventions, 9(1), 3–14. https://doi.org/10.1177/10983007070090010201
Carter, D., Van Norman, R., & Tredwell, C. (2011). Program-wide positive behavior support in preschool: Lessons for getting started. Early Childhood Education Journal, 38(5), 349–355. https://doi.org/10.1007/s10643-010-0406-0
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
Collaborative for Academic, Social, and Emotional Learning (CASEL) (2013). Effective social and emotional learning programs. Preschool and elementary school edition. Collaborative for Academic, Social, and Emotional Learning.
Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79(2), 135–144. https://doi.org/10.1177/001440291307900201
Dart, E. H., Radley, K. C., Briesch, A. M., Furlow, C. M., & Cavell, H. J. (2016). Assessing the accuracy of class-wide direct observation methods: Two analyses using simulated and naturalistic data. Behavioral Disorders, 41(3), 148–160. https://doi.org/10.17988/BD-15-49.1
Dufrene, B. A., Parker, K., Menousek, K., Zhou, Q., Harpole, L. L., & Olmi, D. J. (2012). Direct behavioral consultation in head start to increase teacher use of praise and effective instruction delivery. Journal of Educational and Psychological Consultation, 22(3), 159–186. https://doi.org/10.1080/10474412.2011.620817
Dunlap, G., Kincaid, D., Horner, R. H., Knoster, T., & Bradshaw, C. (2014). A comment on the term “positive behavior support.” Journal of Positive Behavior Interventions, 16(3), 133–136. https://doi.org/10.1177/1098300713497099
Early Childhood Education and Care Act (540/2018). Finlex. www.finlex.fi/fi/laki/alkup/2018/20180540
Embse, N., Ryan, S., Gibbs, T., & Mankin, A. (2019). Teacher stress interventions: A systematic review. Psychology in the Schools, 56(8), 1328–1343. https://doi.org/10.1002/pits.22279
Ferguson, T., Briesch, A., Volpe, R., Donaldson, A., & Feinberg, A. (2018). Psychometric considerations for conducting observations using time-sampling procedures. Assessment for Effective Intervention, 44(1), 45–57. https://doi.org/10.1177/1534508417747389
Finnish National Agency for Education. (2022). Core curriculum for early childhood education and care (Regulation OPH-700-2022). Finnish National Agency for Education. https://www.oph.fi/sites/default/files/documents/National%20core%20curriculum%20for%20ECEC%202022.pdf
Floress, M. T., Beschta, S. L., Meyer, K. L., & Reinke, W. M. (2017). Praise research trends and future directions: Characteristics and teacher training. Behavioral Disorders, 43(1), 227–243. https://doi.org/10.1177/0198742917704648
Fox, L., Hemmeter, M., Snyder, P., Perez Binder, T., & Clarke, S. (2011). Coaching early childhood special educators to implement a comprehensive model for promoting young children’s social competence. Topics in Early Childhood Special Education, 31(3), 178–192. https://doi.org/10.1177/0271121411404440
Fullerton, E. K., Conroy, M. A., & Correa, V. I. (2009). Early childhood teachers’ use of specific praise statements with young children at risk for behavioral disorders. Behavioral Disorders, 34(3), 118–135. https://doi.org/10.1177/019874290903400302
Gitlin, L. N., & Parisi, J. M. (2016). Are treatment effects real? The role of fidelity. In L.N. Gitlin & S. J. Czaja (Eds.), Behavioral intervention research: Designing, evaluating, and implementing (pp. 213–240). Springer Publishing Company.
Heiskanen, N., Karhu, A., Savolainen, H., & Närhi, V. (2024). Implementing positive behaviour support in Finnish early childhood education and care: Leadership teams’ perspectives. European Journal of Special Needs Education, 39(2), 265–280. https://doi.org/10.1080/08856257.2023.2207057
Hemmeter, M. L., Fox, L., Jack, S., & Broyles, L. (2007). A program-wide model of positive behavior support in early childhood settings. Journal of Early Intervention, 29(4), 337–355. https://doi.org/10.1177/105381510702900405
Hemmeter, M., Hardy, J., Schnitz, A., Adams, J., & Kinder, K. (2015). Effects of training and coaching with performance feedback on teachers’ use of pyramid model practices. Topics in Early Childhood Special Education, 35(3), 144–156. https://doi.org/10.1177/0271121415594924
Hemmeter, M., Snyder, P., Fox, L., & Algina, J. (2016). Evaluating the implementation of the pyramid model for promoting social–emotional competence in early childhood classrooms. Topics in Early Childhood Special Education, 36(3), 133–146. https://doi.org/10.1177/0271121416653386
Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptional Children, 42(8), 1–14. https://doi.org/10.17161/foec.v42i8.6906
Karhu, A., Heiskanen, N., & Närhi, V. (2021). Kohti sosioemotionaalisia taitoja ja lasten osallisuutta tukevaa toimintakulttuuria—ProVaka-toimintamallin pilottivaiheen analyysi [Towards a working culture that supports socio-emotional skills and child participation—pilot study analysis of the ProVaka support model]. Journal of Early Childhood Education Research, 10(1), 83–113. https://journal.fi/jecer/article/view/114151
Kato, N., Yanagawa, T., Fujiwara, T., & Morawska, A. (2015). Prevalence of children’s mental health problems and effectiveness of population‐level family interventions. Journal of Epidemiology, 25(8), 507–516. https://doi.org/10.2188/jea.JE20140198
Kincaid, D., & Horner, R. (2017). Changing systems to scale up an evidence-based educational intervention. Evidence Based Communication Assessment and Intervention, 11(3–4), 99–113. https://doi.org/10.1080/17489539.2017.1376383
Koivula, M., Laakso, M-L., Viitala, R., Neitola, M., Hess, M., & Scheithauer, H. (2020). The adaptation and implementation of the German social-emotional learning programme Papilio in Finland: A pilot study. International Journal of Psychology, 55(S1), 60–69. https://doi.org/10.1002/ijop.12615
LaBrot, Z. C., Dufrene, B. A., Whipple, H., McCargo, M., & Pasqua, J. L. (2020). Targeted and intensive consultation for increasing head start and elementary teachers’ behavior-specific praise. Journal of Behavioral Education, 29(4), 717–740. https://doi.org/10.1007/s10864-019-09342-9
Lane, K., Kalberg, J., Bruhn, A., Driscoll, S., Wehby, J., & Elliott, S. (2009). Assessing social validity of school-wide positive behavior support plans: Evidence for the reliability and structure of the primary intervention rating scale. School Psychology Review, 38(1), 135–144. https://doi.org/10.1080/02796015.2009.12087854
Lawson, G. M., McKenzie, M. E., Becker, K. D., Selby, L., & Hoover, S. A. (2019). The core components of evidence-based social–emotional learning programs. Prevention Science, 20(4), 457–467. https://doi.org/10.1007/s11121-018-0953-y
Mahon, D., Gunning, C., Holloway, J., & Lydon, H. (2020). Implementing positive behavior support within preschool settings: Group functional assessment and CW-FIT. Journal of Positive Behavior Interventions, 22(3), 167–180. https://doi.org/10.1177/1098300719890065
Marchant, M., Heath, M. A., & Miramontes, N. Y. (2013). Merging empiricism and humanism: Role of social validity in the school-wide positive behavior support model. Journal of Positive Behavior Interventions, 15(4), 221–230. https://doi.org/10.1177/1098300712459356
Mayer, H., Heim, P., Peter, C., & Scheithauer, H. (2016). Papilio: Theorie und grundlagen. Ein programm für kindertagesstätten zur prävention von verhaltensproblemen und zur förderung sozial-emotionaler kompetenz. Ein beitrag zur sucht-und gewaltprävention. [Papilio: Theory and basics. A programme for day-care centres to prevent behavioural problems and promote social-emotional skills. A contribution to addiction and violence prevention]. Papilio Verlag.
Mcintosh, K., Massar, M. M., Algozzine, R. F., George, H. P., Horner, R. H., Lewis, T. J., & Swain-Bradway, J. (2017). Technical adequacy of the SWPBIS tiered fidelity inventory. Journal of Positive Behavior Interventions, 19(1), 3–13. https://doi.org/10.1177/1098300716637193
McIntosh, K., Mercer, S. H., Nese R. N. T., Strickland-Cohen, M. K., & Hoselton, R. (2016). Predictors of sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 18(4), 209–218. https://doi.org/10.1177/10983007155997
Muthén, L. K., & Muthén, B. O. (1998-2012). Mplus user’s guide (7th ed.). Muthén & Muthén.
Määttä, S., Koivula, M., Huttunen, K., Paananen, M., Närhi, V., Savolainen, H., & Laakso, M.‐L. (2017). Lasten sosioemotionaalisten taitojen tukeminen varhaiskasvatuksessa (Raportit ja selvitykset 2017:17). [Supporting children’s social‐emotional skills in early childhood education]. Finnish National Agency for Education. http://oph.fi/julkaisut/2017/lasten_sosioemotionaalisten_taitojen_tukeminen_varhaiskasvatuksessa
Nelen, M. J., Willemse, T., Oudheusden, M. A. van, & Goei, S. (2020). Cultural challenges in adapting SWPBIS to a Dutch context. Journal of Positive Behavior Interventions, 22(2), 105–115. https://doi.org/10.1177/1098300719876096
O’Donnell, C. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of Educational Research, 78(1), 33–84. https://doi.org/10.3102/0034654307313793
Pelletier, J., Collett, B., Gimpel, G., & Crowley, S. (2006). Assessment of disruptive behaviors in preschoolers: Psychometric properties of the disruptive behavior disorders rating scale and school situations questionnaire. Journal of Psychoeducational Assessment, 24(1), 3–18. https://doi.org/10.1177/0734282905285235
Privacy Protection Act (2018). https://www.finlex.fi/fi/laki/ajantasa/2018/20181050
Schaack, D., Le, V., & Stedron, J. (2020). When fulfillment is not enough: Early childhood teacher occupational burnout and turnover intentions from a job demands and resources perspective. Early Education and Development, 31(7), 1011–1030. https://doi.org/10.1080/10409289.2020.1791648
Scheithauer, H., Bondü, R., Hess, M., & Mayer, H. (2016). Förderung sozial-emotionaler Kompetenzen im Vorschulalter: Ergebnisse der augsburger längsschnittstudie zur evaluation des primärpräventiven programms Papilio® (ALEPP) [Promoting social-emotional skills at preschool age: Results of the Augsburg longitudinal study to evaluate the Papilio® primary prevention programme (ALEPP)]. In T. Malti & S. Perren (Eds.), Soziale kompetenz bei kindern und jugendlichen—entwicklungsprozesse und fördermöglichkeiten. 2. bearb. & erw. Auflage [Social competence in children and young people—development processes and support options. 2nd revised and expanded edition] (pp. 155–176). Kohlhammer.
Scheithauer, H., Bondü, R., Niebank, K., & Mayer, H. (2007). Prävention von Verhaltensproblemen und Förderung prosozialen Verhaltens bei hoch-und niedrig-Risikokindern im Kindergarten: Erste Ergebnisse der Augsburger Längsschnittstudie zur Evaluation des Programms Papilio [Prevention of behavioural problems and promotion of prosocial behaviour in high- and low-risk children in kindergarten: First results of the Augsburg longitudinal study to evaluate the Papilio programme]. Praxis der Rechtspsychologie, 17(2), 376–391.
Schulte, A., Easton, J., & Parker, J. (2009). Advances in treatment integrity research: Multidisciplinary perspectives on the conceptualization, measurement, and enhancement of treatment integrity. School Psychology Review, 38(4), 460–475.
Shepley, C., & Grisham-Brown, J. (2019). Applied behavior analysis in early childhood education: An overview of policies, research, blended practices, and the curriculum framework. Behavior Analysis in Practice, 12(1), 235–246. https://doi.org/10.1007/s40617-018-0236-x
Spilt, J. L., Leflot, G., Onghena, P., & Colpin, H. (2016). Use of praise and reprimands as critical ingredients of teacher behavior management: Effects on children’s development in the context of a teacher mediated classroom intervention. Prevention Science, 17(6), 732–742. https://doi.org/10.1007/s11121-016-0667-y
Stanton-Chapman, T., Walker, V., Voorhees, M., Snell, M., Houchins, D., & Oakes, W. (2016). The evaluation of a three-tier model of positive behavior interventions and supports for preschoolers in Head Start. Remedial and Special Education, 37(6), 333–344. https://doi.org/10.1177/0741932516629650
Stormont, M. A., Covington Smith, S., & Lewis, T. J. (2007). Teacher implementation of precorrection and praise statements in head start classrooms as a component of a program-wide system of positive behavior support. Journal of Behavioral Education, 16(3), 280–290. https://doi.org/10.1007/s10864-007-9040-3
Strain, P., Barton, E., & Dunlap, G. (2012). Lessons learned about the utility of social validity. Education & Treatment of Children, 35(2), 183–200. https://doi.org/10.1353/etc.2012.0007
Suresh, K. P. (2011). An overview of randomization techniques: An unbiased assessment of outcome in clinical research. Journal of Human Reproductive Sciences, 4(1), 8-11. https://doi.org/10.4103/0974-1208.82352
Urban, M., Reikerås, E., Eidsvåg, G. M., Guevara, J., Saegø, J., & Semmoloni, C. (2022). Nordic approaches to evaluation and assessment in early childhood education and care: Final report. Nordic Council of Ministers.
Vlasov, J., Salminen, J., Repo, L., Karila, K., Kinnunen, S., Mattila, V., Nukarinen, T., Parrila, S., & Sulonen, H. (2018). Guidelines and recommendations for evaluating the quality of early childhood education and care (Publications 5:2019). Finnish Education Evaluation Centre.
Voegler-Lee, M. E., & Kupersmidt, J. B. (2011). Intervening in childhood social development. In P. K. Smith & C. H. Hart (Eds.), The Wiley-Blackwell handbook of childhood social development (2nd ed.)(pp. 605–626). Wiley-Blackwell.
Weissberg, R. P., Durlak, J. A., Domitrovich, C. E., & Gullotta, T. P. (2017). Social and emotional learning. Past, present, and future. In J. A. Durlak, C. E. Domitrovich, R. P. Weissberg & T. P. Gullotta (Eds.), Handbook of social and emotional learning. Research and practice (pp. 3–19). Guilford.
Willemse, T. M., Goei, S. L., Boei, F., & de Bruïne, E. J. (2023). School-wide positive behavior interventions and support in Dutch schools for special education. European Journal of Special Needs Education, 38(3), 424–439. https://doi.org/10.1080/08856257.2022.2120331
Wirth, O., Slaven, J., & Taylor, M. (2014). Interval sampling methods and measurement error: A computer simulation. Journal of Applied Behavior Analysis, 47(1), 83–100. https://doi.org/10.1002/jaba.93
Yeung, A. S., Craven, R. G., Mooney, M., Tracey, D., Barker, K., Power, A., Dobia, B., Chen, Z., Schofield, J., Whitefield, P., & Lewis, T. J. (2016). Positive behavior interventions: The issue of sustainability of positive effects. Educational Psychology Review, 28(1), 145–170. https://doi.org/10.1007/s10648-015-9305-7
Zakszeski, B., Hojnoski, R., & Wood, B. (2017). Considerations for time sampling interval durations in the measurement of young children’s classroom engagement. Topics in Early Childhood Special Education, 37(1), 42–53. https://doi.org/10.1177/0271121416659054
