Introduction

Following the development of the evidence-based movement, there has been a growing interest in the transfer of educational research into practice (Fixsen et al., 2019). During the last decade, a lot of activities have been carried out in order to make educational research more relevant and accessible to teachers. Two important perspectives are highlighted in this article: The “research literacy” and the “school-wide implementation” perspective. The perspective on research literacy in education is particularly relevant if the aim is to increase teachers’ or student teachers’ general research competence. Added to the implementation perspective are organizational factors and the context of teaching which promote the successful transfer of effective research-based programs or strategies into practice. Evidence-Based Practice (EBP) in education is a broad term which refers to entire models, programs, strategies, or interventions which have proven to be effective through rigorous scientific studies. Outcomes are evaluated in experimental studies (e.g. RCTs), meta-analyses and systematic reviews (Voyager Sopris Learning, 2023). Research-based practice on the other hand is also based on the best available research, but only parts of a program or an intervention may have proven effective through research. Although the terms are different, their common aim is to increase the use of EBP in schools. The concepts are often used interchangeably in the literature and both are used in this paper, according to their definitions.

The article has a particular focus on the situation in Norway, the reasons for this being that the author is more familiar with the developments in this country, that the empirical data that was collected in Norway on the evaluation of the model has previously been published internationally, and that the outcomes and conclusions seem to match those from other Western countries, particularly those in which the School-Wide Positive Behavior Support (SWPBS) model has been implemented (Australia, Belgium, Canada, Finland, Great Britain, New Zealand, the Netherlands and more, Michael et al., 2023).

The broad and narrow perspective on the relationship between research and practice

The enlightenment model (Hammersley, 2007; Weiss, 1979) has a broad perspective on research. It encourages science-conscious decisions and may over time influence practice indirectly by inspiring discussions and debate (Nutley et al., 2010). It demonstrates how research inspires thinking about problems and solutions and how research gradually and cumulatively influences policy decisions and plans (Weiss, 1979). The evidence-based model, on the other hand, takes a narrower perspective on the relationship between research, practice and policy. This perspective is more prescriptive and recommends that interventions should be theory based and requires carefully described guidelines, handbooks or manuals. Fidelity should be monitored and effectiveness evaluated in randomized controlled trials (RCTs). The evidence-based model is committed to the evidence-hierarchy which rates research designs according to their internal validity strengths (Pilcher & Bedford, 2011). At the top, representing the strongest empirical support are meta-analyses, and systematic reviews, followed by randomized controlled trials. Further down in the pyramid are cohort studies and case-control studies. The evidence-hierarchy has, however, proved difficult to apply in educational research. Research in education has mostly been of a conceptual and theoretical nature (Maughan et al., 2012) and the number of randomized controlled trials has been limited (Cook, 2003; Weare & Nind, 2011). Therefore, evaluation studies are often referred to as “evidence-inspired” or “evidence-informed” in order to signal a more liberal attitude to the kind of research to be included. But Robert Slavin (2002), a pioneer in educational research, concludes that a focus on rigorous experiments and sophisticated evaluations of realistic alternatives in practical settings on a scale that matters are essential to build confidence in educational research among policymakers and educators (Slavin, 2002). However, there are several reasons for the scarcity of intervention studies and effectiveness trials in education. From a critical perspective, the knowledge base of teaching and learning is both incomplete and ambiguous, and much of what goes on in school has not been evaluated. The funding for such studies has been insufficient, both from the Ministry of Education and the Norwegian Research Council (Kunnskapdepartementet, 2017). Another reason for the lack of research on intervention effectiveness may be the lack of interest among educational researchers to conduct research on “what works” while teachers on their side have had little faith in the quality and relevance of educational research. Moreover, standardized and prescriptive interventions which require compliance with detailed manuals or guidelines seem to have fueled the resistance among teachers (Mitchell & Sutherland, 2020). The critical attitudes of those who work in schools may also have tempered the educational researchers’ interest in evaluation research. Goldacre (2013) points out that research-based practice may have been disregarded because it has come into conflict with the teachers' sense of autonomy and their professional judgments. This has not prevented applied research from gaining more attention in schools and effectiveness studies have increasingly been in demand. Moreover, public documents and reports about education increasingly refer to “what works” and the Ministry of Education has requested more research that has an international perspective and the use of randomized controlled designs (Kunnskapdepartementet, 2017).

The research literacy approach

Research literacy describes the capacity of teachers to develop practices that integrate research-based knowledge with practical and ethical concerns. As a mediator between research and practice it may contribute to the professional and personal development of teachers. In the same way as the enlightenment model, the research literacy model has a liberal and inclusive perspective on research designs and methods. Quantitative, qualitative, and mixed-methods designs are accepted, as are cross-sectional and longitudinal research. It also acknowledges professional expertise in interpreting educational research. The research literacy approach seems to be particularly well adapted to teacher training institutions like universities and colleges. But how to conduct research literacy training is eagerly debated, and differences of opinion are expressed about the content and process of such training. Among topics being discussed are practical methods versus methodological ideas, and basic versus advanced and comprehensive strategies. In research on teaching, a hot topic has been the relative importance of general knowledge and knowledge adapted to specific interventions, populations and situations. Students have different backgrounds, abilities and needs, and the situation in which teaching takes place may strongly moderate the design and the outcomes of studies. Even if research literacy as a mediator has an inclusive attitude toward research, it does not seem to conceptualize or emphasize the quality of the transfer. The level of adherence and competence should probably be reported with greater accuracy in the process of transferring research knowledge to practice.

Teacher training institutions and EBP

In order to increase teachers’ research literacy, the support and efforts of teacher training institutions are important. This has not always been the case, and for a long period of time, the enthusiasm for research-based strategies and methods was modest in Norwegian universities and colleges. Experience-based knowledge had far greater credibility and status than research-based knowledge. Future teachers' practice seemed to be more influenced by their practice experiences before, during and after the study than by knowledge-based didactic teaching. And if teachers in their practice run into problems, many would rather ask their colleagues for advice than consult the research literature. This tradition was challenged in the Norwegian strategy Teacher Education 2025: National Strategy for Quality and Collaboration in Teacher Education, which emphasizes research anchoring, practice relevance and interdisciplinary collaboration (Norwegian Ministry of Education and Research, 2018). It is also expected that those who teach professional courses should normally have a doctorate and research competence (Forskningsrådet, 2021). Teacher training programs may promote research literacy through the dissemination of empirically supported strategies, methods and programs and make a clear distinction between practices that are research-based and those that are not. In the report from the BarnUnge 21 project, the Norwegian Research Council asked for new guidelines for teacher training and claimed that updated and quality-assured research on teaching was needed in order to upgrade the staff's competence.

Critical assessment of research

The critical assessment model was originally developed in the field of medical research where evidence-based practice was defined as: “… the integration of best available evidence, clinical expertise, and patient values” (Sackett et al., 1996). Research literacy could be acquired through studying and learning how to critically assess the quality and relevance of research studies. The model has previously been applied in research on social work (Finne & Malmberg-Heimonen, 2023) but also in educational research (Chase, 2019). The critical assessment approach requires students and practitioners to track and access all relevant research studies in order to assess their sample representativeness, the statistical power of the study, the reliability of the measurement instruments, the internal and external validity and the practical relevance of outcomes. But critical assessment may also include the critical review of meta-analyses, syntheses, and summaries of research. As a next step, teachers are encouraged to interpret, test and adapt research-based strategies to their own practice. The critical assessment approach makes high demands on teachers' abilities and efforts, and has been particularly difficult for practicing teachers. Many have problems accessing research and struggle to find relevant literature. Another problem is the lack of sufficient time to read, reflect and discuss research results with colleagues. And finally, few, if any research articles deal with issues that practitioners are concerned with, and they are often written in an incomprehensible language for practitioners.

When EBP was introduced in schools, research repeatedly reported that school-wide interventions were more effective and sustainable than interventions adopted by a few teachers or applied in only some classes (Durlak & DuPre, 2008; Moore et al., 2019; Weare & Nind, 2011). The unit of change thus became, to a greater extent, entire schools rather than selected teachers, pupils or school classes. Research also demonstrated that implementation quality was a strong moderator of outcomes (Durlak & DuPre, 2008). Such findings contributed to implementation quality becoming a central issue in the process of changing school practice.

The school-wide implementation model

The concept of implementation refers to factors that promote or hinder knowledge mobilization and practice change in school. Compared to research literacy, the implementation perspective is more concerned with school and school-contextual factors that promote or hinder the quality and sustainability of evidence-based interventions (Fixsen et al., 2019; Ogden & Fixsen, 2014). The implementation process in schools is usually guided by implementation frameworks. These are conceptual models which organize sets of coherent ideas and concepts and provide an overview of the implementation process that is easily communicated to practitioners (Durlak & DuPre, 2008; Fixsen et al., 2005, 2019). One example is the “Active Implementation Framework” which includes implementation drivers that promote the implementation of evidence-based practices (Ogden & Fixsen, 2014).

Most of the available school-wide intervention programs are evaluated in RCTs or in quasi-experimental studies by the program developers and/or by independent researchers. They are usually supported by handbooks or manuals, which give detailed descriptions of how to implement and monitor practice (fidelity) and outcomes. Schools and teachers are not exactly invited to critically examine the research base of the programs. Instead, they are offered guidelines describing how to build program specific capacity and competence at their school.

The guideline approach

The process of identifying interventions and critically examining their research base may be too time-consuming or too complex to carry out for teachers at ordinary schools. One way to simplify the implementation process is to delegate the task of critical research-assessment to experts or resource people from the schools’ support systems. Their task is to identify interventions that are relevant and adaptable to the school’s needs and resources, and, based on this information, produce guidelines for the school. The experts are usually professionals who are familiar with a number of relevant interventions and know how to critically review the literature (e.g. Hattie, 2009). In this way, they may ease the burden on practitioners by identifying, summarizing, simplifying and assessing the EBP literature.

In the public sector, such experts are sometimes referred to as knowledge brokers. They act as middlemen or agents who inform, interpret, negotiate and intermediate between program owners and program users (Oxford English Dictionary OED, 2023). They may distribute evidence and build capacity by disseminating guidelines and providing training at schools. By contributing to the increase of general and program specific capacity of schools, they help close the gap between research and practice. Knowledge brokers take care of much of the practical work of knowledge transfer by serving the needs of both “creators” or “owners” and “users” of knowledge. By linking research and practice they help break with the tradition where researchers and practitioners have shown little interest in each other. Compared to other school professionals and researchers they are not bound to any specific program or intervention. They may serve a number of schools within a municipality or a region and may also help by adding tacit knowledge of individuals and organizations (Ward et al., 2009). In this way, they promote positive attitudes and encourage openness and readiness to try out new practices (Wandersman et al., 2008). Professional knowledge brokers are still uncommon in some countries such as Norway but are found in several other countries (Metz et al., 2021). In the next section, core implementation components are presented based on a review of international research on factors that promote the use of evidence-based practices in schools.

What enables or hinders the use of research-based knowledge in school?

A review group consisting of the author of this article, Robert Slavin and Jonathan Sharples, examined 34 high-quality international research articles on factors which hindered or promoted the use of research-based knowledge in primary and lower secondary school (Brørup-Dyssegaard et al., 2017). The report was published by the Danish Clearinghouse for Education to identify common implementation drivers across studies of successful introduction of new practices in schools. Without specifically mentioning research literacy, it was clearly implied by several of the promoting factors. Six key components were identified: a) school leadership and management, b) the staff’s professional development, c) the teachers’ attitudes and perceptions, d) the school’s support system, e) implementation quality and f) sustainability. The importance of research literacy was implied by the staff’s professional development, teachers’ attitudes and perceptions and the sustainability of interventions (Brørup-Dyssegaard et al., 2017).

School leadership or management was identified as the single most important predictor of successful change of practice. The Danish report stresses the importance of having a school leader or a management team that is willing to prioritize the necessary resources for the implementation process. Through their contacts with academic institutions and other schools, principals signal their interest and commitment to research. Within their own school, they create a sense of community where staff is encouraged to share experiences and to request feedback from the students.

The staff’s professional development is described as individual or collective professional capacity or competence to implement research-based practices. This implementation driver takes an organizational view on schools and highlights collective decisions and factors as decisive for sustainable changes.

The attitudes and perceptions of teachers and additional staff. Several of the studies examined in the overview showed that positive expectations, including teacher’s confidence in the effectiveness of the intervention, were vital for successful implementation. Important facilitators were the understanding of how well the intervention matched the theoretical views, attitudes and values of the practitioners and the optimal time for the introduction of an intervention at a practice site. Equally important were a high problem awareness and readiness for change. Teachers’ motivation for change seems to increase when they are aware of unsolved problems and acknowledge the need for improvements. Increased motivation is also associated with time and resources for planning and preparation, and whether teachers have a say in the school’s choice of programs and practices.

Support systems. In order to successfully implement EBP, schools need a support system which may assist in identifying and assessing effective and relevant interventions. They may also give advice on how to adapt new practice to local conditions, and to evaluate the implementation and outcomes. The support system may include local resource persons and services, but also academic institutions like universities or colleges. To the extent that schools had been in contact with the universities for training, coordination and evaluation, they reported that the collaboration had been mutually beneficial (Brørup-Dyssegaard et al., 2017). It was also vital that persons from the support system were immediately available when schools needed encouragement or help to solve problems.

Implementation quality is an important mediator of intervention effectiveness and combines adherence or loyalty to the intervention with practitioner competence. Although local adaptation is considered necessary when interventions are implemented at new locations or with new populations, there are different opinions on how much the intervention can be modified before it loses its effect. Evidence-based interventions are rarely implemented as planned, and full compliance between program and practice is rarely expected (Lendrum & Humphrey, 2012). This is often referred to as competent implementation or “flexibility within fidelity” (Kendall & Beidas, 2007). Successful implementation is best served when a central management and control unit is responsive to local needs. But additional obstacles may interfere like lack of time, staff participation in other projects and school meetings, or testing of students and student field trips.

Sustainability. The concept of sustainability signals that the intervention is maintained beyond the active implementation phase, and with the same level of enthusiasm and engagement as originally described. Sustainability is caused by the same factors that contributed to success in the first place. But the sustainability of new practices may be challenged by changes in school policy, or in central or local priorities over which the school has no control. And at the school level, turnover of principals or key personnel may interfere with the maintenance of new practices.

A practical example of how the guideline approach was applied in the implementation of a school-wide project is presented in the next section. This section addresses the process of conceptualization, adaptation, implementation and evaluation of the school-wide positive behavior support model as it was transferred from the US to Norway.

The implementation of the school-wide-positive-behavior support model in Norway

During a visit to the University of Oregon in 2002, a particular version of the School-Wide Positive Behavior Support (SWPBS) model was introduced to staff at the Norwegian Center of Child Behavioral Development. The model was developed by Sprague and Walker (2005) and even if it had been widely implemented in the USA, it had not previously been evaluated. The model’s change theory predicted that all students would benefit from mutually positive relations and positive behavior collectively modeled by staff. The Norwegian adapted version was implemented at four primary schools from 2002 and evaluated with a feasibility study (Sørlie & Ogden, 2007). The schools' expenses were limited to the cost of materials, travel and substitutes. By 2020, approximately 240 primary schools had implemented the model (Borgen et al., 2020). The model is organized with interventions at three levels of increasing intensity, but most of the interventions are placed at the first, which is the universal level. That is: targeting all students at the school. The general idea behind the model is to use preventive interventions in order to keep as many students as possible at the universal level, and if needed with the support from upgraded interventions at the next two levels of the model. The aims of the model are to 1) prevent and reduce acting-out behavior, 2) improve social relations and social competence, 3) improve the school learning climate, 4) increase the use of evidence-based practices and 5) increase inclusion by adapting ordinary education to the needs of all students. The core intervention components were good relationships, school-wide positive expectations and rules, positive teacher directions and instructions, and the encouragement, recognition and positive involvement of students. The model had no academic goals or intervention components, and aimed primarily at improving the pupils' school behavior. Neither was any attempt made to increase the staff's research literacy, but they could access relevant information in handbooks and supplementary literature (Arnesen et al., 2012, 2017; Arnesen et al., 2006).

Informed consent was required by 80 percent of the staff in order to start the implementation. The majority of staff also had to accept the school-wide strategies of long-term commitment, universal training activities, supervision, monitoring of fidelity, evaluation and follow-up activities. The staff was invited to discuss and decide on its collective strengths and limitations, and to develop its own version within the limits defined by the model’s core components. Interestingly, the staff signaled less interest in the research underpinnings of the model than in the how the model worked in practice. The teachers and other staff were eager and impatient to start working with the model. Previous experiences with similar models or programs seemed to be more important than the level of research literacy for the progress of implementation. This indicated that positive outcomes of the N-PALS project could depend more on the quality of the implementation than on the staff's research literacy.

The evaluation and implementation of the N-PALS model

A longitudinal effectiveness study was conducted with a quasi-experimental design (Sørlie & Ogden, 2014). The full version of the N-PALS model was implemented in 28 schools and their student and teacher reported outcomes were compared with outcomes from 20 comparison schools matched on school-size and location. Significant outcomes after three years included:

a) fewer teacher-reported behavior problems within and outside the classroom context, b) improved psycho-social classroom climate, c) fewer students expelled from class due to challenging behavior, d) increased teacher-reported collective efficacy and self-efficacy, and e) increased use of teacher-reported effective discipline practices (Sørlie & Ogden, 2015). Overall, several outcomes of the longitudinal evaluation of the N-PALS model were positive, and effect sizes were comparable to those in other school-based universal prevention studies. The significant Effect Sizes after three years of implementation, as reported by Sørlie and Ogden (2015), were in the range of 0.13 to 0.25. This matches the outcomes of Bradshaw et al. (2012), who reported ES-values of 0.08 and 0.17 based on teacher-assessed student behavior in the United States. The magnitude of the intervention effects on school problem behavior in the Sørlie and Ogden study (2015) also matched the mean Effect size of 0 .20 reported in meta-analyses of universal school wide programs (Durlak et al., 2011; Wilson & Lipsey, 2007).

The importance of implementation quality. The web-based “Effective Behavior Support Self-Assessment Survey” (EBS-SAS) was used to monitor the degree and quality of implementation of the N-PALS (Sugai & Horner, 2009). Measures of treatment integrity determined if active components of the model had been applied, and if the practitioner had been loyal to the intervention’s uniqueness and core components. A survey showed that 80 per cent of the intervention schools had reached the level of acceptable implementation within three years, but greater benefits were achieved for schools with high implementation quality (Sørlie & Ogden, 2015). The fact that the participation in an evaluation study could be positively impacted by the level of implementation became clear in a later register study of all 244 PALS schools. At schools that had not participated in any evaluation only 30 per cent of staff reported that they had reached the required minimum score on the EBS-SAS after 3 years (Borgen et al., 2020). In comparison, 75% of the schools participating in the Sørlie and Ogden (2015) study reached this score.

Summary and comments

In this article, two perspectives on the transfer of educational research to practice have been presented. To a certain extent they are overlapping and have in common the aim of increasing evidence-based practices in school. They differ, however, in the way they describe the relationship between research and practice. Increased focus on and interest in evidence-based practice in school justify the need for a stronger focus on teaching research literacy in education. If research literacy is emphasized in basic teacher training the candidates may influence the openness and readiness of schools and be better prepared for the tasks of developing evidence-based practice. The level of research literacy may also promote the sustainability of effective interventions. The general research competence and capacity of teachers, and the critical assessment of the research base of EBP are central to this approach. Teacher training in research literacy may prepare individual candidates for conducting and contributing to research on their own.

The implementation and guideline approach, on the other hand, seems to be the approach of choice when it comes to the implementation of school-wide evidence-based interventions. In this approach, program-specific competence and capacity are prioritized above research literacy. A stronger focus is directed towards systemic and organizational characteristics of schools, and also on the context of teaching. Common implementation drivers are identified as school leadership, the professional development of staff, school-based attitudes and perceptions, the support system of the school, implementation quality and sustainability. Rather than promoting the research literacy of individual staff, collective competence and the collective commitment to evidence-based practice are emphasized. In order to increase the use of evidence-based practice in schools, both schools and teacher training institutions should be more concerned with implementation quality when new practices are introduced.