Background

“Surely this time, the ‘industrial revolution in educationʼ that Sidney Pressey had predicted decades before was about to come to pass. Surely this time, things would be different” (Watters, 2021, p. 33). The notion that technology can (and should) reform education is nothing new. In her book, Audrey Watters maps the history of personalized learning, and the preceding quote underlines the initial attempts of B. F. Skinner to develop and commercialize a teaching machine in the first half of the 1950s. Evidently, Skinnerʼs teaching machine and subsequent innovations to improve and personalize learning has failed to achieve a revolution in education. Rather, the educational system has repeatedly adopted new technologies, generally driven by a desire to keep up with technological advancements and to meet societal expectations (Selvyn, 2024).

The implementation of new technologies in education has followed a predictable cycle of romanticism (Bigum, 2012), technological determinism (Oliver, 2011) and moral technology panics (Orben, 2020), and the pattern can currently be observed through two separate yet related discourses. At one end there is growing concern about the risks associated with increased screen time, which contrasts sharply with earlier educational policies that promoted a 1:1 ratio of digital devices for students. At the other end, there is an ongoing debate about the popularization of so-called artificial intelligence, and how the educational system should respond to these emergent technologies. Both discussions reflect a strong assumption that technology “possess inherent qualities and are therefore capable of having particular ‘impactsʼ or ‘effectsʼ on learners, teachers and educational institutions” (Selwyn, 2010, p. 68).

The discourse is further complicated as the study of education in Norway has followed an uneasy truce between European-Continental and Anglo-American academic traditions (Biesta, 2011, 2013). The European-Continental tradition understands education as a distinct academic discipline: pedagogy (Pädagogik), deeply rooted within normative and humanistic aspects of pedagogical theories often associated with Bildung (Sjöström & Eilks, 2020). In contrast, In the Anglo-American tradition, education is generally seen as an interdisciplinary field that can be studied through the lens of psychology, sociology, history, and philosophy, and where theories from psychology have especially been dominant (Sæverot, 2014; Sæverot & Torgersen, 2012; Siegel & Matthes, 2022).

In this paper, I argue that the prevailing tensions between these two traditions have been disproportionately skewed in favor of my own field, educational psychology. To substantiate this claim, I will examine two cases: 1) an extensive literature review on the digitization of primary education in Norway (Munthe et al., 2022a) funded by the Directorate for Education and Training, and 2) a recent Norwegian Official Report on learning analytics and adaptive learning (Kunnskapsdepartementet, 2022; NOU 2023: 19, 2023) requested by the Ministry of Education and Research. I will discuss these in the context of (digital) Bildung. The focal point of the discussion will be the relationship between theory of learning and technology. Furthermore, considering the current climate, the discussion on technology will primarily address transformer-based large language models (LLMs) and exclude other architectures (e.g., convolutional neural networks). This emphasis is due to the prominent role of LLMs in public discourse and the early adoption of such models in educational settings. However, the perspective of Bildung remains relevant irrespective of the technology discussed.

Where are we going and why?

Applied research on technology in education faces two major, and often conflicting challenges: staying relevant and identifying the perceived and actual affordances of technology. The development of (digital) technologies is by its very nature fast-paced and thus the research tends to be directed at a moving target (Krumsvik, 2023a). Therefore, since conducting high-quality research requires significant time and resources, and because the traditional peer-review process also prolongs publication, studies often lag behind current societal and policy discourses. On the other hand, assessing the affordances of the latest technology is challenging due to a scarcity of high-quality studies. The distinction between perceived and actual affordances is important, though the term itself is not uncontroversial (Oliver, 2005, 2013). In this discussion affordance is understood as “studying existing repertoires of use and recognising the potential of analogous or newly imagined applications” (Oliver, 2005, p. 411). That is, the affordance of any technology is created through the interpretations and interactions of users, and it is at best unhelpful to assume that technology directly causes learning without theorizing how technology and its use are embedded within social contexts.

The consequence of the two challenges is that implementing new technologies, as Bigum (2012) claims, may result in “a cycle of identifying, buying, and domesticating the ‘new best thingʼ driven largely by claims that the process is ultimately improving learning” (p. 23). In the following, I will provide two cases to further illustrate the issue.

Relevance

The first challenge is evident in the latest systematic review on the digitization of Norwegian primary education (Munthe et al., 2022a). The study is comprehensive, encompassing four main work packages in addition to dissemination of findings: 1) Examining the concept of digitization in educational policy. 2) A scoping review of more than 300 systematic reviews and meta-analyses, based on approximately 8,000 primary studies, to assess the impact of 1:1 ratio of digital tools in school, the impact of digital resources and technology, and the digital competences of different stakeholders in education. 3) Conducting a survey among teachers and school leaders, focusing on key experiences with the adoption of educational technology, the support systems in place, and the critical aspects and challenges of implementing educational technology. 4) Identifying future research needs, appropriate research designs, and emerging technological trends, based on findings in the preceding work packages, and through interviews with both international and national experts in the field.

The empirical data encompasses studies on a multitude of technologies, ranging from virtual, augmented, and mixed reality to (serious) games, multimedia (such as podcasts and multimodal resources), and mobile applications. Although Munthe et al. (2022a, p. 116) briefly discuss machine learning, artificial intelligence, and learning analytics as key trends highlighted by a majority of the interviewed experts, the field of adaptive learning (e.g., VanLehn, 2011) is generally ignored in the review. Munthe et al. (2022a, p. 107) state that 84% of primary schools in their survey use adaptive learning software; however, neither learning analytics nor adaptive learning are included in their systematic search strategy (Munthe et al., 2022b, p. 53). This omission represents a significant research gap in the review and is somewhat surprising given the mandate of the study. Importantly, the review was published in the fall of 2022, at the same time as the first easily available LLMs were accessible to the public. The study serves as a clear reminder that all research, regardless of its rigor and comprehensiveness, is constrained by its temporal setting. The authors cannot be faulted for the fact that, in the interim period, public and political discourse has focused almost exclusively on the perceived threats and benefits of large language models.

Affordance

The second challenge influence the Norwegian Official Report (NOU 2023: 19, 2023). The goal of the report is to “provide the Ministry of Education and Research with a better foundation for decision-making regarding learning analytics, adaptive learning resources, exams, and tests … and recommend necessary regulations and actions to inform policy development by the Ministry of Education and Research and its subordinate agencies” (Kunnskapsdepartementet, 2022, p. 9). The mandate is broad, and the report therefore covers several crucial aspects such as potential biases in data, commercialism of education, platformization and data privacy. The report amounts to four recommendations: 1) Clarify the legal basis for using learning analytics in education, particularly for handling personal data. 2) Develop a standard for data privacy in primary education. 3) Establish frameworks for good learning analysis in primary education, to enhance the freedom of choice for stakeholders and to provide a better basis for pedagogical decisions to promote learning. 4) Develop comprehensive guidelines for ethical and responsible learning analytics in higher education and vocational training that promotes learning and improve the quality of education (NOU 2023: 19, 2023, p. 107).

The recommendations provide clear indication that the ethical and judicial aspects of adaptive learning and learning analytics are of special concern. However, with few exceptions (e.g., Johler & Krumsvik, 2022; Moltudal et al., 2020), little research has been conducted on the pedagogical use of adaptive learning in real-life practices. Like the Greek proverb “one swallow does not a spring make” so is it true that “individual studies do not policy make.” The expert group observes that there is little systematic knowledge concerning the use of learning analytics: “We do not know if learning analytics actually impacts further learning” (NOU 2023: 19, 2023, p. 10). They further note the terms “learning analytics” and “adaptivity” are used and understood in numerous ways, and that the former term is hardly used in the educational sector at all (p. 20). Simply put, “there is little systematic research on learning analytics in the actual pedagogical practice at all levels of the education system” (p. 27). Finally, and unsurprisingly, the survey (Lenz et al., 2023) conducted on behalf of the expert group provides clear indication that educators and educational leaders in general do not have a clear conceptual understanding of either learning analytics or adaptive learning.

Haugsbakk and Klausen (In press) comment in their review of policy documents on adaptive learning that the expert group provide a more nuanced approach to learning analytics than policy documents in general. However, it is fairly evident that the recommendations of the expert group are based more on perceived than empirically founded affordances. Similarly, as part of their input to the expert group, Universities Norway emphasizes the fact that learning analytics is (even) less prevalent in higher education than in primary education “may be due both to a lack of knowledge and access to resources and opportunities, but also to traditions and culture” (NOU 2023: 19, 2023, p. 28). However, the lack of use does not impair the ability of Universities Norway to conclude that “learning analytics offers many opportunities to gain more knowledge about what facilitates good learning” (p. 31). The general notion is that “surely this time, things will be different.” The only conclusion is that more research is needed before policies can be made.

However, my main apprehension with these two cases is not lack of foresight or a wavering empirical foundation, it is poorly defined concepts of Bildung and the lacking theoretical perspectives.

Machines as a theory of teaching

Besides sections on motivational theory, there are few references to theory in Munthe et al. (2022a) and the Norwegian Official Report (NOU 2023: 19, 2023). Munthe et al. (2022a) state:

Several researchers in our sample have noted the absence of learning theory as a foundation in primary studies, or pointed out the excessive variation in theoretical perspectives. Learning theories are crucial for interpreting results. Therefore, it is essential for teachers to understand this, enabling them to assess the relevance of these findings for their own practice. (p. 114)

Though motivational theory is the bread and butter of educational psychologists, such theories are not enough to facilitate learning. Other studies have indicated that perspectives from cognitive psychological are dominant in research on emergent technologies in education. Ludvigsen et al. (2023, p. 14) make a reference to Zawacki-Richter et al. (2019), noting that in the top three journals within AI in education, contributions are frequently weakly theory-based or lack theoretical grounding. Ludvigsen et al. state that without a solid foundation in learning theory and empirical analyses, it is not possible to develop cumulative knowledge concerning emergent technologies within educational settings. Studying and analyzing new technology is not sufficient; it must be linked to research on learning using multiple sources and lenses of knowledge (Krumsvik, 2020). As Richter et al. (2019) state: “The lack of theory might be a syndrome within the field of educational technology in general” (p. 22).

An interesting aspect of the Norwegian Official Report is that the expert group in the first report (Kunnskapsdepartementet, 2022) mentions “learning design” twenty times, whereas the final report (NOU 2023: 19, 2023) has zero mentions. The expert groups states (Kunnskapsdepartementet, 2022, p. 41) that a learning design is a pedagogical plan to facilitate learning, and that the initial starting point should be a justification for a learning theory informing the design. Furthermore, a central aspect of learning analytics is to evaluate the learning design with the goal of improving it. Next, the expert group depicts a flowchart (Sclater, 2017, p. 67, cited in Kunnskapsdepartementet, 2022, p. 42) that is basically Skinnerʼs teaching machine. The expert group continues: “without theoretical grounding of learning analytics and contextual interpretation of the collected data, learning analytics design capabilities are limited. From this perspective, learning design is utterly important as it provides the framework for analyzing and interpreting data, learnerʼs behavior, and successful or inefficient learning patterns” (Mangaroska & Giannakos, 2019, p. 516, cited in Kunnskapsdepartementet, 2022, p. 43). From this perspective, the general notions are monitoring, efficiency, maximizing, and loss of freedom. The logical “theoretical grounding” is behaviorism (Erstad, 2022; Løvlie, 2022; Stenliden & Sperling, 2024), and the expert group is quite aware of this (NOU 2023: 19, 2023, p. 131).

Bildung in the age of large language models

Content validity is a fundamental concept in psychometrics, and refers to the degree to which an assessment instrument accurately reflects the construct or phenomenon it is intended to measure (Haynes et al., 1995), in other words, whether all essential aspects of a phenomenon are accounted for. In education, as in other sciences, it is important to have discussions on the definitions of theoretical constructs. The concept of (digital) Bildung is terribly neglected, often reduced to an instrumental understanding of basic digital competence, online etiquette, and critical judgments of factual information sources (Gran, 2018; Gran et al., 2019).

The reductive understanding of Bildung is easily identified in the review by Munthe et al. (2022a, p. 136), where the term is deemed equivalent to “digital judgment,” which encompasses basic ethical and moral reflections and choices, including appropriate online behavior and privacy considerations, that is, safely navigating the internet, managing personal information, respecting copyright laws, and developing strategies for effective digital and online use. These facets are undeniably important features of Bildung, especially during times of crisis in an age where multimodal disinformation can be easily generated through LLMs in vast quantities (Krumsvik, 2023b, p. 257). However, it is somewhat peculiar that Munthe et al. (2022a) do not refer to the definition of digital Bildung proposed by Kelentrić et al. (2024), even though this framework was described earlier in their review.

Digital Bildung is a process where an individual shapes their identity in a digital context. It involves actively developing oneʼs social, cultural, practical, and ethical competence in interaction with digital environments, and being able to relate oneʼs digital experiences to the surrounding world. It also involves personal maturation that enables the individual to act in accordance with social expectations and ethical norms in a digital culture, as well as to reflect critically and make well-considered and independent decisions. (Kelentrić et al., 2024, p. 14)

This definition encompasses many aspects of Bildung. Still, an important aspect is then overlooked: the overall relationship with society, and more specifically, the “mutual dialectical dependence between subject and society” (Jobst, 2023, pp. 279–280). Society shapes the individual, and the individual shapes society within a “constant, unfinished process of change” (p. 285). This process can be liberating or repressive given the situated practice. Thus, personal maturation should also enable the individual to think critically, and make well-considered and independent decisions concerning social expectations and ethical norms, not just act in accordance with them.

Mølstad (2023) argues that the contemporary Norwegian educational system is clearly shaped through outcome-oriented and assessment reforms, though the Bildung tradition still can be found in the curriculum. The students are intended “to develop both social skills and gain knowledge from the subjects in such a way as to become members of the larger society and participate in democratic processes” (p. 502). Fauskevåg (2022) points out that while the curriculum extensively discusses the concept of competence, it does not equally clarify or define the concept of Bildung. There is a lack of discussion on what Bildung really entails within the curriculum, which makes it difficult to implement in practice (Fauskevåg, 2022, p. 111). Similarly, the expert group (NOU 2023: 19, 2023, p. 18) states that it is essential to assess learning analytics in light of educational systems mission of education and Bildung. However, apart from mentioning that the students should “reflect on oneʼs own learning and understand oneʼs own learning processes” (NOU 2023: 19, 2023, p. 65), it is not obvious how learning analytics can be understood in light of Bildung.

The question then becomes how to foster (digital) Bildung considering emergent technologies. Inspired by Qvortrup (2022), I envision a dialog-based seminar based on engagement with LLMs (see also Krumsvik, 2023b; Ludvigsen et al., 2023).

Basic understanding

Pioneering works in the mid-20th century laid the foundational theories and computational strategies, and the study of artificial intelligence has a long, and somewhat fragmented history, marked a cycle of discovery, critique, innovation, and skepticism (Nilsson, 2003). There is a need to have a basic knowledge of the affordances of a given technology. For instance, LLMs are (sophisticated) text generators that predict the probability of word sequences. These models apply machine learning techniques, where the final probability estimates for the next word in a sequence are derived using simple softmax functions (Vaswani et al., 2017). The softmax function transforms a list of raw scores from a model into a list of probabilities between 0 and 1, where 0 indicates no likelihood of a category (e.g., word) being chosen, and 1 indicates complete certainty. However, as the primary function of LLMs is to emulate convincing natural language, current LLMs lack the capability to explicitly inform users about the estimated prediction, and the uncertainty associated with this estimate. The complexity of these models is determined by their number, typically in the billions (or trillions) of parameters, and the text data they are trained on. For example, if an LLM is asked, “What is the capital of Norway?”, the accuracy of its answer depends on the training data, and though the LLM will produce an answer, the user will not be informed of the uncertainty associated with the result. In other words, the user is highly dependent on their own prior knowledge to evaluate the response from the LLM. To accurately assess the capabilities and reliability of an LLM, external evaluation is necessary (e.g., Chiang et al., 2024).

Exploring the possibilities and limitations

Importantly, LLMs are not a panacea to the many limitations of the educational system today. For instance, LLMs are not calculators; they are not designed to do math (Conway, 2023; Marcus, 2023), though they may through guidance break down and explain mathematical or statistical problems. Yet, many of the subscription-based engines (e.g., GTP 4) provide integration with third-party software (e.g., python or WolframAlpha) to conduct mathematical operations. Non-subscribers (e.g., using GPT 3.5) may ask the LLM to produce, for instance, python code to compute the answers, but the model does not directly provide the end result.

Students may explore different LLMs, assuming that privacy concerns are addressed. The exploration should include both commercially available models and the plethora of publicly available models of different sizes (e.g., Hugging Face, Inc., 2016). They can also explore differences in contextual responses, as the AI-lab at the National Library of Norway, and Ruter Norway, host their models there. They can learn how to implement these models on local machines, and how to engineer the prompt formats needed to generate content. Then, using systematic prompting techniques, such as in-context prompting (Krumsvik, 2023b, p. 77), students can explore the differences between models, and how size and training data affect the responses of the LLMs. The topic of the seminar can be tailored to a specific academic discipline, such as language studies, and the prompts should involve both factual questions and condensation of text (e.g., to create keywords or abstracts). This can facilitate understanding of the limitations of the models, and provide a better conceptual understanding of the technology.

Discussing the implications for the individual and society

Before attending, and during the stages of the seminar, students are likely to hold and develop certain ideas concerning moral, ethical and judicial issues, such as sustainability (De Vries, 2023), copyright infringement and plagiarism (Klosek & Blumenthal, 2024; Marr, 2023), bias and fairness (Erstad et al., 2023; Gallegos et al., 2023), platformization (Cobo & Rivas, 2023), and data privacy (Komljenovic, 2022). The seminar offers an opportunity to discuss these topics both on personal and societal levels, by examining them in the context of laws, social norms, and individual responsibilities. Additionally, if possible, students could tour the site where the university hosts its servers. This would provide a more tangible understanding of the environmental impacts between models hosted on personal laptops, and the commercial models that require warehouse-sized servers, even though the technology is the same.

The seminar could also be used to discuss LLMs in the context of societal issues. For instance, considering the ongoing trend of developing and commercializing LLMs, what impact could less powerful and possibly ad-sponsored models, as compared to paid and more capable models, have on the reproduction of social inequalities (cf. Jobst, 2023)? What is the relationship between Bildung, digital literacy and the use of such models in academic settings (cf. Abbas et al., 2024)?

Concluding thoughts

As I have argued in this paper, an excessive influence of educational psychology may have detrimental effects on policy, potentially leading to an increase in technological determinism and the notion that “machines are a theory of teaching” (cf. Watters, 2021, p. 255). Though, as a proponent of psychological theories and methods of study, I do not believe perspectives from psychology should be alien to the study of education. However, I am strongly convinced that to understand complex phenomena, such as education, one needs multifaceted and diverse theoretical and methodological approaches. Consequently, while I firmly believe that basic experimental research is needed to improve both teacher education and education in general, I contend that a balance is needed between pedagogy and education, and between European-Continental and Anglo-American academic traditions. To provide a bridge between these two traditions, I advocate for a metaparadigm dubbed dialectical pluralism, which is generally associated with mixed (methods) research. This way of thinking aims “to work together constructively and thrive on the many important differences and natural tensions” (Johnson, 2024, p. 101). The basic premise is that multiple perspectives are needed to inform research, policy, and public discourse, as well as the various stakeholders who have a vested interest in the phenomenon.

Thus, while I cautiously consider myself a technological optimist, I do not think it wise to take a happy-go-lucky and stumbling approach into the future. My main research interest will continue to be basic educational research, and the development of experimental designs focused on the study of motivation and creativity within educational settings. However, I strongly recommend a call for Bildung.