Something shifted in K12 educator conversations this past school year. You may have felt it — a low-level hum of anxiety that wasn’t there before. A colleague forwarded a video with a note that said: “Have you seen this?” A school board meeting that took an unexpected turn. A teacher in a professional learning session asked, with genuine worry in her voice, what a researcher’s Senate testimony meant for her second graders.
If you’ve been in those conversations, you already know the name: Dr. Jared Cooney Horvath, whose argument that educational technology actively harms student learning has traveled from a Substack post to a Senate testimony clip viewed over two million times on YouTube to proposed or enacted legislation in sixteen states — all in roughly eighteen months. Whatever you think of his argument, the effect on educator communities has been real. Teachers who have spent years learning to use technology thoughtfully are second-guessing themselves. Coaches who have built their practice on research-based frameworks are being asked to defend ground they didn’t think was contested.
Central to Horvath’s argument is international test score data from the OECD — the Organisation for Economic Co-operation and Development, which administers the PISA assessments used to compare student performance across countries. His claim, stated plainly, is that students who use computers heavily at school perform worse on those assessments, and this is evidence that educational technology harms learning. It sounds like a straightforward reading of the data. It isn’t.
Does Screen Time Actually Hurt Learning? What the OECD Data Really Shows
Here’s what the OECD actually found — in its own analysis of its own data. The 2024 OECD report Managing Screen Time, which draws on PISA 2022 results, distinguishes between using devices for learning and for leisure, a distinction Horvath’s presentation ignores entirely. Students who spend one to five hours per day on digital devices for learning score twenty PISA points higher in mathematics than students who use no devices at all. The negative correlations — the ones Horvath presents as evidence that technology harms learning — are tied to leisure use. Social media. Unsupervised browsing. Entertainment during class time. Not a teacher using an adaptive reading platform with a struggling reader. Not a student working through a purposefully selected math application. The OECD’s own conclusion, in plain language: students who use digital devices moderately for learning tend to perform better and report a greater sense of belonging at school.
There’s one more detail worth knowing. That 2024 report was published in May 2024 — six months before Horvath’s most widely shared piece appeared. The updated data was already available. The distinction the OECD draws between learning use and leisure use was already in the literature. Presenting the OECD data without that distinction, at a time when a huge audience was primed to receive an alarming conclusion, is not a neutral analytical choice. It’s the kind of choice that media literacy teaches us to notice.
Here’s what I think this conversation deserves: not a quick reassurance, and not a dismissal of legitimate questions about technology use in classrooms. It deserves a look at what the research actually shows. Because there is quite a lot of it — decades of peer-reviewed, rigorously designed studies examining whether educational technology improves student learning. And the picture it paints is considerably more nuanced, and considerably more encouraging, than the narrative that’s been traveling at viral speed.
Let’s look at what the evidence actually says.
The Research Base Is Deeper Than You’ve Been Led to Believe
One of the most consistent features of Horvath’s argument is the framing that educational technology simply doesn’t work — that decades of implementation have produced little or no learning benefit. That framing does not hold up to a careful look at the research literature.
Start with a foundational meta-analysis from the North Central Regional Education Laboratory. In 2003, Waxman, Connell, and Gray synthesized findings from 42 studies involving approximately 7,000 students, calculating 282 separate effect sizes across cognitive, affective, and behavioral learning outcomes. Their finding: teaching and learning with technology produces a small but statistically significant positive effect on student outcomes compared to traditional instruction alone — with a mean effect size of 0.41 (p < .001). That held across cognitive outcomes, yes, but also across measures of student engagement and behavior. Technology integration, when studied rigorously, doesn’t just show up in test scores. It shows up in how students engage with learning.
The picture holds when you move to literacy specifically. A 2011 meta-analysis by Cheung and Slavin at Johns Hopkins University examined 85 studies involving more than 60,000 K-12 students, focused squarely on reading achievement. The overall finding was positive — educational technology generally produced meaningful improvement in reading outcomes compared to traditional methods. But Cheung and Slavin found something more specific that’s worth sitting with: the strongest effects weren’t associated with basic computer-assisted drill programs. They were associated with innovative technology applications paired with extensive professional development. The technology mattered. The teacher’s preparation to use it well mattered more.
That finding connects directly to more recent, targeted research. In 2020, Hillmayr, Ziernwald, Reinhold, Hofer, and Reiss published a comprehensive meta-analysis in Computers and Education — a peer-reviewed journal — examining 92 studies of digital tool use in secondary school mathematics and science, covering grades 5 through 13. Their overall effect size: 0.65, statistically significant (p < .001). That’s not a small finding. But the moderating factor that stands out is the same one Cheung and Slavin identified a decade earlier: the provision of teacher training on digital tool use significantly increased the overall effect. When educators understood how to use the tools in the context of their content and their students, the tools worked better. Notably, the effect was larger when digital tools were used alongside other instructional methods — not as a replacement for them.
This brings us to two more recent and more specific studies in this conversation, both of which examine the youngest learners and both of which deserve to be far better known than they currently are.
In 2021, Kim, Gilbert, Yu, and Gale published a meta-analysis in AERA Open — a peer-reviewed journal of the American Educational Research Association — examining 36 rigorous studies of educational apps for children from preschool through grade three. Ninety-two percent of those studies were randomized controlled trials, which is the gold standard in research design. The finding: meaningful positive effects on both literacy and mathematics outcomes for our youngest learners. These weren’t passive screen-time studies. They were studies of intentional, purposefully designed educational applications used in structured learning contexts.
And in 2024, Silverman, Keane, Darling-Hammond, and Khanna published a meta-analysis in the Review of Educational Research — one of the most respected peer-reviewed journals in the field — examining 119 studies of educational technology literacy interventions specifically in K-5 classrooms. Positive effects across reading, comprehension, and writing. Silverman’s framing of what the research actually asks us to do is precise and important: the right question isn’t whether educational technology works. It’s which products, with which characteristics, under which conditions produce meaningful learning. That’s not a defense of every app on the market. It’s a call for the kind of careful, evidence-informed selectivity that good technology coaching has always been about.
Five meta-analyses. Preschool through high school. Literacy and STEM. Consistent positive effects — strongest when teachers are prepared, when tools are chosen thoughtfully, when technology complements rather than replaces good instruction.
That is the type of research base Horvath’s argument bypasses.
The Frameworks Your Practice Is Built On
Here’s something worth naming directly: the field of educational technology has been doing serious, peer-reviewed work on the question of how to use technology effectively for decades. Two frameworks in particular represent that work, and if you’ve been integrating technology with any intentionality, you’ve been working within their logic — whether you used those names or not.
The TPACK Framework: Why Technology Knowledge Isn’t Enough
TPACK — Technological Pedagogical Content Knowledge — emerged from the research of Mishra and Koehler in the early 2000s and has since accumulated a substantial body of peer-reviewed study across content areas and grade levels. The framework’s core insight is both elegant and practical: technology knowledge becomes meaningful in a classroom only when it is genuinely integrated with a deep understanding of the content being taught and the pedagogical strategies that serve that content and those students. A teacher who knows what she’s teaching, knows who she’s teaching it to, and knows how a specific tool can serve both — that is categorically different from undirected screen time. TPACK gives us the language to articulate exactly why intentional, content-driven technology use is not the same as parking students in front of a screen without purpose or guidance. Much of the current policy debate treats those two practices as equivalent. The research does not support that.
The Triple E Framework: A Practical Classroom Test for Every Edtech Tool
Dr. Liz Kolb’s Triple E Framework, developed at the University of Michigan and validated by independent research as both reliable and valid, adds a practical classroom-level lens. It asks three questions about any technology-integrated lesson: Does the technology meaningfully engage students with the learning goal — not distract from it, not entertain alongside it, but genuinely engage with it? Does it enhance student understanding in ways that would be harder to achieve without it? Does it extend learning beyond what the classroom and the school day could otherwise provide? The framework’s organizing principle is four words: learning first, technology second. Not technology always. Not technology instead of teaching. The question the framework asks is always whether the tool is serving the learning goal — and if it isn’t, you choose a different tool, or no tool at all.
These aren’t aspirational documents. They’re peer-reviewed, widely implemented, and grounded in exactly the kind of research that shows up in the five meta-analyses above. When the coalition of seventeen education and library organizations — including the American Federation of Teachers, the National Education Association, AASA, NAESP, NASSP, and the American Library Association — wrote to the Senate Commerce Committee two days before Horvath testified, they made precisely the distinction that TPACK and Triple E are built around. They asked lawmakers to “distinguish between largely unsupervised, entertainment-driven technology use at home and the intentional, monitored, and carefully curated use of technology in schools.” That is TPACK in plain English. It’s Triple E with the academic framing removed. It’s what forty years of coaching has taught me to recognize when I see it in a classroom.
What This Means for Your Classroom and Your Coaching
The anxiety circulating in educator communities right now is understandable. A message arrived with impressive packaging, at a moment when educators were already tired and already uncertain, and it spread very quickly. Some of what Horvath raises about distraction, about undirected screen time, about the cognitive conditions that support deep learning is grounded in legitimate neuroscience and worth taking seriously. The concern that technology has sometimes been adopted too quickly and implemented poorly is real and widely shared among thoughtful people in this field.
But the leap from those legitimate concerns to the conclusion that educational technology should be removed from students’ hands — a leap Horvath makes based on evidence that, read in full and in context, does not support that conclusion — is not one the research justifies. The five meta-analyses above don’t prove that every piece of educational technology in every classroom works well. No honest researcher would claim that. What they show, consistently, is that technology integrated with intentionality, with appropriate teacher preparation, with clear learning goals, and with the right tools chosen for the right purposes — produces meaningful positive outcomes for students from preschool through high school, across literacy and STEM, across a wide range of contexts and demographics.
If you are making deliberate decisions about when and why technology serves your instructional goals — if you’re asking whether a tool engages, enhances, and extends learning before you put it in front of students — the research is on your side. The OECD is on your side. The coalition of professional organizations that represents your colleagues is on your side. You don’t have to navigate this alone. Edtech professional development grounded in these frameworks, not vendor demos or sit-and-get sessions, is what OK2Ask has been designed to provide.
The teacher who raised her hand in our OK2Ask session to ask what Horvath’s testimony meant for her second graders was doing the right thing. She paused. She asked. She looked for resources before she acted. That instinct — to examine a claim before accepting it, to seek evidence before changing practice — is exactly what good teaching, and good technology coaching, and good media literacy all require.
The research on educational technology has been doing the same thing for decades. It’s time more of us knew what it found.
Resources and References
Frameworks
- Mishra, P., & Koehler, M.J. (2006). Technological Pedagogical Content Knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. | tpack.org
- Kolb, L. (2017). Learning First, Technology Second. ISTE. | tripleeframework.com
Research cited
- Waxman, H.C., Connell, M.L., & Gray, J. (2003). A Meta-Analysis of the Effectiveness of Teaching and Learning With Technology on Student Outcomes. North Central Regional Educational Laboratory.
- Cheung, A.C.K., & Slavin, R.E. (2011). The Effectiveness of Education Technology for Enhancing Reading Achievement: A Meta-Analysis. Center for Research and Reform in Education, Johns Hopkins University. ED527572
- Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S.I., & Reiss, K.M. (2020). The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers and Education, 153, 103897. https://doi.org/10.1016/j.compedu.2020.103897
- Kim, J., Gilbert, J., Yu, Q., & Gale, C. (2021). Measures matter: A meta-analysis of the effects of educational apps on preschool to grade 3 children’s literacy and math skills. AERA Open, 7. https://journals.sagepub.com/doi/full/10.1177/23328584211004183
- Silverman, R.D., Keane, K., Darling-Hammond, E., & Khanna, S. (2024). The effects of educational technology interventions on literacy in elementary school: A meta-analysis. Review of Educational Research, 95, 972–1012. https://doi.org/10.3102/00346543241261073
- OECD (2024, May). Managing screen time: How to protect and equip students against distraction. PISA in Focus, No. 124.
The full case study this column draws from:
- Okoye, R. (2026). “A Media Literacy Lesson Hiding in Plain Sight.”
One thought on “Let’s Talk About What the Research on K12 EdTech Actually Shows”