There’s a conversation happening in schools right now, and you’ve probably been part of it. Parents are pushing back on screen time. Administrators are eyeing the edtech budget with fresh skepticism. Teachers are quietly abandoning platforms they were trained on two years ago. And somewhere in the background, a tool that was supposed to transform learning is sitting unused on a Chromebook shelf.
The frustration is real. And honestly? Some of it is earned. But before we declare technology itself the problem, a coalition of seventeen national education organizations — including the NEA, AFT, and AASA — asked policymakers to distinguish between unsupervised, entertainment-driven screen time at home and the intentional, monitored use of technology in classrooms. That distinction matters enormously. And it’s one that often gets lost when the loudest arguments move faster than the evidence.
So here’s the harder question: Did we actually give it a fair shot?
You Did Your Homework
If you’re reading this, there’s a good chance you — or someone in your building — did things right. You looked at the evidence. You asked vendors the uncomfortable questions. You may have even checked whether a program aligned with the ESSA tiers of evidence, the federal framework that rates educational programs from Tier 1 (strong evidence from randomized controlled trials) down to Tier 4 (research-based rationale, study in progress). That kind of intentional procurement is exactly what good practice looks like.
Think of it like a prescription. When a doctor writes you a prednisone pack, the directions are specific: take six pills the first day, five the next, tapering down over a week. The dosage, the timing, the sequence — all of it matters. Skip days, cut it short, or take it whenever you feel like it, and you can’t expect the same outcome that produced those results in the first place. The medication isn’t the problem. The protocol is. In implementation science, this idea has a name: implementation fidelity — the degree to which a program is used as it was designed.
Edtech — and really any evidence-based intervention — works the same way. The Wing Institute describes implementation fidelity as a multi-dimensional construct that includes adherence to the program design, the quality of delivery, the dosage received, and the responsiveness to the learners in front of you. The evidence rating a program earned was based on all of those conditions being present. When we change those conditions, we change the outcome.
But here’s the part that doesn’t show up in the sales pitch: those conditions are typically created at the classroom and school level. And that’s where things tend to quietly fall apart.
The Gap Nobody Talks About
In their influential review, researchers Heather Hill and Anna Erickson found that poor program implementation is among the most common explanations for disappointing results in education programs — and it is chronically underreported, because most evaluations don’t collect the data needed to diagnose it. We see a tool “fail” without ever knowing whether it was actually implemented as designed.
A 2017 EdSurge analysis put it plainly: when researchers evaluate educational technology, they often ignore the implementation environment entirely — treating a program used at 30% of its intended dosage the same as one used fully and consistently. These researchers compared it to studying whether a medication works without checking whether patients actually took it as directed.
This is where the gap opens up. The gap doesn’t come from lack of effort — it comes from lack of visibility. Without some form of ongoing, reflective support — whether that’s a dedicated instructional coach, a professional learning community, a community of practice with colleagues, or sustained professional development over time — there is often no reliable way to know where your implementation has drifted from the intended design. The program looks like it’s running. It might even feel like it’s running. But without an external reference point, implementation drift is invisible from the inside.
The feedback loop doesn’t have to be formal or expensive. It does have to be consistent. A PLC that regularly examines student work through the lens of a shared tool. A community of practice where teachers compare notes on what’s working and what isn’t. Even a committed partner teacher who observes and debriefs with you twice a semester. What matters is that someone — or some structure — is helping you see what you can’t see on your own.
The 2–5 Year Reality Check
Here’s a number that tends to stop people in their tracks: research from the National Implementation Research Network at UNC consistently finds that it takes two to five years to achieve full implementation of an evidence-based program or practice. In practical terms, that means the first year is often about learning the system, the second about stabilizing routines, and only after that do you begin to see consistent impact. Two to five years — not two to five weeks, not a semester, not the time between back-to-school PD and winter break.
Most edtech adoption cycles don’t come close. A tool gets introduced in August, assessed by February, and quietly shelved before the following school year. And the difficult truth is that this often happens right at the inflection point — the moment when implementation is starting to stabilize, when teachers are building the muscle memory that makes a new practice feel natural rather than effortful.
Sustained support structures aren’t extras layered on top of implementation. They are the mechanism by which teachers calibrate their practice over time. A single PD day can introduce a tool. It cannot sustain fidelity across two school years. Without a professional learning community, ongoing workshops, or some form of continued reflection, implementation naturally drifts — and when outcomes disappoint, the tool takes the blame for conditions it never had a chance to meet.
When Time Gets Cut, Fidelity Goes First
A research review about the relationship between implementation and outcomes confirms what many teachers already sense: dosage — the actual time and frequency with which a program is used as intended — is a critical variable in whether that program produces results. The relationship isn’t always simple or linear, but what’s consistent across studies is this: when dosage drops significantly below the program’s intended design, the evidentiary foundation for expecting results drops with it.
This finding has a painful mirror image in the real world. A reading platform designed for 20 minutes of daily use becomes a twice-a-week rotation during testing season. Schedules shift — a new initiative lands, testing season compresses the calendar, a sub doesn’t know how to run the platform — the technology is usually the first thing to get dropped. Nobody announces it. No one logs it. It doesn’t show up in evaluation notes. It just… stops happening. And months later, when the data looks flat, the conversation turns to whether the tool is worth renewing.
The tool didn’t fail. We just didn’t give it long enough. Think of a low-and-slow pork roast — you can’t pull it at the two-hour mark and declare the recipe a failure. The meat doesn’t fall off the bone until it’s been in there long enough, at the right temperature, without interruption. That’s not a flaw in the method. That is the method. And when we make implementation decisions on a schedule that has nothing to do with what the research actually requires, we shouldn’t be surprised when the results don’t show up.
A Better Question for Next Year’s Planning
As you head into summer and start thinking about next year, try bringing a different question to your planning conversations — not “did this tool work?” but “did we implement it as designed, for long enough, with enough support to actually know?”
That question also reframes what it means to sharpen your own practice. Fidelity isn’t about following a script — it’s about understanding a program’s instructional intent well enough to make smart adaptations without losing what makes it work. Teachers who have that understanding can look at any tool and ask not just “am I using it?” but “am I using it in the way most likely to help my students?” That’s not a technology question. That’s a teaching question.
TeachersFirst Can Help
If you’re looking for a place to do that reflective work alongside other educators this summer, we’d love to have you join us.
TeachersFirst is offering a free six-week Core Instructional Practices series as part of our Summer Professional Learning opportunities. These Wednesday evening virtual sessions are designed to help you revisit, sharpen, and recalibrate the instructional strategies that anchor good teaching — the ones that hold up whether your students are working with technology or without it:
- Formative Assessment — knowing what your students actually know, in the moment
- Graphic Organizers — supporting thinking, not just decorating it
- Classroom Routines — the structures that make everything else possible
- Interleaving — spacing and mixing practice for deeper retention
- Vocabulary Instruction — building the language that unlocks content
- Structured Talk Activities — turning student conversation into learning
Can’t make it live? Every session will be recorded and available on demand. Whether you join us on a Wednesday evening or catch up on your own schedule, these workshops will give you a chance to strengthen the instructional core that makes implementation fidelity possible.
Browse the full registration catalog and sign up here: Summer Professional Learning
The technology isn’t the enemy. But it does need a fair fight — adequate time, sustained support, and the kind of reflective practice that comes from doing the ongoing work of growing as a teacher. We hope to see you this summer.