AI, early years outcomes and the danger of plausible nonsense !

Recently, I was reminded of something that has been troubling me increasingly as artificial intelligence becomes more widely used in education.

AI can analyse data. It can identify patterns, produce beautiful reports, compare percentages and generate recommendations in seconds. It can make complex information look incredibly accessible, and I can certainly see the attraction for busy school leaders.

But there is a problem. Just because an explanation sounds plausible doesn’t mean it’s right. And when we are talking about young children, getting it wrong matters.

When data becomes a diagnosis

In preparation for a school visit, I was sent some data by a headteacher who had been given a data analysis by a company who had used the schools own data set.  The data highlighted some areas where attainment was below national figures, including writing, number, and listening, attention and understanding. There were also differences in outcomes between boys and girls.

Nothing particularly unusual there. Schools examine these sorts of differences every year, and quite rightly so.

What concerned me was what happened next.

The report had moved beyond identifying differences in attainment and had begun explaining why those differences existed. It then offered three neat hypotheses, each accompanied by an equally neat solution for the school to implement. I must admit, when I read the suggestions, I wondered how much the person who had written the report understood child development. Then I realised that a person hadn’t written the report, it had been generated by Artificial Intelligence.

The first suggestion was that difficulties with listening and attention were restricting children’s attainment in mathematics and literacy. The recommended response was daily structured listening sessions, explicitly teaching children to sit still, make eye contact and demonstrate ‘good listening’.

The second linked boys’ lower writing outcomes to difficulties with fine motor control. The solution was a fine motor gym, daily worksheets and activities specifically designed to appeal to boys.

The third suggested that boys were not sufficiently engaged because the learning environment wasn’t meeting their particular ‘male learning styles’. The proposed solution was more structured learning in the outdoor provision, with directed activities in early years rather than allowing children to “just play”.

On the surface, it might  sound quite reasonable and plausible if you don’t know about child development, and it was presented as absolute fact. There were percentages, comparisons, confident hypotheses and recommended actions.

But I found myself asking a very simple question.

How do we know any of this is true?

I know the school and I’ve observed the children the report was referencing, and this just didn’t feel right to me.

Correlation isn’t causation

Let’s take listening and attention. If fewer children achieve the expected level in listening, attention and understanding than in speaking, does that mean that difficulties paying attention are responsible for lower outcomes in mathematics?

No. yet this was the conclusion that AI had drawn. It might be a possibility worth investigating, but the data certainly doesn’t establish it.  We don’t even know whether the children who didn’t achieve the listening, attention and understanding goal are the same children who didn’t achieve the number goal and even if they were, that wouldn’t tell us why.

A child might find it difficult to understand complex spoken instructions. Another might struggle with unfamiliar vocabulary. A third might have difficulty processing language in a noisy classroom. Another might be perfectly capable of sustained attention while building an elaborate construction outdoors but find an adult-directed carpet session difficult.

These children have very different needs.

Teaching all of them to sit still and make eye contact would be a remarkably blunt response to a problem we haven’t actually identified. In fact, I would argue that it risks teaching children to perform the appearance of listening rather than developing their understanding. And surely we know better than this?

The EYFS self-regulation early learning goal explicitly recognises that children can give focused attention and respond appropriately even when engaged in an activity. The Department for Education’s guidance also recognises that fidgeting and needing support are developmentally appropriate. The National Autistic Society reminds us that eye contact can be uncomfortable or distressing for some autistic children and should not be insisted upon.

Yet here we have a recommendation which potentially turns compliance into a proxy for learning.

Being still isn’t the same as listening. I think we’ve all taught children who we thought were listening who were simply sitting well. Looking at somebody isn’t the same as understanding them and a quiet child isn’t necessarily a learning child.

And what about the boys?

I was equally troubled by the assumptions about boys. The analysis identified lower outcomes for boys in some areas and concluded that the learning environment might not be meeting their particular learning styles. But what evidence was there for this? Boys are not a homogeneous group. Neither are girls.

Some children love large-scale construction. Some enjoy stories and imaginative play. Some need plenty of movement. Some prefer quieter spaces. Their interests, experiences, developmental pathways and ways of communicating vary enormously.

We should be responding to individual children, not constructing interventions around assumptions about what boys supposedly need. In this particular setting, previous observations had shown children who were engaged, interested, cooperative and eager to learn. There were strong relationships and purposeful play.

That evidence matters too. If an analysis suggests that boys are disengaged, but the adults who know the children and the observations of their learning tell a different story, we need to  investigate the discrepancy. We shouldn’t simply accept the explanation because it arrived in a professional-looking report. But the report presented this conclusion as an issue that needed to be addressed by overhauling the curriculum and provision.

The same applies to writing. Lower outcomes in writing and fine motor development might be related, but what exactly is the difficulty?

Is it transcription? Composition? Phonological knowledge? Letter formation? Postural stability? Shoulder strength? Understanding the purpose of writing? Opportunities to develop and communicate ideas?

Is the child struggling with the physical demands of writing, or do they have little to say because they haven’t yet developed the language and experiences to support their composition? A fine motor intervention might be precisely what an individual child needs. But we shouldn’t arrive at that conclusion simply because two sets of percentages appear to be connected.

AI concluded that because most children had achieved the gross motor goal but not the fine motor goal more work needs to be done to address children’s fine motor skills, and that seems to be really logical. The problem is that the gross motor goal doesn’t measure all aspects of gross motor development that children need to be proficient writers. In fact a visit to classrooms to see these children in action showed that actually, some of them had issues with postural control, bilateral integration and shoulder girdle strength. Something that no fine motor intervention would have developed.  

We need to understand the barrier before we prescribe the solution. Put simply, AI had not seen these children in action, did not know enough about basic child development but had drawn some very dubious conclusions and presented them as fact !

The danger of very confident language

This, for me, is one of the biggest dangers of AI-assisted data analysis.

Generative AI is exceptionally good at producing text that sounds authoritative. It can connect ideas, construct explanations and offer recommendations in language that sounds credible and professional. It can also make assumptions, overlook context and present speculation as though it were established fact. The distinction between ‘this may suggest’ and ‘this demonstrates’ is enormously important.

Once a tentative hypothesis becomes a confident assertion, it is a very short journey to an action plan. Suddenly, we have ten-minute listening groups, fine motor interventions and new timetabled activities.

Staff are expected to implement them. Leaders monitor them. Governors ask about their impact and all of this might have been built on an assumption which nobody has properly tested. In the meantime, children may be missing opportunities for play, movement, conversation, exploration and responsive interaction because time has been allocated to interventions they don’t actually need.

We have created work for adults and demands for children without establishing whether either is necessary.

That’s not effective school improvement.

What the data actually tells us

I am not suggesting that we should ignore attainment data. Far from it.

But data should help us identify where we need to look more closely. It cannot, on its own, explain the development, learning and experiences of individual children. In a relatively small cohort, one child’s outcome can change a percentage quite considerably. Comparisons between years can be particularly misleading when the children involved have very different starting points, experiences and needs.  They should not be a basis for overhauling an entire curriculum.  Data should help us pose questions and explore ideas.

A headline percentage also tells us remarkably little about what an individual child can actually do. Two children may both be recorded as not achieving the number early learning goal.

One might be able to count reliably, understand quantity and subitise confidently, but not yet recall number bonds automatically. Another might recite numbers to twenty without consistently connecting number words to quantities. Both have the same recorded outcome, but different needs.

And this is why we need a much more forensic approach.

Who are the children behind the figures? What do they already know and understand? Which particular aspects of learning are proving difficult? What have they experienced? What does their learning look like in play? What do the adults who know them recognise as their strengths and needs?

We need to consider the curriculum, the enabling environment, adult interactions, opportunities for movement, relationships, language, inclusion and access to learning. Sometimes the barrier may lie within a particular aspect of a child’s development.

Sometimes it may lie in what we are offering, how we are teaching or the demands we place on children. We need to be curious enough to find out.

AI doesn’t know these children

Now, before anyone suggests that I’m a complete Luddite, I’m not. I can see AI could have the potential to reduce workload, organise information, check calculations, identify patterns and help us formulate better questions.

In the right hands, with appropriate safeguards, it might be a useful tool. But, it is a tool, not a substitute for professional knowledge and judgement.

AI hasn’t watched the child spend twenty minutes constructing a bridge, negotiating with their friends and solving problems. It hasn’t noticed the child who communicates brilliantly through imaginative play but becomes overwhelmed when asked to respond in a large group.

It doesn’t know the child whose confidence has blossomed because somebody has finally recognised their strengths. It cannot establish the causes of a child’s difficulties from a spreadsheet that contains only their assessment outcomes.

And it certainly cannot understand the complex relationships between development, experience, teaching, environment and learning without the contextual information required to make sense of them.

There is another danger, too. If we feed individual children’s information into AI systems, we need to be absolutely confident that we are protecting their personal data and using appropriate, approved systems.

But even when data protection is secure, the quality of the analysis still matters. We should be asking whether comparisons are meaningful, whether the conclusions are supported by evidence and whether the recommended actions are genuinely appropriate.

The Department for Education’s own guidance warns that generative AI can produce inaccurate, biased or misleading information, and makes clear that professional responsibility for the final output remains with the people using it.

That seems particularly important when the output may influence what happens to children every day.

The real danger isn’t the technology

Of course, people were drawing questionable conclusions from school data long before AI arrived. We’ve all seen action plans based on broad assumptions, interventions introduced because they worked somewhere else, and children grouped together because they happen to share a particular characteristic. AI hasn’t invented this problem.

But it has made it possible to produce these analyses much more quickly, with considerably more polish and an appearance of authority that can make them difficult to challenge.

And that’s what worries me. We risk confusing the sophistication of the presentation with the quality of the thinking. What if the person reading the report had no understanding of early child development? What if they took AI’s recommendations as gospel and put those interventions in place? There is a real danger that they could do more harm than good!

Because real school improvement is rarely simple. It requires time, curiosity, professional knowledge, careful observation and a willingness to question our own assumptions. It means understanding children and staff as individuals, not simply as percentages or members of a particular group.

Data can tell us where to look. It cannot tell us what we will find. If we allow AI to do our thinking for us, we risk developing increasingly sophisticated solutions to problems we haven’t properly understood. Can we stop outsourcing our thinking? The children deserve better than that.