Being privacy conscious when using AI tools in school

Thinking about how staff and pupils' data shared with AI could identify them

Published:
Renate Samson is an independent education, AI and data policy consultant. 

Are you using AI? Do you use it to support with your day-to-day work?  If so, you are not alone. According to a National Education Union survey conducted in February 2026 three quarters of teachers surveyed are using AI tools for day-to-day work. The survey found that the top three uses of AI were supporting resource creation, lesson planning and administration1. 

The breadth of ways we can use AI are why it is called a general purpose technology. That we can use generative AI, such as Chatbots, for almost any task is incredible.  But the more we  bring AI into our lives and workplaces the more we need to think about what we are using AI for, what information or data we are giving to the models and how safe, secure and private any of that information is - particularly if it is about people.  

After years of using the internet we should all be familiar with data privacy and how we look after data that is about us and other people. We know that anything we share, say, post or do online is not private, and yet despite years of cookie pop-up notifications, terms and conditions, UK-GDPR and news headlines about data breaches losing people’s personal data, we continue to be complacent about the data we share, particularly when it comes to our use of AI.    

Why does this matter?  Well, of the three most common uses of AI outlined in the survey, there is a risk that one of them in particular - administration - might involve using personal or personally identifiable data about pupils. Whether it is classroom management, timetabling, marking, feedback or even the writing of end of year reports which I have heard some teachers say they use AI chatbots to write, there is a chance that data about pupils that could identify them could be shared with an AI.  

In theory using AI for these purposes seems like an excellent and efficient way of getting the job done,  but in practice you may not be compliant with data protection law, especially if you are using your own free to use or subscription consumer model.  

Consumer facing AI chatbots people use at home or on their mobile devices are simply not private.  Any conversation with ChatGPT, Claude, Grok, Gemini, Copilot or other free to use or consumer subscription models - even if you are chatting in incognito mode - should be considered to be a public conversation.  Anything written, spoken or uploaded as prompts to these products is considered fair game to help train the model further. No interaction is private, no piece of data forgotten. Everything is used and all data shared with an AI has the potential to be recycled into other responses given to other people. Furthermore, you might find your conversation history plastered on the internet.

Any use of these models by teachers will not be compliant with data protection law, something which all schools, academies and trusts, and anyone who works in a school, academy or trust, are all legally required to comply with3.  

AI models that have been specifically procured by a school, academy or trust under license are, however, more likely to be compliant with data protection law. For example Microsoft Copilot for Education or Google Gemini for Education offer licences to schools that have data protection assessments as part of the contract. However, close scrutiny of where data is processed and stored, and confirmation that any data shared into the system is excluded from training the AI models, is absolutely vital to be confident the product aligns not just with data protection law but with the Department for Education’s safety expectations4. Furthermore, this should not be considered a one off process but should be kept under regular review. 

That the National Education Union survey didn’t focus on the AI models teachers are using is not surprising.  Understanding what we are using AI for feels much more urgent than knowing what AI we are using.  After all, we are all racing to get to grips with what AI is, what it can do and whether it is an appropriate or useful tool for us to use in our personal as well as professional lives. Thinking about the nuances of the models we use and where the data we share with them goes is unsurprisingly not top of our priority list. But, it should be.  

If the past 25 years and more of the internet, social media, smartphones and data driven technologies has taught us anything, it is that the data we share with technology companies and digital products isn’t always used in ways that have our best interest at the core. When data about us is lost, misused or hacked, when social media knows more about us than our family, when we are subject to scams, data breaches or online harms the consequences can be time consuming, stressful and in some cases life changing.  As people interested in education we all have a responsibility not just to protect the young people we teach but to ensure that when data is shared it is shared to improve human learning not AI learning.  

Back to top