Three questions to ask before you buy

By Chris Calder, AI Education Consultant | Meta Pedagogy

Schools are buying AI tools without asking the right questions

Schools are under pressure to adopt AI. Edtech vendors know this and the sales pitches are relentless. Every platform promises to save staff time, personalise learning and close attainment gaps. Most of these claims are backed by the vendor’s own data, if they are backed by anything at all.

Before purchasing any AI tool for your school, ask three questions. If the vendor cannot answer them clearly, that tells you everything you need to know.

Question 1: Where does the data go?

Any AI tool your staff or students use will process data. The question is what data, where it is stored, who can access it and whether the tool uses inputs to train its model.

Under UK GDPR any processing of personal data requires a lawful basis. If staff are entering student names, assessment data or behavioural information into an AI tool, the school needs to know exactly what happens to that data. If the tool’s terms allow inputs to be used for model training, anything entered could end up in the training set of a system used by millions of other people.

Check whether the tool offers enterprise or education-specific accounts with data processing agreements. Check whether data is stored in the UK or transferred internationally. Check whether the tool has been assessed against the DfE product safety standards (updated January 2026). Check whether your DPO has been consulted.

A tool that cannot provide a clear data processing agreement or that stores data outside the UK without adequate safeguards is not suitable for use in a school, regardless of how good its features are.

Question 2: What does the evidence say about impact?

“Our users report 40% time savings” is marketing, not evidence. Ask for independent research. Ask for peer-reviewed studies. Ask what methodology was used and who funded the research.

Most edtech tools have no independent evidence base at all. The Education Endowment Foundation has repeatedly found that technology interventions in schools produce modest gains at best and that the quality of implementation matters more than the quality of the tool. A tool with strong evidence deployed badly will underperform a simple tool deployed well.

When evaluating vendor claims, ask specifically: what learning outcomes improved, in what context, with what sample size and over what time period? If the answer is a case study from one school with no control group, that is an anecdote, not evidence.

Schools should also ask whether the tool has been evaluated for bias. AI models trained on biased data produce biased outputs. If the tool is being used for assessment, feedback or student-facing interaction, bias in the model becomes bias in the educational experience. Ask the vendor what they have done to test for this and what they found.

Question 3: Does this replace thinking or support it?

This is the question vendors will not ask because the answer might lose them a sale. Many AI tools designed for student use are built to produce finished outputs: completed essays, solved problems, generated revision notes. If a student can use the tool to produce a finished piece of work without doing any of the cognitive work themselves, the tool enables churning.

A tool that supports thinking looks different. It asks questions rather than giving answers. It challenges reasoning rather than producing arguments. It provides scaffolding within the zone of proximal development rather than jumping straight to the solution.

When evaluating a student-facing AI tool, test it yourself. Give it a task your students would be set. If it produces a finished output that a student could submit without understanding anything, the tool is doing the learning for them. The Cognitive Debt Framework provides a structure for evaluating this: can the tool be used in a way that keeps the student doing the cognitive work at each stage, or does it bypass the thinking entirely?

For staff-facing tools (lesson planning, resource generation, feedback drafting), the same principle applies in a different way. If the tool produces outputs that staff use without checking, editing or adapting, the tool is replacing professional judgement rather than supporting it. A good planning tool gives staff a starting point they then shape. A bad one gives them a finished product they copy.

What this means in practice

Schools do not need to avoid AI tools. They need to evaluate them properly before committing budget and staff time. The three questions above are a minimum standard for due diligence.

If a vendor can show you a clear data processing agreement, independent evidence of impact and a product design that supports thinking rather than replacing it, the tool is worth considering. If they cannot answer one or more of those questions, the conversation should end there.

If your school needs support evaluating AI tools or building a procurement framework for edtech, book a free consultation with Meta Pedagogy.