Imagine asking a student two questions after an AI-assisted exercise.
“Did you get the answer?”
“Could you solve a similar problem on your own?”
Those questions can produce very different answers. They also get to the heart of what AI should do in education technology.
A tool may write an explanation, suggest a practice question, or finish a calculation almost instantly. Its educational value depends on what happens next. Does the learner understand more? Can the instructor give better feedback? Does the school save time after someone has checked the work?
AI in EdTech means using artificial intelligence within education technology to generate content, adapt practice, interpret information, or support teaching and administration. Its value comes from improving a specific task while keeping learning goals and human judgment in view.
For a training school, that calls for a closer look than a product demonstration can usually provide.
How does AI in education technology work?
Several different technologies sit under the AI label. Understanding the difference makes it easier to judge what a product is offering.
Generative AI produces new material. A large language model can draft an explanation, propose questions, or respond to a student. It learns patterns during training and generates sequences of small text units, called tokens, in response to its instructions and context. It does not automatically check every sentence against an authoritative source. IBM’s explanation of large language models describes the statistical prediction process behind these systems.
Adaptive learning systems adjust what a learner sees next. A system might use answers and performance history to select easier practice, revisit a concept, or move to a harder problem. Some systems use AI models; others follow rules written by instructional designers. A personalized-looking screen alone does not tell you which approach is being used.
Prediction and classification systems identify patterns. For example, software may flag an unusual change in participation for staff to investigate. Such a flag is an estimate based on available information. It does not establish why someone missed class or whether they will finish a program.
Ordinary automation belongs in the conversation, too. Sending a reminder two days before a class starts can be useful without requiring AI at all. Ask what the feature actually does, what information it uses, and how its output is checked.
How can an AI assistant use your course materials?
A general chatbot may know something about your subject without knowing your curriculum, assessment standards, or school policies.
One way to connect it to approved information is retrieval-augmented generation, usually called RAG. The system searches a selected collection of documents, retrieves relevant passages, and supplies those passages to the language model as context for its response. IBM describes this retrieval-and-generation process in its technical overview.
For example, a school could design an assistant to consult its current course handbook before answering a question about required assignments.
That design still needs testing. The system can retrieve the wrong passage, overlook an exception, or produce an answer that goes beyond the source. “Connected to our documents” is a useful starting point, not a guarantee of accuracy.
Ask a vendor to demonstrate what happens when the handbook does not contain the answer. A clear admission of uncertainty and a route to an instructor may be more useful than a confident paragraph.
What does research say about AI tutoring?
The evidence shows why the design of the learning experience matters.
In a 2025 randomized study published in Scientific Reports, Greg Kestin and colleagues studied 194 students in a Harvard undergraduate physics course. Students experienced a purpose-built AI tutor and classroom active learning across two lessons, switching conditions between weeks.
The AI-tutored group had higher immediate post-test performance. The tutor was deliberately designed around educational practices, including guided engagement and feedback. This was a study of a particular tutor, two physics lessons, and one university setting. It does not establish that any chatbot will improve learning across an entire training program or produce better long-term job performance.
A 2025 PNAS study by Hamsa Bastani and colleagues offers a different warning. In an experiment involving nearly 1,000 high school mathematics students in Turkey, AI access improved performance during assisted practice. However, students given the version resembling a standard chatbot performed worse on a subsequent unaided assessment than students without AI access. The reported reduction was 17% in grades, not 17 percentage points. A version designed with learning safeguards largely mitigated that negative effect.
These studies used different students, subjects, and tools, so their results are not directly interchangeable. Taken together, they suggest a practical question for schools: What is the learner doing while the AI is helping?
If the tool supplies the reasoning and the answer, a completed worksheet may conceal a gap in understanding. If it asks the learner to attempt a step, explain a choice, and use feedback, the activity gives the school something more meaningful to evaluate.
Where AI can add value for a training school
The following are illustrative uses to evaluate in your own setting, rather than outcomes established for Steams Online customers.
Guided practice between classes. Consider an adult learner studying a technical concept after work. An assistant could ask which part is confusing, offer a different explanation, and provide another practice problem. The instructor should still check whether the learner can apply the concept independently. For hands-on training, a good text answer cannot establish competent physical performance.
Preparing teaching materials. An instructor could start with approved objectives and ask for alternative examples or draft quiz questions. Reviewing several candidates may be faster than starting from a blank page, provided the instructor checks accuracy, answer keys, difficulty, and alignment with the course.
There is evidence for time savings in a defined setting. An EEF-funded trial independently evaluated by NFER involved 259 teachers at 68 secondary schools in England. Teachers assigned ChatGPT and an implementation guide spent 31% less time on the measured Year 7 and 8 science lesson and resource planning activities: about 25 minutes less per week on average. A limited sample of materials showed no noticeable quality difference, but the evaluators cautioned against drawing a firm quality conclusion from that sample. The study did not show a 31% reduction in teachers’ total workloads.
Helping learners approach unfamiliar language. A draft glossary or a plainer-language explanation could make technical material easier to approach. Keep the correct professional terminology alongside the explanation. Have a competent reviewer check translations and technical meaning, particularly where an error could affect safe practice.
Finding the right information. A course assistant could help a learner locate the relevant assignment instructions. An administrative assistant could help staff find a policy passage. Test whether the response reflects the current document and whether people can open the source themselves.
Preparing for a human conversation. An instructor might use a summary of reviewed assessment results to decide what to discuss with a learner. The summary can organize the conversation; the instructor still needs to understand the person, the work, and any circumstances the data misses.
Match the tool to the kind of decision
A draft practice question is relatively easy for a qualified instructor to inspect before use. A decision about passing a learner, awarding credit, or approving a safety-sensitive skill has more serious consequences.
That difference should shape the workflow. Use AI-generated material as a candidate for review. Keep final assessments tied to your standards and appropriate human oversight. Use approved tools and authorized data, especially when student information is involved.
NIST’s Generative AI Profile identifies confident false content as a risk. It is one reason fluent writing should never be treated as evidence that an answer is correct.
You also need a usable path for students who cannot access the tool or prefer to ask a person. A feature has limited educational value if using it becomes an extra obstacle to completing the course.
A practical way to find out whether it helps
Choose a small task with a result you can observe. For example: help students practice one difficult concept before the next class.
Write down what would count as success before starting:
- Students can solve a fresh, comparable problem without AI assistance.
- Explanations meet an instructor-approved rubric.
- The instructor spends less total time preparing and correcting materials.
- Incorrect answers, student confusion, and requests for help are recorded.
- Tool costs and staff setup time remain reasonable for the benefit achieved.
If you compare groups, give them comparable material, time, and assessments. If you compare with a previous cohort, recognize that differences in learners or teaching may explain some of the change. A school pilot can inform your next decision without proving that the tool caused every improvement.
Include the work that is easy to miss: uploading and maintaining source material, checking outputs, supporting students, and correcting mistakes. A ten-minute draft that needs an hour of repair has not saved preparation time.
Most importantly, look beyond usage. Frequent logins tell you that people opened a tool. They do not tell you what those people learned.
Where Steams Online fits
AI needs to fit the course your school actually delivers. Objectives, assignments, assessment records, and instructor feedback provide the context for judging its usefulness.
Steams Online’s learning management tools support course materials, assignments, quizzes, grading, and progress tracking. Those records can help your team organize learning and review results as it evaluates new teaching methods.
Before adding an AI feature, choose the problem you want it to solve. Perhaps students need another opportunity to practice. Perhaps instructors need a faster way to prepare examples. Define that need first, then ask the technology to demonstrate its value against it.
The most useful question at the end of a trial is a human one: “What can our learners or staff do better now?”
