Balancing Support and Student Agency: What Three Years of Using Generative AI as an Academic Tutor Taught Me

About three years ago, I gave my 8th grade students access to an AI tutor chatbot built through SchoolAI, one of several platforms I experimented with alongside MagicSchool AI and Playlab. Some students used it exactly as I had hoped, working through confusion, asking follow-up questions, and extending their understanding. Others did what I feared: they copied the AI’s responses and moved on.

That tension became the center of my work for the next three years. The potential was too clear to walk away from, but I could not ignore what I was seeing: some students were outsourcing their thinking rather than extending it.

So I started asking a different question. Not, “Should we use AI in the classroom?” but:

How do we embed reflective and responsible AI use into teaching and learning so that students think critically about AI output, extend their own reasoning with it, and stay in ownership of their learning?

That question became my problem of practice and the foundation of an action research project focused on cultivating reflective and responsible AI use in secondary mathematics.

Here is what I tried and what I learned.

Start With Literacy, Not Access

The biggest mistake I made in my first year was giving students the tool before giving them the context.

They knew how to type into a chatbot. They did not yet know how to think with one.

So I restructured my approach. Before students began using an AI chatbot for academic support, I spent the first two months of school dedicating about 20 minutes each week to AI literacy. I drew on freely available resources, each serving a different purpose. AI4K12 provided foundational concepts around how AI systems work. Code.org offered structured lessons on machine learning. Common Sense Media helped students evaluate AI output and think critically about bias and reliability. MIT’s Day of AI grounded these concepts in hands-on scenarios.

We continued revisiting these ideas throughout the year as students began using the chatbot, allowing them to practice AI literacy in authentic contexts rather than encounter it only as theory.

Build the Habit Before the Shortcut Takes Hold

Even with AI literacy in place, I noticed students reaching for the chatbot the moment they felt stuck, before giving themselves a real chance to think.

AI is fast, available, and patient. It does not make students feel embarrassed for not knowing something. But that ease can also make it tempting to skip the productive struggle that helps build understanding.

To address this, I created a structured flowchart for students to follow before consulting the chatbot: read the problem twice, write down what you know, check your notes, and only then, if you are genuinely stuck, go to the AI.

I also gave students sentence frames to guide their interactions. Prompts such as “I think the first step is ___ because…” or “I am stuck on ___; can you guide me?” encouraged students to arrive at the chatbot with their own thinking rather than simply requesting an answer.

After one semester, 62% of students identified “trying the problem first” as their primary strategy for avoiding over-reliance on AI. That was exactly the habit the flowchart was designed to encourage. Seeing it appear in student survey responses suggested that the routine was beginning to become internalized.

Watch the Conversations, Not Just the Outcomes

One of the most valuable practices I developed was regularly reviewing student chat logs, not simply to monitor what students were asking, but to understand how they were thinking.

The logs became a window into misconceptions I might not otherwise have caught and a record of individual growth over time.

In September, for example, one student’s interactions were short and passive:

“How do I figure this out?”“What is the answer?”

By January, the same student was writing:

“Do we add 4 on both sides to get 2 equals 2x?”

She was also beginning to catch her own errors during the conversation. Her interactions had shifted from waiting to be told what to do toward actively reasoning with the AI. Over the same period, her engagement level, as recorded by the platform, moved from minimal to high.

That kind of growth may not always be visible on a traditional assessment. But it becomes visible in the conversation.

Ask Students What They Are Experiencing

After one semester of structured AI tutor use, I surveyed 57 students. Several patterns stood out.

85% said they still preferred asking their teacher when the teacher was available. In this context, the AI appeared to be functioning as a supplement rather than a replacement.

95% reported that they were still thinking through problems themselves at least some of the time.

55% said they were setting personal limits on their own chatbot use.

When students were stuck at home, they ranked class notes as their first resource, followed by online videos, with the AI tutor coming third. AI was not their default resource.

The survey also included metacognitive questions rated from 0 to 10. The strongest result was students’ ability to recognize what they already knew and what they still needed to work on, with a mean of 6.91 out of 10. Students rated their ability to identify where they were confused at 6.37, while recognizing gaps in their understanding and determining what to study next were rated 6.18 and 6.19, respectively.

These are the outcomes I care about most: not speed or task completion, but self-awareness and ownership of learning. Some areas were still developing. Students rated using AI to practice asking better questions at 5.39 out of 10 and using correct mathematical vocabulary at 5.32. Those findings now help shape the next phase of my work.

What This Means for Any Classroom

I teach 8th grade mathematics, but the principles are not subject-specific. Any educator bringing AI into the classroom faces a similar tension: How do we make the tool supportive without making thinking optional?

A few lessons I would share with any colleague beginning this journey:

Teach how AI works before you teach with it. AI literacy should not be treated as separate from AI use. It provides the foundation students need to engage with these tools critically and responsibly.

Give students structure, not just permission. A flowchart, sentence frames, or a simple protocol can help create space for thinking rather than allowing AI to replace it.

Read the conversations. Student interactions with AI can provide valuable formative assessment data, revealing misconceptions, questions, and reasoning processes that may not appear on traditional assessments.

Ask students directly. Survey them. Listen to how they describe their experiences. Their relationship with these tools is different from ours, and their feedback can sharpen instructional design.

The Work Continues

Across chat log comparisons, student surveys, and classroom observations, the evidence from my classroom tells a consistent story: when generative AI is implemented strategically within structured learning routines, it can support students in developing greater metacognitive awareness without making their own thinking optional.

Metacognitive awareness is growing. Communication and expressive confidence are next.

And somewhere in a classroom, a student is typing not, “Just give me the answer,” but:

“I think the first step is this, because…”

That shift, small as it may sound, is exactly what this work is for.