Four Years of Building Verónica

AI-Generated Audio Summary

Prefer to watch? Verónica explains her four roles and how she works, in under three minutes.

It is eleven at night. A teacher candidate in Texas has a Spanish proficiency exam in three weeks, a full-time job, and a question about why her last spoken answer scored lower than she expected. Four years ago, the honest answer to that question lived in a workshop she could not attend. Tonight she asks Verónica, who has her recording, the rubric, and her scores from the last six weeks.

We have told the story of why we started building Verónica before, most fully in Our Approach to AI, a note from our founder, Luis J. Hernández. This post looks back at four years of building her and sets the result beside what other researchers have found about tutoring, feedback, retrieval, and model-based scoring, work we read before writing the first line of code and have kept reading since. We wrote it after spending most of October doing something we would recommend to anyone who builds with AI: mapping every place our tutor runs and asking what, exactly, reaches her before she answers.

Where she started

Much of our work for the last decade has been preparing bilingual and ESL teachers in Texas, where bilingual certification requires a Spanish proficiency exam, the BTLPT. Books and online modules handled listening and reading well enough. Speaking and writing needed someone in the room: a person who could hear the answer, read the paragraph, and say what to fix. Our founder taught those workshops at night and on weekends, and many students still could not come.

When large language models became available to developers in late 2022, we saw a way to keep that person present. The idea has deep roots in education. Lev Vygotsky, whose work from the 1920s and 1930s is still a point of reference in the field, described how a learner can do with a knowledgeable guide what they cannot yet do alone, and how that guidance is the path to doing it by themselves (Vygotsky, 1978). The term in common use today for that guide, the more knowledgeable other, grew out of his work. Wood, Bruner, and Ross (1976) gave the guidance itself its name, scaffolding, and the name stuck. We spent 2023 in beta with real learners and introduced Verónica publicly in May 2024. She has been inside every course since.

The page that describes her gives her four roles: study buddy, AI evaluator, smart tutor, and a source of data-driven insight. Each one answers a question researchers were asking long before anyone could chat with a model.

What fifty years of tutoring research asks

Tutoring researchers have asked one question for fifty years: does the conversation teach? Whether a computer can hold the conversation at all stopped being the interesting part decades ago.

Intelligent tutoring systems predate large language models by decades, and the evidence on them is substantial. A meta-analysis of 107 effect sizes from 14,321 participants found that learners using these systems outperformed learners in teacher-led, large-group instruction (g = .42) and learners using textbooks or workbooks (g = .35), and that the difference between tutoring systems and one-to-one human tutoring was not statistically significant (Ma, Adesope, Nesbit, & Liu, 2014). Those systems were nothing like a modern chatbot. They worked because they modeled what the learner knew and responded to it.

AutoTutor, one of the most studied conversational tutors, taught the field something about dialogue itself. When learners were asked a question that needed a paragraph to answer, their first attempts were typically one word to two sentences. The tutor’s job was to draw out the rest, with hints, prompts, and corrections aimed at the specific misconception in the answer, across 50 to 200 conversational turns. Learning gains ran about 0.8 standard deviations, depending on the measure and the comparison (Graesser, Chipman, Haynes, & Olney, 2005). A tutor that simply delivers the paragraph has skipped the part that teaches.

And a systematic review of 74 studies of chatbots in education, published just before generative AI arrived, ended with a warning we have kept on the wall. Most chatbot evaluations measured usability, acceptance, or technical correctness, not learning, which led the authors to conclude that chatbot development in education was “still driven by technology, rather than having a clear pedagogical focus of improving and supporting learning” (Wollny et al., 2021, p. 13). Everything below is our attempt not to be that sentence.

Role one: a study buddy who has read the lesson

Verónica sits inside every lesson page. What makes her useful there is what she has read before she answers.

Two technical papers shaped our answer. The first introduced retrieval-augmented generation, the approach everyone now calls RAG. Its authors pointed out that a model relying only on what it absorbed in training “cannot easily expand or revise” that memory, cannot readily show where an answer came from, and may produce hallucinations; attaching a retrieval step to a curated source lets the knowledge be revised and inspected, and in their tests produced “more specific, diverse and factual language” (Lewis et al., 2020). The second showed that language models use information at the beginning and end of a long input far better than information buried in the middle, a U-shaped curve the authors described as being lost in the middle (Liu et al., 2024).

Put those together and you get our rule for what reaches Verónica: the complete lesson, plus a small number of selected passages. Pouring the whole library into the prompt would bury the relevant passage in the middle, which is the failure Liu’s team measured. On a lesson page she now receives the full readable text of the lesson, not an excerpt. Behind her sits a shared knowledge library of more than 17,000 entries, organized into 84 collections and built from our courses, books, frameworks, and glossaries. No course sees all of it. A Special Education lesson draws from the Special Education collection; a TOEFL lesson draws from the TOEFL collection. On our website, each answer uses at most eight retrieved passages, chosen for the question at hand, and when the library has nothing relevant she says so and offers the next most useful thing she can ground, rather than inventing a fact to fill the gap.

She also answers in twelve languages, English and Spanish first, because a learner who can ask in the language they think in asks the question that is actually confusing them. In proficiency practice the same tutor is told to do the opposite and hold the target language, because the exam penalizes switching. We chose both behaviors for the learning; the model could do either.

Role two: an evaluator who shows her work

For speaking and writing, Verónica scores. This is the role that made us build her, and the one that demands the most care.

Valerie Shute’s review of the feedback literature defines formative feedback as “information communicated to the learner that is intended to modify his or her thinking or behavior to improve learning,” and distills decades of research into four words: it should be nonevaluative, supportive, timely, and specific (Shute, 2008, p. 153). That definition is the specification for Verónica’s feedback. Every observation quotes something the learner actually said or wrote. Growth areas are capped so the learner can act on them. A model answer sits next to the learner’s own. The rubrics were written by educators who have prepared candidates for these exams, and they describe what the exam measures: task fulfillment, organization, and language use.

A rubric alone is not enough, and the research on model-based scoring says why. When researchers tested strong models as judges of open-ended chat responses, the models agreed with human raters more than 80 percent of the time, about the rate at which humans agree with each other. The same study documented systematic biases: a preference for whichever answer appears first, a preference for longer answers, and a tendency to favor answers written by models like themselves (Zheng et al., 2023). Agreement with human preferences on chat is not evidence that a score means what an exam means.

So we do three things a rubric cannot do by itself. We check our scoring against the official sample responses the state publishes, and a daily check confirms it stays aligned. In key places one model verifies another’s work before the learner sees it. And we grade only what the evidence can honestly show: pronunciation and delivery are assessed from the recording, never guessed from a transcript, a decision we explained in A Good LLM Doesn’t Make a Good Teacher. For the same reason, we transcribe learners’ speech with the model that proved most accurate on accented, second-language speech, even though it costs more.

Her scores also carry weight beyond practice. When a learner’s work meets the bar in those evaluations, that result is what triggers the digital credential we issue, verifiable by anyone and the learner’s to share. We explained why the credential waits for evidence in How We Use AI, and Why Our Digital Credentials Require Evidence. That is one more reason we are careful about what she scores and how.

Role three: a smart tutor who knows your scores, and never invents one

In the student dashboard, Verónica can see a learner’s performance across every enrolled course: a heatmap by domain and competency, a study plan ranked by how much each weak area affects the exam score, and the concepts that overlap between courses. We described the program leader’s side of this in From Scores to Insight.

Two studies published in 2025 explain why this role needs discipline. In a randomized controlled trial in a large university physics course (N = 194), students who worked through a lesson with an AI tutor learned significantly more than students who received the same content in an active-learning class, with a median of 49 minutes on task against the class’s 60, and they reported more engagement and motivation. The effect was large: 0.63 standard deviations by regression and, once the authors corrected for a ceiling on the test, between 0.73 and 1.3 (Kestin, Miller, Klales, Milbourne, & Ponti, 2025). The authors are clear about what made the difference: the tutor was built on the same pedagogical practices as the class. They also warn that general chatbots “are generally designed to be helpful, not to promote learning,” and show “uncanny confidence” when they are wrong (p. 1).

The second study shows the other side. Nearly 1,000 high school mathematics students were given a standard ChatGPT-style tutor, a version whose prompts were designed to protect learning with teacher-designed hints rather than direct answers, or no AI at all. Both AI versions improved performance on practice problems. But students who had the unrestricted version did worse than students with no AI once it was taken away, and the authors concluded that “unfettered access to GPT-4 can harm educational outcomes.” The guarded version largely removed that harm (Bastani et al., 2025).

Our design response is the same in the dashboard and in the lesson. Verónica coaches from the learner’s real data, and she is told plainly what she does not have. Her view of a learner’s scores arrives signed by our platform, so she cannot be handed a number the learner never earned, and she distinguishes finishing a course from passing an exam. In practice she points to the next step instead of doing the work for the learner. When a learner asks a narrow question she answers it narrowly, without a lecture; when a learner asks for feedback or a variation, she produces the whole thing. Our reading of both studies is that the tutor that helped was the one that kept the learner doing the work, and that is the behavior we build for.

Role four: insight for the people who teach

Verónica’s fourth role is the least visible to a learner and may matter most. Program leaders and professors can see, by cohort and by student, where a group is strong and where someone has stalled. In Praxitude, Praxi, her sister tutor for scenario-based practice, evaluates pedagogical reasoning and target-language use together and hands the professor structured notes about where a student needs a person.

We use the phrase “needs a person” on purpose. Sherry Turkle, who has studied how people relate to machines for four decades, calls the new generation of companion chatbots “artificial intimacy” and describes them as “committed to the performance of empathy.” Their designers want us to believe the programs not only understand us but care. Generative AI, she writes, “does not care” about the outcome of any conversation, not “in the way humans use the word” (Turkle, 2024, pp. 2 to 3). Verónica is a learning system with a name and a voice, not a performance of empathy, and part of her job is to point a learner toward the human who can help: the professor, the coordinator, or one of us. Her page puts it this way: the first line of support, not the last.

The research on how learners experience language chatbots points the same way. A study of 176 future teachers in Spain and Poland who used conversational agents for four weeks found positive perceptions of ease of use and positive attitudes, with more moderate intention to keep using them (Belda-Medina & Calvo-Ferrer, 2022). A review of 25 empirical studies found that chatbots’ main affordances for language learning are timeliness, ease of use, and personalization, and that their recurring challenges are technological limits, cognitive load, and the novelty effect (Huang, Hew, & Fryer, 2022). The novelty wears off. A tutor has to be useful on the fortieth session, not the first, which is why Verónica lives where the work already is, in the lesson, the practice, and the dashboard, rather than as a destination learners have to remember to visit.

What four years taught us

The model is the least of it. We have changed models several times in four years, and we evaluate each new one through a pedagogical lens before it reaches a learner. What has not changed is the teaching built around the model: what she reads, how she paces, when she holds a language, how she gives feedback.

Wiring matters more than size. A tutor is only as good as what reaches her. She is not a chat window. Every request she makes to a model goes through our own code and carries what the learner is studying, the passages retrieved for it, the rubric when she scores, and the rules she follows, and every score comes back in a structured format we designed, built around the same rubric. This month we mapped every place Verónica runs, about fifteen services across our website and applications, and tightened what each one receives: the complete lesson, course-scoped sources, signed learner facts, and a clear distinction between a search that found nothing and a search that could not run. A larger library would not have done any of that.

We have not done systematic research on Verónica, and we say so. The studies above are other people’s research on other AI tutors, with other learners. We read them to learn from that experience, and we have borrowed freely from what they found. They do not show that Verónica produces the same results. The only way to know that is to study her, and that is a separate piece of work we have not done. What we can verify, we verify daily, starting with scoring against official samples. What we have not measured, we do not claim.

Our learners know the tutor is part of the course, and they teach us. Nobody meets Verónica by surprise. Her page says she is included in every course, our AI Terms explain what she stores, and our welcome videos introduce her before the first lesson. Many learners tell us the tutor is one of the reasons they chose us. Their feedback is the other place we learn from, and most of it arrives in conversation or in a short email after exam day: a candidate writes to say she passed and to thank us for the practice. We are glad every time, and we keep those messages. A thank-you note is not a pass rate, though, and a passed exam has many causes, so we treat the feedback as direction and act on it. Earlier this month a candidate described, in detail, how a speaking practice felt rushed compared with the exam she was preparing for. She was right, and the practice now uses the official exam timing by default, with a faster option for learners who want to train speed. The twelve languages and the hold-the-target-language rule started the same way, with someone describing their week.

Where to go from here

If you want to see what this looks like from a learner’s seat, Verónica explains it herself in an eight-part tour that runs under five minutes, and the Verónica page describes each role in more depth. If you build or evaluate AI tutors for your own program, the interactive walkthrough of the architecture we presented at Texas State University shows how the pieces fit. And if you are an educator working through the same questions, we would like to compare notes. Write to us.

References

Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122

Belda-Medina, J., & Calvo-Ferrer, J. R. (2022). Using chatbots as AI conversational partners in language learning. Applied Sciences, 12(17), 8427. https://doi.org/10.3390/app12178427

Graesser, A. C., Chipman, P., Haynes, B. C., & Olney, A. (2005). AutoTutor: An intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions on Education, 48(4), 612–618. https://doi.org/10.1109/TE.2005.856149

Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12610

Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: An RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15, 17458. https://doi.org/10.1038/s41598-025-97652-6

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474.

Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638

Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901–918. https://doi.org/10.1037/a0037123

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795

Turkle, S. (2024). Who do we become when we talk to machines? An MIT Exploration of Generative AI. https://doi.org/10.21428/e4baedd9.caa10d84

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds.). Harvard University Press.

Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are we there yet? A systematic literature review on chatbots in education. Frontiers in Artificial Intelligence, 4, 654924. https://doi.org/10.3389/frai.2021.654924

Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems, 36, 46595–46623.

Verónica is an AI tutor built by Enabling Learning to support, not replace, the work of human educators. The research cited here describes other tutoring systems and other populations; it does not report measurements of Verónica.