The problem
Reading a dense paper gives you the feeling of understanding. That is cheaper than the real thing. You highlight a few sentences, nod at the hard part, file the topic as known, and weeks later you cannot explain it to a colleague without hedging. Every learning tool I tried made this worse because they let me mark my own progress.
The rule the whole system is built around
A claim of knowledge does not count until typed proof attaches, and self-report never moves state. There is no "mark as learned" button anywhere in the product. This is not a UI choice. Every write that moves a mastery number happens server-side through a service-role client, and the browser can only read owner-scoped rows. There is no client-authored path to a mastery score.
How mastery is composed
Mastery for a topic is a single 0–100 score built from three channels you cannot grade yourself:
- Teach-back, 50%. You explain a concept in your own words. The model scores it against the module's ground truth on clarity, accuracy, depth and analogy quality, and names the specific gaps and contradictions.
- Tests, 30%. Questions generated from your own uploaded sources, scored question by question against a rubric rather than a peekable answer key.
- Retention, 20%. Spaced retrieval on FSRS-6, with 30, 60 and 180-day floors. A card is not "known" because you got it once.
When a signal is missing the formula eases off instead of punishing you. Teach-back and cards alone blend 70/30. Tests without cards blend 60/40. The full 50/30/20 only applies once all three exist.
Architecture
Two repositories with a deliberate split. The product is a Next.js 15 App Router app on React 19 and strict TypeScript, running the in-process agent runtime on Mastra with the Anthropic and OpenAI SDKs behind it. The backend is a Python 3.13 FastAPI service that owns the work the frontend should not do in-process: a RAG ingest pipeline that extracts, chunks and embeds sources into a shared Qdrant collection, and an MCP server exposing around twenty learning and wiki tools over full OAuth 2.1.
The seam matters. Both services share one Postgres and one vector collection, but the frontend never re-implements RAG and the backend never touches schema migrations or the scoring engine. Re-ingesting a source deletes its existing vectors by file-id before inserting fresh ones, so reprocessing replaces rather than duplicates and can never reach another user's points.
The agents
The product is not a chatbot with a system prompt. Agents are registered per model tier (fast, smart, pro) and the route picks a tier per request, because different jobs deserve different cost and capability.
The project agent runs under a strict grounding hierarchy: try the project wiki first, then RAG over your uploaded sources, then web search, and if none of them cover the question, say so plainly rather than invent. Tool failures fall through to the next source instead of retrying in a loop, and the tool loop is bounded at sixteen steps as a safety net.
What I would call the hard part
Making the model do the language work while the server owns the truth. The teach-back evaluator prompts for strict JSON, strips code fences, parses, then clamps every field into range with sensible fallbacks. A malformed response becomes a reasonable score instead of throwing. Bloom taxonomy level is computed internally to drive the engine and never shown. Mastery and gaps surface in plain language, with no academic chips and no "you reached Level 4 Analyze".