Scorecard

What should a trial actually test?

Test how the mentor teaches through difficulty and what the learner can subsequently do—not just whether the mentor can produce a working demonstration.

Stanford’s November 2026 evidence review distinguishes AI used for learning support from AI used to produce work. It reports mixed learning effects and notes that more positive learning experiences do not necessarily translate into stronger learning outcomes. The review principally updates evidence published from October 2025 through June 2026; it is not a study of executive households or a validation of premium mentorship. (scale.stanford.edu)

A 20-minute adult screening interview

Use this before introducing the child. The timing is a proposed interview structure, not an optimal teaching dosage.

Time Request What to listen for
0–3 minutes Explain what you would assess before choosing this child’s project. Questions about demonstrated skills, interests, available practice time and tool restrictions—not an immediate package pitch.
3–7 minutes Explain one relevant concept twice: first to a beginner, then to a coding-ready teenager. Accurate simplification, a useful example and a question that checks understanding.
7–13 minutes Diagnose a small, unfamiliar bug or flawed evaluation. A testable hypothesis, controlled investigation, explanation of the fix and verification—not repeated unexplained prompting.
13–17 minutes Walk through the proposed accounts, data and release decisions. Specific product knowledge, appropriate uncertainty, a private-demo option and clear permission boundaries.
17–20 minutes Show a redacted progress report and explain the complete quote. Evidence of learner capability, transparent assistance, usable teaching time and clear exit terms.

For a machine-learning project, a suitable adult technical probe is an evaluation that accidentally uses test information during model development. Scikit-learn’s documentation explains that such leakage can produce overly optimistic performance estimates and recommends keeping training and test operations properly separated. The candidate should explain the issue in plain English, not merely recognize a library function. (scikit-learn.org)

For a generative-AI application, use a small, synthetic information set and ask how the mentor would test supported answers, missing information and misleading retrieved instructions. OWASP documents how external content can alter model behavior and recommends controls including least privilege, output validation and human approval for sensitive actions. Keep the exercise inside an authorized sandbox. (genai.owasp.org)

A four-session learning-evidence pilot

Agree the goals and cost before starting. Shorten, split or otherwise adapt sessions for the learner; do not treat this structure as a universal schedule.

Session Teaching purpose Evidence to retain
1. Establish the baseline Let the learner attempt a small relevant task; identify prerequisites and agree the project’s learner-owned core. Starting capability, assistance needed, learner preference and revised scope.
2. Build with explanation Teach one component and ask the learner to predict or explain its behavior. A small artifact, the learner’s explanation and the source of substantial contributions.
3. Change and debug Introduce a modest changed requirement or fault without supplying the answer immediately. What the learner changed, how they investigated and what hints were needed.
4. Revisit and transfer Revisit an earlier concept, then apply it to a slightly different example. Retention, transfer, continuing gaps and the learner’s view of the relationship.

A suitable bounded example is a hobby-information assistant using invented material. The learner-owned core could be explaining how information is selected, adding one item, testing an unsupported question and explaining why the system should sometimes decline to answer. A public website is not required for the pilot.

How should assistance and authorship be recorded?

Use a brief, non-punitive assistance scale:

  • H0: completed without conceptual help, using agreed reference or accessibility tools.
  • H1: completed after a general hint.
  • H2: completed with step-by-step guidance.
  • H3: the substantive step was supplied by the mentor or generated by AI and has not yet been independently demonstrated.

These labels describe a task, not the child’s ability or worth. H3 can be appropriate during instruction; the renewal question is whether the agreed capabilities are becoming more independent.

For each milestone, record:

Learning target Evidence Assistance Contribution Next check
Explain why the assistant cannot answer a missing fact Learner identifies the absent information and changes the fallback response H1 Learner explanation; mentor scaffold; AI-generated draft code Try a different missing-information example next session

Do not infer authorship percentages from code appearance or commit history. Ask directly which parts were learner-created, mentor-supplied, generated or jointly revised. Use short demonstrations and notes rather than making lesson recordings a condition of assessment.

How should the test change by age and readiness?

Keep the purchasing criteria consistent, but change the task:

  • Ages 8–11: use concrete examples, sorting, prediction, explanation and adult-operated demonstrations where appropriate. Do not require an adult-style coding portfolio.
  • Ages 12–15: assess demonstrated foundations, not age alone. A 12-year-old and a 13-year-old may require different account arrangements even when their coding skills are similar.
  • Ages 15–18: for more advanced projects, ask for a baseline comparison, unfamiliar test cases, clear attribution and an explicit deployment decision.
  • Any age: preserve agreed accessibility tools and accommodations. A brief independent check should remove answer substitution, not necessary support.

These are proposed assessment adaptations, not developmental norms or a fixed AI learning roadmap.

Sources cited on this page

  1. Al Hub for Education ↗scale.stanford.edu
  2. 12. Common pitfalls and recommended practices — scikit-learn 1.9.1 documentation ↗scikit-learn.org
  3. LLM01:2025 Prompt Injection - OWASP Gen AI Security Project ↗genai.owasp.org