Skip to content

Building AI Capability

Learn on work you can afford to get wrong.

AI capability grows when people take a real piece of work through failure, correction and repeated use. Leaders need to make room for that practice and take part themselves.

In The Token Department, user skill is the fourth factor in the return on AI. My Becoming Engineers talks describe the progression from asking a chatbot to directing repeatable systems. The difficult part is learning how to build and judge them. Start now, on a bounded workflow inside your own work.

Neo practising kung fu in the dojo in The Matrix.
The four factors depend on one another. User skill develops through the work of improving the other three. This is a conceptual relationship, with no numerical scale.
Chapters
  1. The reliability dip
  2. Room to learn
  3. Three capabilities
  4. Your first workflow
  5. Hiring and apprenticeship
  6. The four-factor loop
  7. Key takeaways
  8. Sources and notes

Get through the reliability dip

An early demo gives you confidence before it gives you judgement. You ask for something, get an impressive result and assume the remaining work will be similar. I recognise that feeling. In Becoming Engineers, I call that stage "AI Guru".

Then you try to make it work again. A needed input is missing. An exception breaks the workflow. The agent follows an old instruction, loses context or produces a plausible answer with a silent error. Now you have to find out why.

I call the next stage "AI Idiot". In my own learning, I had built enough automation to give myself a second job maintaining it. More instructions and more agents created more things to understand. The reliability dip is the gap between a successful example and a system you can depend on.

The reliability dip

Explore the learning curve

Select a stage or scrub the curve. Arrow keys work on the slider.

The reliability dip Perceived capability rises after an early demo, falls when failures appear and recovers through deliberate practice. This is an illustrative mental model. Perceived capability Experience and deliberate practice The reliability dip Perceived capability rises after an early demo, falls when failures appear and recovers through deliberate practice. This is an illustrative mental model. Perceived capability Experience and deliberate practice

"AI Guru"

A good example works. Confidence rises before the inputs, exceptions and recovery paths have been tested.

Try next. Keep the successful example, then rerun it with a missing input. State what the system should do.

A mental model from my Becoming Engineers talks. The curve describes perceived capability through experience. It is illustrative, with no measured scale or promised timetable.

The way out is deliberate practice. Choose a small task, state what good looks like, inspect the actual output and retain the failure. Change one part of the method and run it again. A useful lesson leaves something behind that the next run can use.

An omitted source becomes a required input. A repeated correction becomes an instruction or a test. A failed handoff becomes an explicit dependency. You learn why the system behaved that way, and which part you can safely change.

The time to learn cheaply is now. Keep early attempts reversible, use permitted inputs and review results before anyone depends on them. Leaders should help build and evaluate one workflow themselves. That experience improves their ability to judge a proposal, fund the right support and recognise an unfinished demo.

Give people room to learn together

Designer Fund and Foundation Capital's AI in Design 2026 report offers evidence from tech and design practitioners and leaders. Its Teams chapter draws on a Q1 2026 survey and practitioner interviews. It describes that community's experience. I take the organisational lessons as hypotheses to test in other kinds of work.

Peer learning rose from 24% in the previous survey to 70%. Reliance on leadership recommendations fell from 32% to 16%. The people using the tools were increasingly teaching one another.

Learning increasingly happens among peers

70%report peer learning in 2026
16%use leadership recommendations

Respondents to AI in Design. The bars use a common zero to 100% scale.

Peer learning

202524%
202670%

Leadership recommendations

202532%
202616%

A culture of tinkering

Strongly supported organisations55%
Other support28%

Other support means moderate or minimal support. This is an association, with no causal effect established.

42%pay out of pocket for approved tools
21%use unsanctioned tools

Funding and access categories may overlap. These figures are signals for an internal tool review.

Source. AI in Design survey, Q1 2026, Teams chapter. Designer Fund and Foundation Capital. Earlier-year comparisons are those reported in the chapter. Different survey years need not contain the same respondents.

Strongly supported organisations also reported a culture of tinkering more often, 55% against 28% with moderate or minimal support. That association does not establish causation. It does suggest a better question for a leader than whether another training course is available. Can people practise during the working day?

In the survey, 42% report paying out of pocket for approved tools. The survey also records 21% using unsanctioned tools. Treat that stack as evidence of unmet needs and useful experiments. Ask what works, then establish a permitted route and a payment path. Personal spending should not become the price of participation.

Katie Dill, quoted in the report, names three adoption blockers. People have no room for risk, permissions are unclear, or nobody knows how to pay. Clearing them is concrete management work.

"we are all 'early career.'"

Share the failed attempt alongside the fix. A polished demonstration hides the part that another learner most needs to understand. Voice mode can help you explain the workflow in detail. Turn that explanation into a brief that someone else can run and challenge.

Build across roles and keep an owner

The Teams chapter reports that 65% are taking on adjacent product and engineering tasks. It also finds 34% reporting messier collaboration and less clear ownership. Broader capability creates coordination work. Small teams need operating discipline early.

I think dependable delivery needs three capabilities. They can overlap in one person, but I would build them across a mixed team.

Dependable delivery needs all three

Explore the responsibilities
Three intersecting capabilities Domain judgement, production engineering and the agentic layer meet in dependable delivery. A mixed team can provide them together. Team Domain judgement Economics, process, exceptions Agentic layer Context, harness, graph Production Engineering, reliability Three intersecting capabilities Domain judgement, production engineering and the agentic layer meet in dependable delivery. A mixed team can provide them together. Team Domain judgement Economics, process, exceptions Agentic layer Context, harness, graph Production Engineering, reliability

A mixed team owns the result

The specialist can explain an exception precisely enough to test. The engineer understands its commercial consequence. The agent uses the agreed context, tools and checks.

Keep accountable. Name the workflow owner. Experienced engineers remain responsible for production architecture, maintenance and reliability.

Three capabilities meet in dependable delivery. The overlap represents shared judgement, rather than a claim that every person must master every discipline. Select a capability to explore its responsibility and the failure when it is missing.

Domain and commercial understanding supplies the economics, process, incentives and exceptions. The specialist needs to explain what a good decision requires and who has authority to make it. An exception must be specific enough to test. The engineer needs to understand why it matters to the business.

Production engineering makes that decision process work with real systems. It covers integrations, version control, permissions, failure recovery and ongoing reliability. A domain specialist who can build a prototype still needs an experienced engineer to own production architecture and maintenance.

The agentic layer makes the model's work repeatable. An agent is an AI system that uses tools across several steps. Its craft includes three disciplines, defined in the Glossary.

  • Context engineering organises the information the agent needs, with its source, freshness and retrieval method. Practise by checking which inputs reached one failed run.
  • Harness engineering connects instructions, tools, permissions, checks and stopping rules. Practise by turning a repeated correction into a reusable skill, then testing it. AGENTS.md holds working instructions. Skills hold task methods. The harness connects them to execution.
  • Graph engineering makes the work's dependencies explicit. Practise by drawing one workflow, marking what can run together and what must wait for evidence or a person's decision.

A good learning task crosses these boundaries. The specialist writes the exception and the acceptance rule. The engineer makes the integration safe. Together they inspect the agent's result and decide whether it meets the rule.

Keep one named owner for the workflow and explicit owners for production changes. Shared understanding helps people collaborate. Responsibility still needs a name. Non-engineers gaining implementation skills does not make them the engineers on call.

Be your own first customer

Start with a recurring piece of your own work. You understand the inconvenience and can inspect the result. Keep its scope small enough to recover from a failed attempt.

Name the owner and map the baseline before building. Record the trigger, inputs, decisions, handoffs, waiting time and final output. Agree the exceptions and the standard for acceptance. Choose the model and route deliberately, then keep them recorded while you compare runs.

Build one complete example before expanding it. Write your own evaluations against representative inputs, awkward exceptions and known failures. An evaluation, or eval, tests the behaviour you need. Check both that the agent completes the intended action and that it stops when a needed input or permission is absent.

Keep a failure log with the input, observed behaviour, cause if known, correction and result of the next check. Preserve the difference between a diagnosis and a guess. Test whether the correction improves fresh examples as well as the case that exposed it.

Use operating measures to decide whether the workflow deserves more investment.

Measure the work that survives review

Throughput
Accepted outputs per week, alongside elapsed time per workflow.
Review and rework
Human minutes spent checking and correcting each accepted output.
Errors
Failures per case, including those discovered after acceptance.
Full cost
Tokens, tools, engineering and human checks, including failed runs.
Proposed measures for one workflow. Compare similar work over a stated period and retain failed runs in the cost. These are definitions to agree, rather than measured results.

Count throughput as accepted outputs over time. Record human review and rework separately from model runtime. Measure errors against an agreed definition and keep track of errors found after acceptance. Full cost includes tokens, tools, engineering support and human checks. Value time at a stated rate and count each charge once.

Seat counts and prompt counts can show use. The investment decision needs the resulting change in delivery. Hours released become value when someone uses that capacity or realises the saving. A workflow that creates more review work needs another iteration, even if its first draft arrives faster.

Assess judgement and teach the next cohort

The Teams chapter describes formal reviews, career ladders and hiring criteria lagging behind expanding expectations. It also reports hiring emphasis on AI fluency, systems thinking and strategic judgement, with craft still important. Joel Lewenstein describes the need to rethink one's process repeatedly as tools change.

I would look for evidence that someone can revise a method after seeing it fail. Ask why they chose the workflow, what they checked and which part they would still give to a specialist. Adaptability includes recognising when a useful new tool makes an old process unnecessary.

The apprenticeship question is harder. Junior execution has also been how people learn to recognise a good result. If agents do that work, teams must deliberately preserve the learning inside it. Supervising a machine requires knowledge that beginners are still acquiring.

I would give juniors small, real workflows under supervision. Ask them to attempt the reasoning, predict likely failures and check an agent's output against the source. Then review disagreements with an experienced practitioner. Have them repair a flawed result and explain why the repair is better.

Teach the underlying craft as part of that review. Trace a number to its inputs. Read the changed code. Explain why a design decision helps its user. Rotate responsibility for verification and let juniors see the consequences of errors before increasing their scope.

The cost of that supervision belongs in the team's plan. A senior who can generate more work still needs time to teach. Faster production alone gives us no assurance that the next generation is learning to judge it.

Keep capability in the four-factor loop

Building AI capability changes the other factors in The Token Department. A skilled user can recognise which model a task needs, improve its route and repair the harness. A new model or harness can also change what that person needs to learn.

Review those decisions together. Start the next cycle from accepted work, costs and failures. Keep the methods that transfer. Retest those that depend on a particular model or tool. The practical return on learning is a better next run with less correction from you.

Key takeaways

  • Learn through a bounded workflow that you can inspect, repeat and recover.
  • Give peers time, permission and a payment route to develop and share useful methods.
  • Combine domain judgement, production engineering and agentic craft, with named responsibility for delivery.
  • Measure accepted work, review effort, errors and full cost against a baseline.
  • Assess recovery and judgement in live work. Teach juniors the craft required to verify it.
  • Revisit model selection, effective routing, harness quality and user skill together.

Sources and notes


  1. AI in Design 2026, Teams, Designer Fund and Foundation Capital. Survey findings are labelled Q1 2026. The report combines design-sector survey responses with practitioner interviews. Its methodology describes 900+ responses and notes that the sample may skew towards active AI adopters. Treat the findings as directional, rather than a population census. The percentages describe its respondents, rather than all workers or organisations. The 2025 and 2026 comparisons are those reported by the authors. Support and tinkering are associated, with no causal test established here. The funding categories can overlap and should not be added together. ↩

  2. Teams, role changes. The report describes adjacent product and engineering work and unclear ownership. The three-capability model and the proposed division of responsibility here are my synthesis. Neither the percentages nor the diagram estimate how many roles a business can remove. ↩

  3. Teams, hiring and careers. Fishbowl and Rework are credited there to Garrett Fowler, founder of Offsite. The report closes with open questions about juniors and apprenticeship. The supervision and assessment criteria proposed here are my recommendations. The learning curve and repeated-practice method draw on Becoming Engineers. ↩

Relevant posts & resources

Say hello

Draft a message about the work you need to complete.

Local draft preview. No message is sent. Review the draft, then copy it for use elsewhere.