Chapters
Get through the reliability dip
An early demo gives you confidence before it gives you judgement. You ask for something, get an impressive result and assume the remaining work will be similar. I recognise that feeling. In Becoming Engineers, I call that stage "AI Guru".
Then you try to make it work again. A needed input is missing. An exception breaks the workflow. The agent follows an old instruction, loses context or produces a plausible answer with a silent error. Now you have to find out why.
I call the next stage "AI Idiot". In my own learning, I had built enough automation to give myself a second job maintaining it. More instructions and more agents created more things to understand. The reliability dip is the gap between a successful example and a system you can depend on.
The reliability dip
Explore the learning curveSelect a stage or scrub the curve. Arrow keys work on the slider.
"AI Guru"
A good example works. Confidence rises before the inputs, exceptions and recovery paths have been tested.
Try next. Keep the successful example, then rerun it with a missing input. State what the system should do.
The reliability dip
Missing context, edge cases and silent errors expose the distance between a demonstration and dependable work.
Try next. Inspect the failed output against its source. Record what happened and turn one recurring error into a check.
Context, harness and graph discipline
The next run starts with better inputs, a reusable method, explicit dependencies and evidence that the correction works.
Try next. Test the revised workflow on fresh cases. Keep an owner for maintenance and measure accepted work and review effort.
The way out is deliberate practice. Choose a small task, state what good looks like, inspect the actual output and retain the failure. Change one part of the method and run it again. A useful lesson leaves something behind that the next run can use.
An omitted source becomes a required input. A repeated correction becomes an instruction or a test. A failed handoff becomes an explicit dependency. You learn why the system behaved that way, and which part you can safely change.
The time to learn cheaply is now. Keep early attempts reversible, use permitted inputs and review results before anyone depends on them. Leaders should help build and evaluate one workflow themselves. That experience improves their ability to judge a proposal, fund the right support and recognise an unfinished demo.
Give people room to learn together
Designer Fund and Foundation Capital's AI in Design 2026 report offers evidence from tech and design practitioners and leaders. Its Teams chapter draws on a Q1 2026 survey and practitioner interviews. It describes that community's experience. I take the organisational lessons as hypotheses to test in other kinds of work.AI in Design 2026, Teams, Designer Fund and Foundation Capital. Survey findings are labelled Q1 2026. The report combines design-sector survey responses with practitioner interviews. Its methodology describes 900+ responses and notes that the sample may skew towards active AI adopters. Treat the findings as directional, rather than a population census. The percentages describe its respondents, rather than all workers or organisations. The 2025 and 2026 comparisons are those reported by the authors. Support and tinkering are associated, with no causal test established here. The funding categories can overlap and should not be added together. Full source note 1
Peer learning rose from 24% in the previous survey to 70%. Reliance on leadership recommendations fell from 32% to 16%. The people using the tools were increasingly teaching one another.
Learning increasingly happens among peers
Respondents to AI in Design. The bars use a common zero to 100% scale.
Peer learning
Leadership recommendations
A culture of tinkering
Other support means moderate or minimal support. This is an association, with no causal effect established.
Funding and access categories may overlap. These figures are signals for an internal tool review.
Strongly supported organisations also reported a culture of tinkering more often, 55% against 28% with moderate or minimal support. That association does not establish causation. It does suggest a better question for a leader than whether another training course is available. Can people practise during the working day?
In the survey, 42% report paying out of pocket for approved tools. The survey also records 21% using unsanctioned tools. Treat that stack as evidence of unmet needs and useful experiments. Ask what works, then establish a permitted route and a payment path. Personal spending should not become the price of participation.
Katie Dill, quoted in the report, names three adoption blockers. People have no room for risk, permissions are unclear, or nobody knows how to pay. Clearing them is concrete management work.
"we are all 'early career.'"
Share the failed attempt alongside the fix. A polished demonstration hides the part that another learner most needs to understand. Voice mode can help you explain the workflow in detail. Turn that explanation into a brief that someone else can run and challenge.
Build across roles and keep an owner
The Teams chapter reports that 65% are taking on adjacent product and engineering tasks. It also finds 34% reporting messier collaboration and less clear ownership. Broader capability creates coordination work. Small teams need operating discipline early.Teams, role changes. The report describes adjacent product and engineering work and unclear ownership. The three-capability model and the proposed division of responsibility here are my synthesis. Neither the percentages nor the diagram estimate how many roles a business can remove. Full source note 2
I think dependable delivery needs three capabilities. They can overlap in one person, but I would build them across a mixed team.
Dependable delivery needs all three
Explore the responsibilitiesA mixed team owns the result
The specialist can explain an exception precisely enough to test. The engineer understands its commercial consequence. The agent uses the agreed context, tools and checks.
Keep accountable. Name the workflow owner. Experienced engineers remain responsible for production architecture, maintenance and reliability.
Understand the work and its economics
Own the process, incentives, decision rights and exceptions. Define what an acceptable result means and why an exception matters commercially.
Without this capability. The team can automate a process that solves the wrong problem, applies the wrong rule or saves time nobody can use.
Make the workflow dependable
Own integrations, Git and pull requests, permissions, recovery and monitoring. Decide how a reviewed change reaches production and who maintains it.
Without this capability. A prototype can work once but fail at an integration, expose excess permissions or leave nobody able to recover the service.
Make the method repeatable
Own context, harness and graph engineering. Move working instructions into AGENTS.md, task methods into skills and acceptance rules into verifiable checks.
Without this capability. The result depends on a one-off prompt. Missing context and untested handoffs return each time the work starts again.
Domain and commercial understanding supplies the economics, process, incentives and exceptions. The specialist needs to explain what a good decision requires and who has authority to make it. An exception must be specific enough to test. The engineer needs to understand why it matters to the business.
Production engineering makes that decision process work with real systems. It covers integrations, version control, permissions, failure recovery and ongoing reliability. A domain specialist who can build a prototype still needs an experienced engineer to own production architecture and maintenance.
The agentic layer makes the model's work repeatable. An agent is an AI system that uses tools across several steps. Its craft includes three disciplines, defined in the Glossary.
- Context engineering organises the information the agent needs, with its source, freshness and retrieval method. Practise by checking which inputs reached one failed run.
- Harness engineering connects instructions, tools, permissions, checks and stopping rules. Practise by turning a repeated correction into a reusable skill, then testing it.
AGENTS.mdholds working instructions. Skills hold task methods. The harness connects them to execution. - Graph engineering makes the work's dependencies explicit. Practise by drawing one workflow, marking what can run together and what must wait for evidence or a person's decision.
A good learning task crosses these boundaries. The specialist writes the exception and the acceptance rule. The engineer makes the integration safe. Together they inspect the agent's result and decide whether it meets the rule.
Keep one named owner for the workflow and explicit owners for production changes. Shared understanding helps people collaborate. Responsibility still needs a name. Non-engineers gaining implementation skills does not make them the engineers on call.
Be your own first customer
Start with a recurring piece of your own work. You understand the inconvenience and can inspect the result. Keep its scope small enough to recover from a failed attempt.
Name the owner and map the baseline before building. Record the trigger, inputs, decisions, handoffs, waiting time and final output. Agree the exceptions and the standard for acceptance. Choose the model and route deliberately, then keep them recorded while you compare runs.
Build one complete example before expanding it. Write your own evaluations against representative inputs, awkward exceptions and known failures. An evaluation, or eval, tests the behaviour you need. Check both that the agent completes the intended action and that it stops when a needed input or permission is absent.
Keep a failure log with the input, observed behaviour, cause if known, correction and result of the next check. Preserve the difference between a diagnosis and a guess. Test whether the correction improves fresh examples as well as the case that exposed it.
Use operating measures to decide whether the workflow deserves more investment.
Measure the work that survives review
- Throughput
- Accepted outputs per week, alongside elapsed time per workflow.
- Review and rework
- Human minutes spent checking and correcting each accepted output.
- Errors
- Failures per case, including those discovered after acceptance.
- Full cost
- Tokens, tools, engineering and human checks, including failed runs.
Count throughput as accepted outputs over time. Record human review and rework separately from model runtime. Measure errors against an agreed definition and keep track of errors found after acceptance. Full cost includes tokens, tools, engineering support and human checks. Value time at a stated rate and count each charge once.
Seat counts and prompt counts can show use. The investment decision needs the resulting change in delivery. Hours released become value when someone uses that capacity or realises the saving. A workflow that creates more review work needs another iteration, even if its first draft arrives faster.
Assess judgement and teach the next cohort
The Teams chapter describes formal reviews, career ladders and hiring criteria lagging behind expanding expectations. It also reports hiring emphasis on AI fluency, systems thinking and strategic judgement, with craft still important. Joel Lewenstein describes the need to rethink one's process repeatedly as tools change.Teams, hiring and careers. Fishbowl and Rework are credited there to Garrett Fowler, founder of Offsite. The report closes with open questions about juniors and apprenticeship. The supervision and assessment criteria proposed here are my recommendations. The learning curve and repeated-practice method draw on Becoming Engineers. Full source note 3
I would look for evidence that someone can revise a method after seeing it fail. Ask why they chose the workflow, what they checked and which part they would still give to a specialist. Adaptability includes recognising when a useful new tool makes an old process unnecessary.
The apprenticeship question is harder. Junior execution has also been how people learn to recognise a good result. If agents do that work, teams must deliberately preserve the learning inside it. Supervising a machine requires knowledge that beginners are still acquiring.
I would give juniors small, real workflows under supervision. Ask them to attempt the reasoning, predict likely failures and check an agent's output against the source. Then review disagreements with an experienced practitioner. Have them repair a flawed result and explain why the repair is better.
Teach the underlying craft as part of that review. Trace a number to its inputs. Read the changed code. Explain why a design decision helps its user. Rotate responsibility for verification and let juniors see the consequences of errors before increasing their scope.
The cost of that supervision belongs in the team's plan. A senior who can generate more work still needs time to teach. Faster production alone gives us no assurance that the next generation is learning to judge it.
Keep capability in the four-factor loop
Building AI capability changes the other factors in The Token Department. A skilled user can recognise which model a task needs, improve its route and repair the harness. A new model or harness can also change what that person needs to learn.
Review those decisions together. Start the next cycle from accepted work, costs and failures. Keep the methods that transfer. Retest those that depend on a particular model or tool. The practical return on learning is a better next run with less correction from you.
Key takeaways
- Learn through a bounded workflow that you can inspect, repeat and recover.
- Give peers time, permission and a payment route to develop and share useful methods.
- Combine domain judgement, production engineering and agentic craft, with named responsibility for delivery.
- Measure accepted work, review effort, errors and full cost against a baseline.
- Assess recovery and judgement in live work. Teach juniors the craft required to verify it.
- Revisit model selection, effective routing, harness quality and user skill together.
Sources and notes
-
AI in Design 2026, Teams, Designer Fund and Foundation Capital. Survey findings are labelled Q1 2026. The report combines design-sector survey responses with practitioner interviews. Its methodology describes 900+ responses and notes that the sample may skew towards active AI adopters. Treat the findings as directional, rather than a population census. The percentages describe its respondents, rather than all workers or organisations. The 2025 and 2026 comparisons are those reported by the authors. Support and tinkering are associated, with no causal test established here. The funding categories can overlap and should not be added together. ↩
-
Teams, role changes. The report describes adjacent product and engineering work and unclear ownership. The three-capability model and the proposed division of responsibility here are my synthesis. Neither the percentages nor the diagram estimate how many roles a business can remove. ↩
-
Teams, hiring and careers. Fishbowl and Rework are credited there to Garrett Fowler, founder of Offsite. The report closes with open questions about juniors and apprenticeship. The supervision and assessment criteria proposed here are my recommendations. The learning curve and repeated-practice method draw on Becoming Engineers. ↩
