Skip to content

Running a Token Department

Organise seven practical tools around model selection, routing, the harness and user skill.

A Token Department needs shared tools that help teams turn AI capacity into accepted work. Organise those tools around the four factors that shape the return: model selection, routing, harness quality and user skill.

Start with an important workflow and an accountable owner. Agree what an acceptable result looks like, which information the system may use and where someone must approve its actions. Buy the capacity, connect the tools and measure the work that actually passes review.

Connect the tools to the four factors

Model selection needs records of usage and spending. Routing needs a maintained way to assign tasks to exact models and effort settings. The harness needs connections, permissions and information protection, plus observability and session analytics. User skill needs training and practice, with feedback from the work people deliver.

Each tool view is illustrative. All values and account states are synthetic.

Four factors connect to seven tool groups through their primary ownership. Select a tool to inspect an illustrative screen. Model selection owns usage and spending. Routing owns routing. The harness owns connections, permissions, information protection, and observability and session analytics. User skill owns training and practice, including the existing learning management system, supervised practical work and reviewed feedback. Session evidence informs all four factors.

The circles are the existing four factors. The seven tool groups describe the capabilities needed to manage them. One product may cover several groups, and existing enterprise systems may already supply much of the tooling. The lines show primary ownership. Evidence should still travel between the groups.

Inspect the tools through one workflow

The example screens follow a company briefing, the workflow used in the Model Routing essay. They illustrate what someone running the department might inspect or configure. Every value and account state is synthetic.

Example screens and synthetic values. These illustrate the tooling, with no live accounts or working enterprise controls.

Usage and spending

Model selection. Forecast demand, fund capacity and account for the work.

Usage meter and budget controlsIllustrative configuration

Budget for the work

Illustrative monthly allocation, including all attempts.

US$120 usedUS$200 cap
US$0US$200
Spending by workflow
WorkflowExample spend
Company briefingsUS$80
Source classificationUS$30
Draft preparationUS$10

Budget records support the next purchase. Usage windows and human review still need their own records.

Return to the four-factor map

Routing table

Routing. Choose an eligible model and effort for each task.

Routing table and model gatewayIllustrative configuration

Route the company briefing

Bind each route to an approved model and effort.

Illustrative routing table
TaskModel roleEffort
Sort sourcesSmall modelLow
Check figuresCapable modelHigh
Draft the briefCapable modelHigh
Review the resultIndependent routeHigh

Model roles are placeholders for exact approved configurations. Check the route and fallback against the work.

Dispatch requires the correct tools, permission and usable capacity. The workflow owner accepts the final briefing.

Return to the four-factor map

Connections

Harness. Connect approved models, information and tools.

Connector catalogueIllustrative configuration

Approved connections

The briefing can reach these sources and tools.

Model gateway
Approved primary and fallback models
Company files
Source documents for this briefing
Working notes
Retained context and corrections
Draft workspace
Files created during the workflow

Record each connection's owner, purpose and available actions.

A connection makes a service available. The permission rules determine what this person or agent can do through it.

Return to the four-factor map

Permissions

Harness. Control who can read, change and approve.

Identity and access controlsIllustrative configuration

Agent service account

An illustrative policy for the company-briefing workflow.

Read approved source files
Allowed
Edit the working draft
Allowed
Send the briefing externally
Owner approval
Change access rules
Administrator approval

Separate read, write and approval rights. The shared function sets the policy with the relevant system and business owners.

Return to the four-factor map

Information protection

Harness. Apply data rules before information or actions leave.

Data and action policy checksIllustrative configuration

Check information before dispatch

A synthetic example of one information-handling rule.

Internal sourceApply the data ruleApproved model
Before the check

Client name: Example Co
Task: Summarise the supplied source.

After the check

Client name: [redacted]
Task: Summarise the supplied source.

Redaction is one possible rule. The organisation also decides which information may leave and where the model may run.

Control tool actions as well as model requests. An agent writing to a business system needs an appropriate execution policy.

Return to the four-factor map

Observability and session analytics

Harness. Understand sessions, costs, outcomes and working methods.

Session instrumentation and evaluation evidenceIllustrative configuration

Company briefing session

An illustrative session with synthetic tokens, times and review outcomes.

Task
Company briefing
Owner
Briefing team
Model
Capable model
Effort
High
Elapsed
12 minutes
Review outcome
Accepted after correction
40,000 tokens
Uncached input
28,000
Cached
8,000
Output
4,000
Session evidence for all four factors
FactorWhat to examine
Model selectionModel quality, cost and latency. Compare the reviewed result, recorded cost and elapsed time for the selected model and effort.
RoutingAllocation, retries and fallbacks. Inspect the task allocation and any retries or fallback routes before changing dispatch rules.
HarnessTools, context and approvals. Investigate tool failures, missing context and approval interruptions in the session record.
User skillBriefing, delegation and review. Review how the user briefed the task, delegated work and checked the result. Token volume alone does not establish competence.
Session timeline
  1. Brief received

    Owner, sources and acceptance standard recorded.

  2. Route selected

    Model, effort, instructions and tools recorded.

  3. Review returned

    Figure checks and the requested correction retained.

  4. Owner accepted the brief

    Final result and review outcome linked to the run.

The harness owns instrumentation. Its evidence serves all four factors. Link recurring errors to workflow checks and focused practice in the LMS, then review the next result.

Return to the four-factor map

Training and practice

User skill. Connect the LMS to practical work and feedback.

Learning management system and practice recordsIllustrative configuration

Learning connected to the work

Use the existing LMS for the learning record.

  1. Learn in the LMS

    Assign the module on briefing, sources and approval rules.

  2. Practise the workflow

    Complete a company brief under supervision.

  3. Review the evidence

    Assess the brief and record feedback on the demonstrated skill.

Next learning assignment
The run review found a source-checking error. Assign a focused practice case, then review the next result.

Some corrections also become reusable instructions or checks in the harness.

Course completion records participation. Practical review supplies evidence of capability under the conditions tested.

Return to the four-factor map
Seven illustrative screens follow one company-briefing workflow. All amounts, times and account states are synthetic. The screens describe proposed capabilities, with no live accounts or working enterprise controls. Course completion records participation. Reviewed practical work provides evidence of skill under the conditions tested.

Model selection needs usage and spending records

Forecast complete workflows, including retries and tools. Compare the capacity you buy with actual use. Keep shared pools, individual limits and their reset windows visible. A monthly budget can fit while a short usage window prevents an important job from finishing.

Attribute the bill to the work it supports. Include the cost of unsuccessful attempts and the shared operating service. Record human review separately, with a stated rate if you put a money value on it. More accepted work can justify more spending. The purchase should follow the work and its required standard.

Routing needs an explicit allocation method

Maintain a routing table that names an exact model and effort for each kind of task. Check that the route has permission, tools and enough capacity before dispatch. Choose the least costly reliable route that meets the standard, allowing for review and rework.

A gateway can apply these rules to connected requests. Keep a tested fallback with the context and tools needed to resume. A model name in a backup column gives you a candidate to test.

The harness needs connected controls and evidence

Connections make approved sources, model endpoints and tools available. Maintain a catalogue that records what each connection reaches and who owns it. Test the connection in the application where the agent will work.

Permissions govern what each person and agent may do through those connections. Reading source material, changing a draft and sending a deliverable need separate decisions. Give actions with serious consequences an explicit approval rule.

Information protection applies the organisation's rules to data and actions. A request may need restricted fields removed or an approved execution environment. Check the actual route taken by the information. Tool actions also need controls around the files and systems they can change.

Observability and session analytics record what happened across a workflow. Link the brief, model and effort settings, context, tool calls, token use, attempts, approvals, costs and review to the result. Traceability is part of this wider record. Logs need appropriate access and retention because they can contain sensitive information.

The harness supplies the instrumentation. The evidence supports decisions across all four factors. Compare models on comparable work, inspect routing and fallbacks, investigate context or tool failures, and review how people brief, delegate and check the output. Interpret activity alongside the resulting work and review outcome. Token volume alone does not measure skill.

User skill needs training tied to practice

Use the existing learning management system, or LMS, where it can hold the relevant courses, assignments and learning records. Connect it to practical tasks that exercise briefing, delegation, judgement and review.

Course completion records participation. Reviewed work provides evidence of what a person can do under the conditions tested. Give the learner a representative task, assess the result against a clear standard and record the feedback. Decide when the learner is ready to work with less supervision.

Bring recurring mistakes back into the training material. Some corrections also belong in the harness as a reusable skill, instruction or evaluation. The run record helps identify the problem. The LMS can retain the learning assignment and evidence, through an integration or a manual record where needed.

Keep the work accountable

The shared function maintains model access, the operating rules and common tooling. Each business workflow also needs an owner who sets its standard, approves exceptions and accepts the output. Existing technology, procurement, security and learning teams can supply parts of the function.

Measure comparable workflows on accepted output per period, total cost per accepted result, time to acceptance, review effort and failures. Keep the cost of running the Token Department in that comparison. Use the evidence to decide which factor to improve next.

Start with the systems you already have. Add a tool when a specific gap prevents the workflow from meeting its standard or makes it unnecessarily costly to run.

What changes for the business

Know where capacity goes. See which workflows use it, who owns them and what the accepted output costs.

Match effort to the work. Choose routes that meet the required standard, with explicit review and fallback rules.

Control consequential actions. Apply information, access and approval rules consistently, with a record of what happened.

Improve capability through use. Turn reviewed work into better training, reusable methods and better choices next time.

Judge the department by the accepted work it enables, including the cost of running it.

Sources and notes

  • The Token Department introduces the four-factor framework and continuing organisational responsibility.
  • Model Routing explains total cost per accepted result and supplies the company-briefing example.
  • The Harness explains instructions, context, tools, permissions and execution.
  • Wonderful AI Gateway informed the six operational tool categories. Training and practice completes the mapping to the fourth factor.
  • Wonderful AI Transformation informed the discussion of workflow ownership and continuing capability development.

The tool map and example screens are original explanatory graphics. They describe a proposed operating arrangement. The examples demonstrate no live product integration, realised saving or measured training outcome.

Say hello

Draft a message about the work you need to complete.

Local draft preview. No message is sent. Review the draft, then copy it for use elsewhere.