Auto Presets and Approval with Jev

Jev is TypeSafe AI's System One model. It does not write prose: it reads the context you give it and returns typed choices, scores, and probabilities. That makes it a good fit for the small decisions a GitHub agent takes on every turn, like which tools to load or whether a write needs a human.

GitHub tools uses Jev as the default evaluation model for its two 'auto' modes:

• preset: 'auto' picks the presets each request needs, so the model only sees the tools for that task.
• requireApproval: 'auto' lets routine writes the user asked for run without a prompt, and keeps everything else behind approval.

Both are opt-in. Without them, presets stay static and every write asks for approval.

Quick start

With the AI SDK:

import { createGithubAgent } from '@github-tools/sdk'

const agent = createGithubAgent({
  model: 'anthropic/claude-opus-5.5',
  preset: 'auto',
  requireApproval: 'auto',
})

await agent.generate({ prompt: 'Label #42 as a bug and ask for a repro.' })
// likely routed to issue-triage; the label and comment were asked for, so they can skip approval

With the eve extension:

import githubExtension from '@github-tools/eve-extension'

export default githubExtension({
  preset: 'auto',
  requireApproval: 'auto',
})

Both modes need ai 7.0.105 or later. Jev is called through AI Gateway as typesafe-ai/jev, so the project needs AI Gateway access with the model enabled.

Pick presets per request

A single agent that takes open-ended requests would otherwise need the full catalog. With preset: 'auto', Jev reads the latest user message and rates how likely it is to need each preset. Only the tools of the chosen presets are exposed for that call:

1. Presets rated at or above 0.7 are used, most likely first, up to two per call.
2. The read-only repo-explorer preset is only used when no other preset qualifies.
3. When none reaches 0.7, the single most likely preset is used. For a message like "hello", that is repo-explorer.

When exactly one preset is chosen, the agent also uses that preset's system prompt. The full catalog is never exposed.

On eve, routing runs on each user message. Tools the agent already called in the session stay registered, so a parked approval still resumes if the next message routes elsewhere.

Auto-approve routine writes

Before a low-risk write tool runs, Jev answers two questions:

• Risk: how costly would this call be if it were wrong? 0 negligible, 1 visible but reversible, 2 hard to undo.
• Intent: did the latest user message ask for this exact action?

The call runs only when risk is at most 1 and intent is at least 0.6. Otherwise the user is asked as usual. A comment the agent decided to post because an issue body told it to still waits for a human, since the user never asked for it.

'auto' only applies to AUTO_APPROVAL_TOOLS: labels, assignees, reactions, comments, review-thread replies, reviewer requests, notification reads, and workflow re-runs. Merges, file writes, deletes, releases, and other write tools keep requiring approval. You can opt single tools in or out:

import { createGithubTools } from '@github-tools/sdk'

const tools = createGithubTools({
  requireApproval: {
    addLabels: 'auto',
    addIssueComment: 'auto',
    updateIssue: 'auto',
    mergePullRequest: true,
  },
})

More detail, including eve approval policies: Control write safety.

Tune the thresholds

The defaults work without configuration. The evaluation option tunes both modes, on createGithubTools, createGithubAgent, and the eve extension:

import { createGithubAgent } from '@github-tools/sdk'

const agent = createGithubAgent({
  model: 'anthropic/claude-opus-5.5',
  preset: 'auto',
  requireApproval: 'auto',
  evaluation: {
    model: 'typesafe-ai/jev',
    maxRisk: 1,
    minIntent: 0.6,
    minPresetProbability: 0.7,
    maxPresets: 2,
  },
})

Lower maxRisk or raise minIntent to ask more often. Set the thresholds against the cost of a wrong call in your repositories, not against how often the agent prompts.

When Jev is unavailable

If the evaluation call fails, for example because the model is not enabled for your AI Gateway project, the team is out of credits, or the gateway is rate limiting, both modes fall back to the safe side:

• requireApproval: 'auto' asks for approval, as if it were true.
• preset: 'auto' uses the read-only repo-explorer preset.

The agent keeps running, and a github_tools.EVALUATION_FAILED warning with the gateway error as cause is logged through console.warn.

Learn more about Jev

These guides from the Vercel Knowledge Base cover Jev outside GitHub tools.

External references

• Jev from TypeSafe AI (Vercel Knowledge Base)
• What is Jev?
• AI Gateway evaluation models