Jev: Inside TypeSafe AI's First System One Decision Model

Jev is not a language model in the usual sense. TypeSafe AI released the Jev AI model on 15 September 2026 as the first of what it calls a System One model: an engine that takes a state and one or more questions and returns typed answers with probabilities, never a sentence. The company reports 70 to 500 millisecond end-to-end latency, together with headline claims of up to 200 times faster and 400 times cheaper than comparable large language model workflows on the same class of decision.

This piece looks past the announcement. It explains the mechanism that lets Jev skip the sentence entirely, reproduces that mechanism against an open model running on a single Hyperstack NVIDIA GPU, and checks TypeSafe’s own numbers against an independent benchmark that LangChain published within days of launch.

This piece draws on four sources throughout: TypeSafe’s own announcement and documentation, the SGLang documentation behind the scoring mechanism reproduced further down, and LangChain’s own evaluation of Jev against three large language model judges, published as Jev-as-a-Judge for Agent Evals. Figures that come from TypeSafe’s own comparisons are marked as such wherever they appear, separate from the independent measurements.

The Decision Your Code Keeps Paying Full Price For

Most of what software asks a language model to do is not writing. It is a judgement with a small number of possible answers: is this support ticket urgent, which team should own it, is a shell command safe to run, does a retrieved passage answer the question that was asked. None of that needs a paragraph. It needs one of a handful of known labels.

The usual approach reaches for the same tool regardless of the question: a full generative call, a sentence or a JSON object to wait for, and code that parses that answer back into the label it wanted in the first place. Inside an agent loop, that pattern repeats on almost every iteration.

while not task_done:
    action = llm(context)          # full generation, just to pick a tool
    result = run_tool(action)
    context += result
    task_done = llm(context)       # full generation again, just to check "done or not"

Every pass through that loop calls a generative model twice for a judgement with only a handful of possible answers: which tool to call next, and whether the task is finished. Both are single-token decisions, priced and timed as though they were prose.

The gap between the two ways of answering comes down to how many times the model has to run. Writing an answer means running the model once per output token, because each token depends on the one written immediately before it. A bounded decision needs exactly one pass, because the set of possible answers is already known before the model is ever called.

Jev’s premise, as TypeSafe AI frames it, is that language generation is the wrong interface once code already knows the set of possible answers. If every outcome can be enumerated ahead of time, the system needs a model that scores those outcomes, not one that writes about them.

What Jev Is

TypeSafe’s own documentation defines System One models as a class of AI models built to make fast, structured decisions that software can use directly. A System One model evaluates a state and returns typed answers and probabilities, and it does not write replies, produce code, or explain its own reasoning.

A request to Jev carries two things: a state, which is the text or JSON describing the situation, and one or more questions about that state. Every question declares its answer shape in advance, using one of three primitives.

Choice selects one option from a list defined by the caller and returns a probability for every option, not only the winner. Score places the input on an ordered scale defined by the caller and returns a probability-weighted position on that scale rather than a single label. Noul, TypeSafe’s own name for the yes-or-no primitive, returns the probability that a stated question is true. Every answer comes back as a number between 0 and 1 that application code can act on directly, with no option outside the ones the caller declared.

A Score answer such as 1.74 on a zero-to-two scale is not a rounding artefact. It is the probability-weighted average across every rung on the scale, so a result close to 2.0 signals near certainty on the top label, while 1.74 signals a strong lean towards the top label with real weight left on the one below it, information a single winning label would discard.

A single request can carry several questions about the same state, and every one of them is answered from the same evaluation, not a separate call for each.

{
  "model": "jev-latest",
  "state": "I was charged twice for the same subscription.",
  "questions": {
    "urgent": {
      "type": "noul",
      "instructions": "Does this need attention right now?"
    },
    "team": {
      "type": "choice",
      "instructions": "Which team should own this ticket?",
      "criteria": {
        "billing": "Charges, invoices, refunds",
        "technical": "Product bugs and errors",
        "account": "Login and access problems"
      }
    }
  }
}

{
  "urgent": {"probability": 0.61},
  "team": {
    "choice": "billing",
    "probabilities": {"billing": 0.91, "technical": 0.06, "account": 0.03}
  }
}

Application code reads that response directly. There is no prose to parse and no answer outside the declared options, because team can only ever be one of the three names supplied in the request.

How This Differs From Structured Output

It is easy to confuse Jev’s mechanism with structured output, because both restrict what an application receives. The two do different work inside the serving engine.

A structured-output call constrains a model’s generation to a schema, so a request for a ticket’s team might return {"team": "billing"}. The schema prevents an invalid object, but it does not guarantee a correct one. The model still writes that object one token at a time: the opening brace, the field name, the value and the closing brace, decoded exactly as a sentence would be. A schema narrows the shape of the output. It does not remove the decode loop underneath it.

Jev’s fixed-answer scoring exits before that loop starts. Given the same three allowed teams, it reads one score per team from a single forward pass and normalises them into a distribution, with no token of output ever generated.

Where That Probability Comes From

Seeing why Jev can skip the decode loop starts with what a normal generation step does. A tokenizer converts a prompt into token IDs. The model processes that sequence and produces one vector for the next position, holding a raw score, a logit, for every token in the vocabulary, tens of thousands of numbers for a typical open model.

Turning a logit into a probability means running softmax across every other logit in that whole vocabulary.

Ordinary generation does not even run that plain version. It divides every logit by a temperature first, sharpening the distribution below 1 or flattening it above 1, before a token is sampled.

That whole cycle, a forward pass, decoding rules, one appended token, repeats until a stop token or a length limit ends it.

Reading Three Logits Instead of Tens of Thousands

A bounded decision only needs the very first vector, and only a handful of its entries. Labelling three answers A, B and C inside the prompt, and ending that prompt exactly where the model would normally start writing the label, puts the logits for all three at a single, known position. The model still produces its usual vocabulary-sized vector for that position. It holds the logit for A, the logits for B and C, and every other token in the vocabulary besides.

Every logit outside the three declared labels is discarded, and softmax runs only across the three that remain.

Why the Labels Are Single Letters, Not Whole Words

A visible word is not necessarily one token. billing might be one token for one tokenizer and several for another, and technical support spans more than one position under any tokenizer. Comparing multi-token phrases would need sequence scoring: score the first token, append it, score the next, and combine the values for the whole phrase, with length becoming part of the comparison. A single-letter label avoids that problem entirely, because every option sits at exactly one output position while the meaning behind it still lives in the prompt next to the letter.

A second trap sits underneath the first. Tokenizers often fold a leading space into a token, so A and  A can resolve to two different token IDs, and a chat template can insert that whitespace without it being obvious from the prompt text alone. The way around this is to render the full prompt through the model’s own chat template, check the exact continuation it expects at the answer position, and confirm through a tokenizer call that each label resolves to exactly one token before trusting a scoring request against it.

Reproducing the Mechanism on a Hyperstack NVIDIA GPU

Jev itself is closed. Its weights, its Reinforcement Learning for Calibrated Decisions training run and its evaluation stack are not published. The restricted-softmax mechanism behind it, however, already sits inside open serving engines, which means the trick can be built and measured against a model under full control rather than taken on trust. The steps below reproduce that mechanism using SGLang, run here on a single Hyperstack NVIDIA H100 virtual machine rather than a laptop.

SGLang is the same serving engine behind the Kimi K3, the Qwen3.8 Max and the MiniMax H3 tutorials, so a virtual machine already configured for one of those deployments needs no extra setup for this exercise. A single NVIDIA H100 is more GPU than a 0.5 billion parameter model needs, which is the point: the model here is small on purpose, chosen so the mechanism, not the hardware, is what gets measured.

Starting the Model Server

A clean environment needs two packages: SGLang itself, and requests for the client that follows.

python3 -m venv .venv
source .venv/bin/activate
pip install "sglang[all]==0.5.10.post1" "requests==2.34.2"

The server is the only process doing GPU work. Everything after this is a client script talking to it over HTTP.

python -m sglang.launch_server \
    --model-path Qwen/Qwen2.5-0.5B-Instruct \
    --host 127.0.0.1 \
    --port 30000

The first launch downloads Qwen2.5-0.5B-Instruct from Hugging Face. Later launches reuse the local cache, and the model stays resident in GPU memory on port 30000 for every request that follows.

Defining the Choices and Building the Prompt

The client, decide.py, starts with the labels and the ticket being routed. The dictionary keeps both representations of every answer together, the letter that gets scored and the meaning returned to the application, so their order cannot drift apart between tokenizing, scoring and mapping the result back.

# decide.py
import json
import requests

BASE_URL = "http://127.0.0.1:30000"
MODEL = "Qwen/Qwen2.5-0.5B-Instruct"

# A is the token that actually gets scored. "billing and payments" is what
# the application does with the answer once it comes back.
choices = {
    "A": "billing and payments",
    "B": "technical support",
    "C": "account access",
}
ticket = "I was charged twice for the same subscription."

# decide.py (continued)
choice_lines = "\n".join(
    f"{label} = {meaning}" for label, meaning in choices.items()
)
prompt = f"""Ticket:
{ticket}
Question:
Which category matches the ticket?
Allowed labels:
{choice_lines}
Return only the label.
Label:
"""
print(prompt)

Ticket:
I was charged twice for the same subscription.
Question:
Which category matches the ticket?
Allowed labels:
A = billing and payments
B = technical support
C = account access
Return only the label.
Label:

The prompt ends at Label:, the exact position whose vocabulary vector gets read. SGLang is never asked to generate the token that would normally follow it.

Resolving the Label Token IDs

Before anything can be scored, the real token ID behind each of A, B and C has to be confirmed against this specific tokenizer, not assumed from any other model.

# decide.py (continued)
label_token_ids = []

for label in choices:
    response = requests.post(
        f"{BASE_URL}/tokenize",
        json={"model": MODEL, "prompt": label, "add_special_tokens": False},
        timeout=30,
    )
    response.raise_for_status()
    token_ids = response.json()["tokens"]
    if len(token_ids) != 1:
        raise ValueError(f"{label!r} is not a single token: {token_ids}")
    print(f"{label!r} -> {token_ids}")
    label_token_ids.append(token_ids[0])

'A' -> [32]
'B' -> [33]
'C' -> [34]

Each label returns a single integer, so every option occupies exactly one vocabulary position. Had any label produced two or more IDs, the raise ValueError above would have stopped the script there rather than let a silent mismatch reach the scoring call.

Scoring the Three Positions

With the token IDs confirmed, the decision itself is a single HTTP call to SGLang's /v1/score endpoint, documented as computing token probabilities for specified tokens given a query, intended for classification tasks, scoring responses and log-probability computation.

# decide.py (continued)
response = requests.post(
    f"{BASE_URL}/v1/score",
    json={
        "model": MODEL,
        "query": prompt,
        "items": [""],
        "label_token_ids": label_token_ids,
        "apply_softmax": True,
    },
    timeout=120,
)
response.raise_for_status()
score_response = response.json()
print(json.dumps(score_response, indent=2))
scores = score_response["scores"][0]

{
  "scores": [
    [
      0.6777623295783997,
      0.31087860465049744,
      0.011359035037457943
    ]
  ]
}

query carries the full prompt. The empty string inside items tells SGLang to score the position immediately after it. label_token_ids names the three positions to read, and apply_softmax normalises those three readings into a proper probability distribution. SGLang tokenized the prompt, ran the model once, and returned three numbers in the same order the token IDs were sent, with no token generated to produce them.

Mapping the Scores Back to a Decision

The last step is ordinary code, turning three floats back into the labels the application understands.

# decide.py (continued)
probabilities = {
    choices[label]: float(score)
    for label, score in zip(choices, scores, strict=True)
}
decision = max(probabilities, key=probabilities.get)
print(json.dumps({"decision": decision, "probabilities": probabilities}, indent=2))

{
  "decision": "billing and payments",
  "probabilities": {
    "billing and payments": 0.6777623295783997,
    "technical support": 0.31087860465049744,
    "account access": 0.011359035037457943
  }
}

Billing wins, but only at 0.678, well short of the 0.9-plus a reader might expect. A threshold of 0.70 would catch exactly this case and send it for a second look rather than route it without review, which is the kind of decision that belongs in application code rather than inside the model.

Measuring the Difference Against Standard Generation

The mechanism explains why scoring should be faster. Measuring it is more convincing than assuming it. The scoring path above was run against a standard chat-completion call on the same SGLang server, the same model, and the same 100 support-ticket cases, both starting at the same instant. Standard generation decoded up to 32 tokens per answer, an explanation alongside the label.

That gap is also a cost figure. Every millisecond a request holds a GPU is a millisecond another request cannot use it, so the same measurement that explains Jev’s latency claim also explains why a workload built on this pattern needs less GPU time per decision than one built on ordinary chat completions, whether the model behind it is Jev or an open one served from a Hyperstack GPU billed by the minute.

Why the Number Next to the Answer Matters

A typed answer solves half the problem. The label tells application code what won. The probability tells it how close the race was, and that second number is what the code should branch on.

{
  "choice": "billing",
  "probabilities": {"billing": 0.52, "technical": 0.46, "account": 0.02},
  "confidence": 0.06
}

Routing that ticket without a second look would be reckless. Billing won, but barely, and a low-confidence win should never take the same branch as a decisive one. One workable definition of confidence is the margin between the top answer and its closest rival, rather than the winning probability taken alone.

if probabilities["billing"] > 0.9:
    route_automatically(ticket, team="billing")
elif confidence < 0.3:
    send_to_human_review(ticket)
else:
    escalate_to_larger_model(ticket)

TypeSafe trains Jev with what it calls Reinforcement Learning for Calibrated Decisions, RLCD for short. The target is calibration, not only accuracy: a batch of answers each carrying 90 per cent confidence should turn out correct on roughly 90 per cent of those cases. Measuring that means binning every prediction by its stated confidence and comparing each bin’s average confidence against how often that bin was right.

The Hallucination Claim, Examined Closely

TypeSafe says Jev cannot hallucinate. That statement holds under a narrow definition, and it is worth separating from a broader one that it does not support.

Declaring billing, technical and account as the only valid answers means the response can never come back as legal, an option that does not exist in the schema. It also cannot return malformed prose where code expected a label. That part of the hallucination problem does not apply to it.

It can still confidently choose the wrong valid option. A login bug scored as billing instead of technical is schema-valid and still wrong, and now the wrong customer gets refunded over an issue that was never about a charge. A schema rules out an answer that does not exist. It says nothing about whether the answer that does exist is the right one.

Where Jev Earns Its Place Inside an Agent

As a System One model, Jev works best alongside a generative model, not instead of one. The generative model plans, writes, explains and calls tools, the work that needs language. Jev handles the frequent small decisions that surround that work.

Model Routing

A simple lookup does not need the same model as a demanding architecture review. Jev can score the incoming request and select the least expensive model likely to complete it. The router decides which model should answer, not the answer itself.

Tool Risk Gating

Before an agent runs a shell command, Jev can classify it as read-only, reversible or destructive, alongside separate questions about whether it touches production or leaves an approved directory. A high-confidence read-only command can continue unattended. A destructive or uncertain one pauses for a person.

{
  "model": "jev-latest",
  "state": "rm -rf /data/tmp",
  "questions": {
    "risk": {
      "type": "choice",
      "instructions": "Classify this shell command.",
      "criteria": {
        "read_only": "Inspects state, changes nothing",
        "reversible": "Changes state, but the change can be undone",
        "destructive": "Deletes data or is otherwise hard to undo"
      }
    }
  }
}

Verification

An agent can report that a task is finished while its tests are still failing. Jev can inspect the state and answer bounded questions: did the tests pass, is the agent repeating the same failed action, does the output follow policy. A hard test still catches what it was built to catch. Jev adds a semantic check for what that hard rule was never designed to see.

Problems Jev Can Solve Today

Jev sits within a broader set of AI models for automation, but it fits a specific niche inside that set. The workloads where it earns its keep share three properties: every possible answer can be named ahead of time, a careful person could judge the input quickly by hand, and the decision happens often enough that latency or cost start to matter.

Does This Hold Up? An Independent Benchmark

TypeSafe’s own numbers for the Jev AI model deserve a careful reading. A company benchmarking its own product tends to choose the comparison that flatters it, and TypeSafe AI reports 70 to 500 millisecond latency, $0.042 per million input tokens with output free, and headline claims of up to 200 times faster and 400 times cheaper than comparable large language model workflows. Those figures are a ceiling worth treating with care, not a promise for every workload, even where the underlying mechanism behind them is real and measured earlier in this piece.

A comparison run by a party with nothing to gain from Jev looking good is worth more. LangChain ran exactly that kind of test, comparing Jev against three large language model judges, GPT-5.6 Luna, GPT-5.6 Terra and Claude Sonnet 4.6, using its own Deep Agents harness and LangSmith for the dataset. A weather agent answered five fixed requests, each response was captured once, and every judge scored the same frozen traces across 100 repetitions, so differences in the results measure the judge rather than the agent.

Across 500 repeated pass or fail judgements against a human oracle, Jev matched the human on every single one. GPT-5.6 Terra matched on 99.8 per cent, GPT-5.6 Luna on 96.4 per cent, and Claude Sonnet 4.6 on 80.0 per cent.

Accuracy alone tells half the story, since a judge can be right on average while swinging between different answers on the exact same input from one run to the next. Measuring variance across 100 repetitions on a single frozen case is where that gap opens further.

Cost and latency followed the same pattern. Jev averaged $0.00035 and 0.44 seconds per call. GPT-5.6 Luna averaged $0.00039 and 2.50 seconds, GPT-5.6 Terra $0.00289 and 2.83 seconds, and Claude Sonnet 4.6 $0.02811 and 2.16 seconds. The full project cost $0.34 for Jev against $28.17 for Claude Sonnet 4.6.

LangChain combined accuracy and consistency into a single signal value: oracle agreement multiplied by repeatability, the chance that two independent calls on the same frozen trace reach the same verdict.

One agent, five test cases and a narrow evaluation slice is not the final word on every workload. It is, however, an independent measurement rather than a vendor’s own comparison, and it points in the same direction the mechanism itself predicts.

Where Jev Falls Apart

Jev becomes less useful the moment the answer space stops being known ahead of time.

• It cannot write a response, summarise a document, generate code, or explain its own reasoning.
• It is unreliable at arithmetic, counting, date comparison or exact string manipulation. Those belong in ordinary code.
• It struggles when a decision needs several hidden reasoning steps chained together. Splitting the judgement into smaller questions, or using a reasoning model, works better.
• It cannot extract an unknown value directly out of text. Finding candidate values first with something else, then letting Jev choose among them, is the workable pattern.
• Irrelevant context can reduce its accuracy, so only the state a decision needs should be sent.
• Closed weights, an early access window, text-only input and still-thin independent calibration data outside the one benchmark cited above mean it is early to place full trust in a decision with real consequences.

A simpler rule sits underneath all of this. If deterministic code already solves the problem correctly, the code should stay. A plain if statement is faster, cheaper and easier to test than any model.

A Sane Way to Roll This Out

A cheap model can still turn out expensive if its mistakes generate retries, manual review or an incident, so a rollout has to measure the whole workflow, not the token price on its own.

1. Choose one bounded, low-risk decision. A small, clearly named set of possible answers, not the riskiest step in the pipeline.
2. Write the rubric before calling the model. Decide in words what belongs in every option. The rubric is part of the system, not an afterthought added later.
3. Collect representative examples first. Include the ambiguous and adversarial cases on purpose.
4. Run it in shadow mode. Let it answer every real case without changing behaviour, and log its probabilities beside the current outcome.
5. Plot accuracy against confidence and set thresholds from that data. The cut-off should come from the shadow-mode numbers, never a guess.
6. Automate the safest branch first. Keep a person, or a stronger model, as the fallback for anything the thresholds flag as uncertain.
7. Version everything. Pin the model version, the questions, the criteria and the thresholds together, so a change can be replayed against the same evaluation set.

Raising the bar too far means the system never makes a mistake because it never decides anything, which is not automation. It is expensive logging. The coverage constraint above keeps that from happening.

Build the scoring pattern on your own NVIDIA GPU

Try a Jev-Style Decision Engine on Hyperstack

One NVIDIA H100, SGLang, and an open model are enough to measure the pattern behind Jev for yourself, on the same platform already serving Kimi K3, Qwen3.8 Max and MiniMax H3.

One NVIDIA H100 GPUSGLang /v1/score463 ms measuredBilled by the minute

Launch an NVIDIA GPU virtual machine on Hyperstack today.

FAQs

What is Jev?

The Jev AI model is TypeSafe AI’s first System One model, released 15 September 2026. As a probabilistic AI model, it takes a state and one or more typed questions and returns typed answers with probabilities in a single forward pass, rather than generating text, and cannot write prose, code or explanations.

Can Jev’s scoring mechanism be reproduced without TypeSafe’s API?

Yes. SGLang’s /v1/score endpoint exposes the same restricted-softmax mechanism used by Jev against any open model, so a Choice or Noul-style answer can be computed from one forward pass on a Hyperstack NVIDIA GPU, with no access to Jev’s own weights required.

How is Jev different from structured output?

Structured output constrains a class of AI models for structured outputs to a schema, but generation still happens token by token underneath it. Jev’s fixed-answer scoring reads three or more logits from a single forward pass instead, and never enters the decode loop, so no token is ever generated.

How much GPU do you need to reproduce this pattern?

A single NVIDIA GPU is enough for a small model such as Qwen2.5-0.5B-Instruct, the model used in the walkthrough above. Production workloads with larger models still need only one forward pass per decision, so the sizing question is model size, not decision volume.

Does Jev replace LLM-as-judge evaluation?

Not entirely. Jev belongs to a small set of LLM alternatives for automation built specifically for bounded decisions, and LangChain’s benchmark found it more accurate and consistent than three LLM judges on a narrow agent-evaluation task, at a fraction of the cost. Open-ended judgements that need written reasoning still belong to a generative model, not a typed decision engine.