Non generative AI. What Jev returns instead of sentences

If your code asks a model for JSON so it can pull one value out of it, you are paying to generate text nobody will read. Jev answers exactly that situation: it takes your data and a list of typed questions, returns values with a probability distribution, and produces no prose on the way. That is a real architectural difference rather than a marketing angle. It is also younger than most branches in your repo, and on one axis it performs worse than the product page suggests.

The model is jev-1.13.0, and TypeSafe calls the class it belongs to a System One model. The name is Kahneman's: fast automatic judgment rather than slow reasoning. Every number below comes either from the vendor's documentation or from my own run over this blog, and I say which for each one. The price and the limits are from docs.typesafe.ai. The vendor states itself that the rate limits can change without notice.

One caveat up front, because it changes how you should read the rest. The product launched on 15 September 2026, in early access off a waitlist. It is two days old, single-vendor, ships no published calibration curve and names no customers. What follows describes how it works and what I measured, not a recommendation to put it on a payment path.

What a System One model is

A language model was trained to predict the next token. Whatever you want out of it arrives as text, and you recover the structure yourself. Jev was trained to do something else: return a decision and a probability. The docs call the training approach RLCD, reinforcement learning for calibrated decisions, and set it beside RLHF for chat models and RLVR for reasoning models. There is no generating layer in it. You cannot ask it to explain itself, because there is nothing to build the explanation from.

You define the answer space, with three question types. Choice picks one option from a set you supply and returns a probability for every option plus a single confidence figure. The cap is 255 options per question. Score rates the state on an ordered scale of 2 to 10 levels that you describe in words, and returns a position on that scale together with the distribution across levels. Noul answers a yes or no question with one number from 0 to 1, and carries no separate confidence, because that number already is the distribution.

The practical consequence: the model cannot return a value outside the list you gave it. That does not mean it cannot be wrong. It means you parse nothing, and you never handle the case where the model invented a sixteenth category out of fifteen allowed ones.

What one call looks like

A request has three fields: state with the content to evaluate, model, and a questions map whose keys you choose. Answers come back under those same keys. The key never reaches the model; it is purely a handle on your side of the wire. The response example below is abridged: it leaves out the mandatory usage field carrying the token counts.

{
  "state": { "ticket": "My shoes arrived in the wrong size." },
  "model": "jev-latest",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "returns":  "Exchanges, refunds, wrong or damaged items",
        "shipping": "Delivery status, delays, lost packages",
        "billing":  "Charges, invoices, payment problems"
      }
    },
    "urgent": { "type": "noul", "instructions": "Is this time-sensitive?" }
  }
}

{
  "model": "jev-1.13.0",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "returns",
      "confidence": 1.0,
      "probabilities": { "returns": 1.0, "shipping": 0.0, "billing": 0.0 }
    },
    "urgent": { "type": "noul", "noul": 0.18 }
  }
}

The most important property of this shape is invisible in a two-question example. Every question in one request sees the same state and is evaluated in parallel, in isolation from the others. One answer is not context for another. Adding a question barely moves the response time, because the state is ingested once and the questions run beside each other. In the vendor's own cookbook, thirteen questions about the GDPR article sent as one call came out 12.2 times cheaper and 10 times faster than thirteen separate calls, with the answers unchanged. The same result appears elsewhere in the docs as 11.5 times and 9.6 times, so one of those pages is stale.

That parallelism produces a way of working the docs call the most important concept in the whole guide: decompose the judgment into atoms. Instead of asking "rate this pitch", you ask separately about market size, feasibility and differentiation, and keep the weights in your code. Changing a priority is then changing a coefficient, not rewriting a prompt.

Where it actually differs from an LLM asked for JSON

Everyone who builds this asks the same first question: why bother, when I can turn on structured output and get the same enum. The answer that survives scrutiny is narrower than the vendor's pitch, and worth knowing precisely.

Structured output guarantees the shape of the answer, not its content and not its distribution. You get one value sampled from a distribution you cannot see. You can peek at it through logprobs, the raw probabilities of the next tokens. That is where the real boundary sits. OpenAI shows at most 20 alternatives per token position. The Anthropic API shows none at all. With a sixty-way taxonomy there is simply no route to per-option probabilities. What is left is asking the same model N times and counting how often each answer came up, which costs N calls and gives you an estimate carrying its own noise.

Then there is where that number comes from. An option's probability inside an LLM is a next-word probability, computed across a vocabulary of more than a hundred thousand tokens. You are the one who narrows it down to your categories afterwards. On the way it picks up priors that have nothing to do with your task. Zheng and co-authors measured one of them across twenty models and three benchmarks. Models prefer certain option IDs, plainly the letters A/B/C/D, regardless of what sits behind them. A label spanning several tokens gets a different number than a short one. And constraining the format during decoding shifts the distribution away from what the model actually believes.

The same applies to the most common workaround, asking the model to rate its own certainty from 0 to 1. That number is an utterance, not a measurement. A 32B-class model asked for confidence on SST-2 produced eight distinct values, and across four datasets between 45 and 93 percent of the mass sat in the five most common numbers. I have written before about why a self-reported score drifts away from real accuracy, and what that does to a threshold in your code.

Two things stay true regardless of what RLCD turns out to be worth. First, how many options you can handle at all. Second, that the distribution is computed across your options rather than across tokens. That is all that survived my checking.

What it found on my own blog

I wrote three scripts and ran them over this site's content. The most interesting one asks which other post an article should link to. This blog keeps a maintained internal link graph, and a build gate enforces it. Every post needs at least one inbound and one outbound prose link, in both languages, pointing at the same set of targets. The gate enforces the shape of the graph. It cannot say which target is the right one, and that part I had been doing by hand.

The state is the article text. The options are every other post in the same language. One call ranks the whole catalogue. That gives me a free test: does the model hit the links I picked by hand.

The second column matters more than the first. An article body still contains the text of every existing link. A sentence reading "as I wrote about silent failures" hands the model the answer. The second column is the same run with those anchors cut out, and only that is an honest measure of the leak. The top pick dropped by one case in 74, the top 5 by 23 out of 268, and mean confidence by 0.047.

The denominator is 268 rather than 288 for a simple reason. Ten posts carry more than five links while the ranking stopped at five, so the rest could not have been hit anyway.

What the result does not mean is more interesting. The two runs name the same best target in only 45 cases out of 74. Yet in 26 of those 29 disagreements both picks are still links the post already has. The leak stays inside the set of existing links. It shifts the pick from one of them to another.

There is one more number that bothers me. The Polish and English versions of the same post name the same best target in only 29 cases out of 37. My invariant requires both languages to point at the same set, so the model would not hold it on its own.

The practical finding is a disappointing one worth stating: for the 37 published posts this tool has nothing to suggest. The hand-made graph already holds the targets the model would pick. The value is at publish time, on a post that has no links yet.

The question that measured nothing

Alongside the ranking I added a Noul for every post: does this article contain a place where an outbound link would genuinely help the reader. The idea was to avoid forcing a link into a piece that stands on its own.

Zero of the 74 runs came in below 0.5. The mean was 0.871 on the full text and 0.891 on the blind one. The question separated nothing, because asked that way the answer for any engineering article is yes. The same happened to two Nouls in the post-quality script: "does the body keep the title's promise" and "is there checkable evidence" never dropped below 0.5 for any of the 74 files. Two Polish posts landed on exactly 0.50, and the script only flags below that. The report printed "nothing flagged" and that was true, and useless.

The model behaved correctly. The question was the defect. This is the cost the pitch does not mention: a typed answer guarantees the interface, not the sense. A badly posed question returns a number that looks exactly like a useful number, and nothing in the system turns red. I collected the failure modes that end in a 200 while the result is already wrong separately, because those are the ones that cost the most time to diagnose.

The bug that only surfaced because there were two questions

The second script takes queries from Search Console and asks which page should own each one. Beside the Choice about the page I put a Noul: does this site publish any page that answers this query at all. The first run returned something impossible. The query "bounded autonomy" routed to the post with exactly that title at probability 0.93, while coverage came back at 0.26.

The cause was mine. I had written the page list into the option descriptions of the Choice. Option descriptions are seen by that one question only. The single thing every question shares is the state. So the second question was judging coverage without a single page in front of it.

I moved the list into the state and the numbers moved with it. Mean coverage across the 36 shared queries went from 0.271 to 0.751. That particular query went from 0.26 to 0.95. Language routing was fixed as a side effect: on the same 36 queries the first run picked a Polish target 5 times and the second 14. Across the full 114-query second run it picked one 46 times.

It is worth noticing what caught that. Not a gate, not a test, not a type. A contradiction between two answers about the same thing. Had I asked one question, I would have received a plausible ranking and no reason to doubt it. The confidence figure would not have saved me either, because it is computed from the same distribution that was broken. It is the same mechanism by which a model's own verdict on its own work is not evidence that the work is right. A second, independent measurement is what saves you.

What Jev will not do

The vendor keeps a page about the model's weak spots and is honest on it. Read it before, not after.

• It does not count. Characters in a word, occurrences of a term, items in a long list. It recognizes the shape of an answer rather than tallying, and the error grows with the size of the thing being counted.
• It does not compare dates. Which came first, how far apart, whether one falls in a window. It gets worse with mixed formats and with quarters or settlement periods.
• You cannot recover a real quantity from a Score. A 1.3 on a three-level scale is a position on a scale you described, not a percentage and not an amount. Thresholding it is fine; reconstructing a number is not.
• Numeric representations underperform semantic ones. Asking about colours by name works better than asking about hex values.
• It gets lost in excess. Accuracy falls as the state swells with content unrelated to the decision. The limit is 64k tokens for the state and all questions together, plus a second, tighter one: 32k for the state and the single longest question. That budget exists to hold many questions, not a whole document.
• It is not hardened against hostile input. State is treated as data, not as something adversarial. Text written to steer a classification can steer it. That alone rules it out as your only security gate.
• It reads literally. Scoping words, negations and implied conditions are taken at face value. Their diagnostic is good: if you look at a wrong answer and find yourself explaining what you really meant, that explanation is the missing half of the question.

Input is text only. No images, no audio, no video. And the obvious thing that still needs saying: it will not write a reply, a summary, code or a rationale. The docs say plainly that you can force it to by chaining Choices, that the output will be poor, and that it will be very slow.

What it loses to the boring options

This is where it gets uncomfortable for the pitch. The central promise is calibrated probabilities without your labels. The trouble is that calibration stops being an advantage the moment you have a few hundred labelled examples. Then temperature scaling is enough. That is one number fitted to your data, twenty lines of code, and on most datasets it works surprisingly well. No labels at all? Eight examples per class, an afternoon's work, was enough for SetFit to match a few-shot model many times its size on its authors' benchmark, with no prompts at all. And with a stable list of categories you can train your own small classifier. A model in the ModernBERT class is 149 million parameters. You do not train it from scratch; you download the released checkpoint and fine-tune it on your own categories, which runs in minutes on one GPU. After that it charges you nothing per call.

For ranking the case is sharper still. A Score question asked separately of each candidate is one model pass per candidate. That is exactly how a reranker works, except here you pay per token. The alternative computes vector representations once, offline, and then only searches a store. If your question is "sort this corpus by similarity", a typed question is a wrapper around something that has existed for years and costs less.

Then there is the evidence. The vendor publishes an evaluation dashboard whose reference labels are an average of the answers of two large generative models. For a product sold as an alternative to LLM judgment that is circular: accuracy on that board means agreement with a two-LLM consensus, not correctness. It is worth remembering that an LLM judge carries its own measured biases, starting with a systematic preference for whichever answer it compares first, so the reference label is not neutral either. On that same data Jev lands mid-pack: 67.8 percent agreement averaged over the four workflows, level with sonnet 5 to within a tenth of a point. Above four other models, with haiku 4.5 at 53.6, and below opus 5 at 73.1 and the board's best at 74.1. Per workflow it runs from 61.7 to 76.0 percent. The other axis of that same board shows what is actually on offer: $0.0004 and 0.4 seconds per case, eight times cheaper and twenty-five times faster than the next cheapest and next fastest configurations. The pitch is price-performance, not intelligence, and should be read that way. The zero-hallucination line is likewise a claim about types rather than about truth: a Choice cannot invent an option outside your list, but it can pick the wrong one.

Nor did I find a single chart in the documentation showing whether a stated 0.9 really does mean nine hits in ten. No standard calibration measure. No paper on RLCD. That is this product's central promise, and simultaneously the only one you cannot check today except on your own data.

When it makes sense

The rule I am keeping after this run is simple. Reach for a typed question when the answer set is closed and larger than twenty, when you need the distribution across those answers rather than one value, when you have no labelled data and it is not worth producing any, and when code consumes the result rather than a person. Four conditions together, not one of four.

Three categories and a thousand labelled examples? Your own small classifier wins. An open question whose answers you cannot list? The LLM wins. Sorting a large set by similarity? Vectors computed once win. What is left is a narrow but real middle: many closed judgments about the same data, a threshold in your code, and an escalation path for everything below it. That is exactly the construction I described as four escalation levels, from flagging a case for later to stopping the agent outright, with a probability on the input side instead of a hard rule.

This experiment cost 372 calls, 2.82 million input tokens and $0.1185, and I threw one of the runs away as buggy. The price is low enough to stop being an argument, which is exactly why the attention belongs on the question that actually decides it: can your judgment be closed into a list of options, and do you have somewhere to send the cases the model is unsure about. Without the second, a confidence threshold is decoration.