Jev
probabilistic evaluations for software

A person can write a ticket without making Jev their conversation partner. The software extracts the relevant context, calls Jev and reads the values. It can then display a suggestion to that person, trigger an authorized process or ask an LLM to draft a reply. This separation explains the product’s appeal: some steps need a category rather than a conversation.

To triage a GitHub issue, you can supply its contents and a list of categories. With Choice, Jev returns the selected option and the associated probabilities. Your application uses the result to suggest a destination. This is an example of using the API described by TypeSafe.

The call contains shared context, called the state, and 1 or more questions. The program specifies the expected form: a choice from a list, a rating on a scale or the probability of a yes answer. It can process the result without extracting a category from a paragraph.

Enlarge ↗

These 4 original infographics take inspiration from Daniel Moka’s Jev Engineering 101, published September 22, 2026. They show one way to organize an application’s work, without describing Jev’s internal components. Images generated with OpenAI and reviewed; example values are fictional.

From the API result to an action

• The application supplies the context. A ticket mentioning a declined payment does not establish the transaction’s actual status. Before acting on it, code must retrieve the relevant records from the service concerned. See the State documentation.
• Questions within a request share the same state. If a question needs another question’s answer, the program prepares a subsequent request. The diagram represents 1 API request without assuming a single internal model pass.
• Format and correctness require separate checks. An answer the SDK can handle may still be wrong. Set the timeout, permissions and fallback before executing an action; choose thresholds based on acceptable errors. See how to keep permissions in code.

Enlarge ↗

An agent can use this API to choose a destination or assess a document, then hand the writing or investigation to another model. Using Jev for a specific step does not demonstrate that it can replace the model driving Claude Code.

On small screens, diagrams can scroll horizontally. Figures can be enlarged to read their labels.

• TypeSafe says it was founded in 2024 and is based in San Francisco.
• The team page lists Diogo Almeida as CEO, Erik Gafni as CTO and Sasha Sheng as COO.

These details describe the company; they do not validate its model’s performance.

Diogo Almeida is a coauthor of InstructGPT. The viral label “co-inventor of ChatGPT” reduces collective work to personal credit; the author list does not establish “lead author of InstructGPT” either. The relevant bibliographic fact is his contribution to this work on learning from human feedback.

• TypeSafe borrows “System One” from Daniel Kahneman’s distinction between fast and deliberate thinking.
• Jev refers to William Stanley Jevons and the idea that lower costs can increase usage.

These references explain the product’s name; they do not describe its neural layers. TypeSafe’s explanation of the names.

Enlarge ↗

The documentation describes Choice, Score and Noul.

• Choice requires a mutually exclusive selection. Separate binary questions may suit multiple compatible labels.
• Also consider cases that fit no category: an option missing from the list cannot be selected.

• In the ThursdAI interview, a TypeSafe representative explains how to describe Score levels at 19:51, then why non-exclusive tags need separate questions at 23:00.
• In AICodeKing’s video, the extraction example at 03:55 selects from values supplied to the model.

An option’s probability and the confidence field answer 2 different questions.

• The first concerns that option.
• The second summarizes how concentrated the probabilities are across the options, according to TypeSafe’s definition. It should not be read directly as the probability of being correct.

To assess Choice calibration, take the cases where the selected option receives a probability close to 80%. Compare that number with the proportion of answers that are correct. This is an illustrative example: no Jev result is measured here. With Noul, instead check whether events to which Jev assigns a yes probability close to 80% occur in roughly 80% of cases.

These comparisons require reference answers, enough examples and a test set that was not used for tuning. Guo et al., 2017 discusses calibration methods. A good average on 1 dataset guarantees neither each decision nor results in another domain.

Also measure coverage: what share of cases does the application handle without review? A system that leaves almost everything to a person can make few errors while automating very little. SelectiveNet studies this tradeoff. Ovadia et al. examines what happens to uncertainty measures when the data changes. These studies are not about Jev.

• A low ECE, the average calibration gap across bins, can hide a poorly calibrated rare class or an error among automated cases. Publish sample counts per bin, a reliability diagram and errors among accepted cases.
• The Brier score measures the squared difference between probabilities and outcomes; the convention used for multiple classes must be specified.
• For Score, an average severity remains a rating on the chosen scale.

TypeSafe also states in its launch post that higher confidence accompanies higher accuracy. This relationship can be tested: the ASSAY-001 responses allow accuracy and coverage to be measured after filtering by confidence. That does not turn this field into a probability of success; the measurement chapter details the 2 calculations.

The “Measured probabilities and confidence thresholds” chapter distinguishes 2 measures: calibration of the selected option’s probabilities and accuracy of answers retained by a confidence threshold. The ASSAY-001 data allows both to be recalculated; Agent Journal reports the second on a different task.

Videos sometimes lose this distinction: Fireship at 03:15 equates confidence with the frequency of correct answers. The video chapter corrects this explanation and retains the original passage.

An autoregressive generative LLM builds its response by predicting the next tokens from the context and what it has already produced. This is the process described in Hugging Face’s generation documentation. The output can be free text or JSON, potentially constrained by a schema.

TypeSafe announces a new architecture and parallel sampling in its launch post. The API receives a state and typed questions, then returns their results. According to the vendor, outputs are computed in parallel rather than generated token by token. The diagram compares this claimed behavior with an autoregressive LLM; it does not describe Jev’s internal components.

The stated distinction concerns intended use: Jev provides typed answers and probabilities that code uses in its decision logic. An LLM can also be called by software and produce structured output; in a conversation, a person reads its text. The recipient of the response and the way the model produces it are therefore separate considerations.

TypeSafe calls its training method RLCD, for Reinforcement Learning for Calibrated Decisions. Its technical introduction outlines the objective. “System One” names the model family; it does not describe a published architecture.

The research for this report found no publication with enough detail on the architecture, data and training objective to reproduce Jev. Other documents may exist beyond this scope. In the ThursdAI interview at 05:59, the TypeSafe representative discusses RLCD without disclosing the architecture. Describing Jev as a model “without internal reasoning” would go beyond the available information.

The abstracts of DeepSeek-R1 and Tülu 3 supplement this selection without counting as acquired PDFs. The next chapter explains their relation to Jev’s training.

A conventional loss function can already encourage correct probabilities. For an event with true probability q, the expected logarithmic score q·log(p) + (1−q)·log(1−p) is maximized at p = q. This theoretical result does not guarantee that a model trained on finite data will be calibrated in a new domain. The formula illustrates a principle; it is not attributed to RLCD.

The paper Rewarding Doubt, version 6 studies learning confidence expressed in text, using a logarithmic reward and PPO.

• Its 17-page PDF was acquired; the mechanism, results, transfer and proof were examined.
• The first version dates from March 4, 2025; version 6 from February 28, 2026.
• The theorem describes an optimum of the objective, while empirical results vary across datasets.

No published link to Jev has been established.

The abstracts and version histories of DeepSeek-R1 and Tülu 3 provide additional post-training context. Their experiments were not audited for this report. They provide no evidence about Jev’s parameter count, backbone or training data.

In the TechCrunch interview from September 18, Diogo Almeida says Jev is trained exclusively on synthetic data. This remains a reported claim: neither the corpus nor its generation procedure was audited here. Synthetic data can also derive from other models or existing sources; the label does not demonstrate that benchmarks are free from contamination.

The HTTP reference documents POST https://api.typesafe.ai/v1/systemone, with a Bearer token and a JSON body containing model, state and questions. GET /v1/models lists models: /v1/systemone is therefore not the entire service’s only route.

• The shared state can be a string, object or JSON array. It can contain numbers and booleans.
• “Text input” here means that no native image, audio or video input is documented for the model; it does not require converting every JSON value into text.
• Question identifiers are used to retrieve answers and do not replace explicit instructions.

{
  "model": "jev-1.13.0",
  "state": {
    "ticket": "The payment failed and I can no longer access my account.",
    "language": "en"
  },
  "questions": {
    "destination": {
      "type": "choice",
      "instructions": "Which category best describes the main subject of the ticket? Evaluate the categories using the supplied text.",
      "criteria": {
        "facturation": "Declined payment, invoice or refund.",
        "acces_compte": "Unable to sign in or account access blocked.",
        "incident_technique": "Outage or technical error unrelated to payment or account access.",
        "autre": "None of the preceding topics."
      }
    },
    "payment_failed": {
      "type": "noul",
      "instructions": "Does the message report a declined or failed payment?"
    },
    "impact": {
      "type": "score",
      "instructions": "Assess the impact described, based only on the text.",
      "criteria": [
        "No impact described.",
        "Inconvenience with a workaround.",
        "Feature unavailable to the person.",
        "Several users blocked."
      ]
    }
  }
}

{
  "model": "jev-1.13.0",
  "answers": {
    "destination": {
      "type": "choice",
      "choice": "facturation",
      "probabilities": {
        "facturation": 0.88,
        "acces_compte": 0.08,
        "incident_technique": 0.03,
        "autre": 0.01
      },
      "confidence": 0.81
    },
    "payment_failed": {
      "type": "noul",
      "noul": 0.97
    },
    "impact": {
      "type": "score",
      "score": 1.7,
      "legend": {
        "0": "No impact described.",
        "1": "Inconvenience with a workaround.",
        "2": "Feature unavailable to the person.",
        "3": "Several users blocked."
      },
      "probabilities": {
        "0": 0.0,
        "1": 0.3,
        "2": 0.7,
        "3": 0.0
      },
      "confidence": 0.6
    }
  },
  "usage": {
    "input_tokens": 500,
    "output_tokens": 80
  }
}

The envelope contains model, answers and usage.

• An output_tokens counter can exist even when its price is zero.
• Recording the model actually returned and the actual usage is more precise than recording only the requested alias.
• The Python client reviewed accepts missing counters, represented by None; the JavaScript v0.6.0 types declare them as numbers. Log missing values instead of counting them as 0.

Structures and error codes are listed in the native reference.

"""Documentation example, not executed for this report.
Requires typesafe-sdk and TYPESAFE_API_KEY in the environment.
Example limited to the Choice question from request.json.
"""
from typesafe_sdk import Choice, TypeSafeClient

with TypeSafeClient() as client:
    response = client.system_one(
        model="jev-1.13.0",
        state={
            "ticket": "The payment failed and I can no longer access my account.",
            "language": "en",
        },
        questions={
            "destination": Choice(
                instructions="Which category best describes the main subject of the ticket? Evaluate the categories using the supplied text.",
                criteria={
                    "facturation": "Declined payment, invoice or refund.",
                    "acces_compte": "Unable to sign in or account access blocked.",
                    "incident_technique": "Outage or technical error unrelated to payment or account access.",
                    "autre": "None of the preceding topics.",
                },
            ),
        },
    )
    answer = response.choices["destination"]
    # No automatic action: thresholds need their own evaluation.
    print(response.model, answer.choice, answer.probabilities, answer.confidence)

// Documentation example, not executed for this report.
// Requires @typesafe-ai/sdk and TYPESAFE_API_KEY on the server.
// Example limited to the Choice question from request.json.
import { choice, TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient();
const response = await client.systemOne({
  model: "jev-1.13.0",
  state: {
    ticket: "The payment failed and I can no longer access my account.",
    language: "en",
  },
  questions: {
    destination: choice("Which category best describes the main subject of the ticket? Evaluate the categories using the supplied text.", {
      facturation: "Declined payment, invoice or refund.",
      acces_compte: "Unable to sign in or account access blocked.",
      incident_technique: "Outage or technical error unrelated to payment or account access.",
      autre: "None of the preceding topics.",
    }),
  },
});
const answer = response.answers.destination;
// Choice is already inferred here; this guard only checks the received type.
if (answer.type === "choice") {
  console.log(response.model, answer.choice, answer.probabilities, answer.confidence);
}

• In the JavaScript v0.6.0 client inspected, the timeout is 10 seconds per attempt and 2 retries may follow the initial call for certain errors. It exposes no total retry budget.
• The Python SDK distinguishes timeouts per HTTP operation from a total retry budget.

These settings must accompany latency measurements. JavaScript retries, JavaScript options, Python retry policy.

Consider this ticket: “The payment failed and I can no longer access my account.” Asking “Which team?” imposes an unwritten priority. An exhaustive list does not resolve the ambiguity. Define the business rule first: does 1 team own the incident, are multiple labels allowed, or should someone review the case?

Breaking the task down can help: detect the topic, missing information and impact separately, then apply a rule. It also creates more opportunities for error.

• 2 answers can conflict; the code needs to know how to handle that.
• When a question depends on a previous answer, use successive calls and count their total time.

Extraction with Choice remains limited to the candidate values. To select a date, the possible dates must already have been found or enumerated. Choosing among 3 dates does not prove that Jev can freely extract any value from a document. Check the quality of the candidate-generation step separately.

Enlarge ↗

Classification and categories supplied at call time already existed. A classifier maps an input to categories. With natural language inference, or NLI, a category can be turned into a proposition, then the model can be asked whether the text supports it. Categories therefore do not always need to be fixed in a head trained for a single task. Yin et al.’s 2019 work and the Hugging Face zero-shot pipeline provide concrete precedents to compare with Jev.

The Jev API combines Choice, Score and Noul questions over a shared state. This makes them easier to use in code without retraining a model for each set of options. The service can simplify application development even if the techniques it uses already existed. Ease of use alone does not prove a new architecture or better calibration.

TypeSafe’s launch post states limitations itself:

• short input favors its demonstration;
• the large multipliers would be at the upper end of expected gains;
• the reference answers for the workflows are model responses.

The adapter asks LLMs for probability distributions, which cost more to produce than a decision alone. Comparing the same outputs is useful for evaluating what the API returns; an application that needs only a category should also test that narrower requirement.

Claude’s structured outputs already constrain decoding according to supported schemas. The documentation details the limitations, including refusals, truncation and a casing exception for enum and const values. Distinguish an instruction to “respond in JSON,” a mode that guarantees JSON syntax and schema-constrained decoding. Portraying all LLMs as unable to follow a schema would distort the comparison. Format compliance does not guarantee a correct decision for any competitor.

Pixel is a project by Livio Gamassia. Like me, Livio is an active member of the DevWithAI community. The sources inspected are described in the chapter on community projects. Its routing, graph and explicit scores already support decisions within their scope. Embeddings add semantic search; they do not turn every result into exact evidence. No comparison run here establishes that Pixel and Jev handle all natural-language criteria with equal quality.

On Hacker News, the classifier description is accepted. On September 15, petesergeant describes Jev as a zero-shot classifier. CompleteSkeptic replies “exactly right!”. This account identifies itself as TypeSafe’s CEO in the same thread. The agreement concerns a functional description; it neither publishes the architecture nor validates the claimed performance.

The discussion also distinguishes a valid type from a correct answer. On publishing the architecture, the same account says the team has discussed a paper while keeping the details confidential. The comment provides neither a publication commitment nor a date.

An idea’s precedence and copying someone’s work are separate questions. Nandakishor M, the author of Laya, claims earlier work on non-autoregressive decisions. Preprint records from 2025 exist; his post also acknowledges that his first system did not handle new questions at runtime. These facts support a discussion of the claimed novelty. They do not demonstrate that TypeSafe copied his code or method. The abstracts were consulted, without auditing the PDFs.

The video analyses also compare favorable views with counterexamples: Theo’s checkers game, errors described by Gary Explains and Erwan’s comment triage. Fireship’s reservations about format are useful; his explanation of confidence needs correcting. None of these passages establishes fraud on its own.

The 150 ms figure appeared in the LinkedIn post that prompted this research. I found no primary source establishing it as a general service latency. The range in the launch post and dated trial measurements are more useful when planning a comparison.

The TypeSafe evaluations give equal weight to 4 workflows: security incidents, agent traces, invoices and support. Their reference answers come from GPT-6 Astra and Claude Fable 5.1 with high reasoning; the compared models use their default settings. The column named accuracy therefore measures agreement with these references, rather than with independent human annotations.

Rounded values read from the vendor's charts on September 21.

• Jev costs less and responds faster in this aggregate; sol and Opus 5 have higher agreement.
• On invoices, Jev reaches 61.8%, compared with 79.1% for sol.
• Jev's 4 agreement rates are 61.7%, 71.6%, 61.8% and 76.0%. Their mean is 67.775%, consistent with the displayed 67.8%.

The launch multipliers cannot be recalculated by arbitrarily dividing 2 rounded cells in this table.

Enlarge ↗

The basic calculation is straightforward: billed input tokens × price per token. At the direct price cited in this report, 1,000 tokens cost $0.000042. 1 million calls of that size therefore cost $42 in input charges. This calculation assumes neither that every call is correct nor that 1 call is enough to finish a task.

Processing multiple questions and reusing context changes the billed volumes.

• Record usage counters when available. If they are missing, log “unknown”, never 0 tokens: the Python and JavaScript clients do not declare these fields in the same way.
• An estimate based on the ticket does not replace the bill.
• Quotas, errors and retries also count toward the workflow.

• Cost per correct decision divides the full cost by the number of correct decisions, specifying whether those corrected by a person are included.
• For a repeated task, also publish the cost per completed case and measured review time in minutes.

A solution that lowers model costs but requires more corrections can cost more overall.

Enlarge ↗

In a sequential workflow, stage durations add up. Independent questions can run in parallel, but the gain depends on quotas and the response-time distribution. Measure the time from receiving a case to producing a suggestion the application can use, then calculate p50 and p95 for that full duration.

The anisselbd phishing benchmark covers 2,000 emails with synthetic bodies. The first protocol asks Jev 9 questions and asks Haiku 4.5 for a direct verdict. Jev achieves 62.6% correct decisions, compared with 81.3%. These 2 configurations do not isolate the model's contribution.

A second protocol asks both models the same 5 questions, then trains a regression on their signals. On the 1,000 held-out test cases: 95.0% with Jev, 93.2% with Haiku, compared with 91.8% for the rule-based baseline. The Jev/Haiku gap is not significant at the conventional 5% level (McNemar p = 0.063). The authors also report Jev's cost and latency advantage with this decomposition.

Question wording and the way answers are combined are part of the evaluated system. Moving from a direct verdict to 5 signals changes the protocol; these results remain specific to this corpus.

nDCG@10 measures ranking quality for the top 10 documents. A reranker reorders documents that have already been retrieved. The English results do not establish a winner; an interval containing 0 does not establish equivalence either. These tests alone do not measure the quality of an answer written from the documents.

These observations do not support a comparison between regions: inputs, questions and protocols differ. They provide more precise reference points than a general latency of 150 ms.

The ASSAY-001 report evaluates Jev on Banking77 and CLINC150, 2 public intent-classification corpora. For this report, the published Banking77 and CLINC150 responses were retrieved and their hashes checked. The recalculation covers probabilities, decisions and the confidence field. It uses the published campaign without making new model calls.

ECE compares the probability of the returned option with the rate of correct decisions, grouped into bins.

• The scoring script uses 10 bins: [0, 0.1], then (0.1, 0.2], through (0.9, 1]. A value of 0.9 therefore belongs to the second-to-last bin.
• Each absolute gap is weighted by the proportion of responses in its bin.

ECE uses the probability of the selected option, even in the 3 cases where it differs from the displayed maximum; it does not use confidence here.

The protocol sets ECE ≤ 0.05 as the criterion for this study. CLINC150 meets it; Banking77 does not. This threshold is not a general standard. ECE also depends on the bin boundaries: using left-closed bins, [0, 0.1) through [0.9, 1], CLINC150's ECE changes from 0.0204 to 0.0209 on the same amended response set. Neither binning scheme changes the verdict at the 0.05 threshold. These results do not isolate the effect of class count; the corpus with more options has the lower ECE here.

The second amendment, described in the report, raises the tolerance for probability sums after measurement, from 0.001 to 0.02. It restores 145 Banking77 responses and 368 CLINC150 responses whose sums are 0.99 at the published precision. The Vercel documentation for the TypeSafe AI provider states that probabilities are rounded to 2 decimal places and may not sum to exactly 1. The data are consistent with this precision: 101 distinct hundredth values, excluding floating-point representation noise. This count alone does not reveal the internal rounding process. A sum of 0.99 is therefore not enough to establish a type error.

Restoring these responses lowers accuracy on both corpora. It increases Banking77's ECE and slightly reduces CLINC150's. Changing the tolerance also changes the evaluated population; both sets of results are needed to understand its effect.

Another check finds 3 choices whose probabilities are 1 percentage point below the displayed maximum: 0.46 versus 0.47 and 0.47 versus 0.48 in Banking77; 0.48 versus 0.49 in CLINC150. The API reference describes choice as the most likely option. Monotonic rounding applied in the same way to each option cannot, by itself, reverse their order. These 3 discrepancies remain unexplained; the internal values before display are not published.

The ASSAY files also let us check what a confidence ≥ 0.9 filter retains.

• In the following table, coverage is the share of usable responses retained by this filter.
• Accuracy is calculated only among those retained responses, using the amended tolerance for probability sums.

The 72.2% figure repeated in roundup articles comes from Agent Journal. The author describes 300 synthetic cases; 126 have confidence of at least 0.9, and 72.2% of their responses are correct. Raw responses were not found in the links or the targeted search. This remains an author-reported result, not recalculated in this report.

Agent Journal reports 42% coverage. The task requires combining weak signals in synthetic texts; it differs from Banking77 and CLINC150 intent classification. The author specifies the jev-latest alias, without a resolved version that can be verified in raw responses. These results can coexist: protocols, data and possibly versions differ. They do not isolate the cause of the gap.

All 3 experiments retain errors after selection by confidence. The accuracy and coverage of this selection are not ECE measures. Since confidence is not defined in the API as the probability of being correct, comparing the 0.9 threshold with 72.2%, 92.61% or 96.11% does not measure the calibration of option probabilities.

• For Agent Journal, the absence of raw distributions and associated labels prevents this calibration recalculation.
• For ASSAY, these data are available and ECE is reported above.

On October 8, Astronomer observed that each decision model has its own confidence scale: on the intermittent-heavy set, medians of 0.94 for Jev 1.13, 0.84 for decider-0.8b and 0.45 for GLiNER2.5-Decide. Jev’s confidence moved by up to 0.14 between 2 repeated calls. A threshold chosen for Jev therefore does not transfer to another compatible model.

• The tutorial Building a Harness with Jev, published by Sydney Runkle and Hunter Lovell on September 17, explains an integration.
• The figures repeated in ExplainX come from another source: the jev-as-a-judge repository.

The tutorial, the article reporting the results and the benchmark serve different purposes.

The test uses 5 fixed traces, each repeated 100 times. 1 annotator supplies the reference labels: 4 successes and 1 failure. On the Dublin trace, asking for clarification seems reasonable but does not satisfy the rubric, which expects a search. The choice of rubric is therefore part of the result.

These rates match the published per-case summaries. Individual decisions are absent from the archive reviewed, so they were not recounted. Any presentation of “100%” must keep the 5 traces and their repetitions visible.

The summaries in benchmark.json also give the mean of the quality variances calculated within each of the 5 traces. The table recalculates ratios from those summaries; individual judgments are unavailable for recalculating the variances themselves.

These ratios describe the stability of a continuous score on the same traces. They do not multiply either the rate of correct decisions or the accuracy of the probabilities.

• The Jev judge combines 3 Noul answers for quality and makes another call for the binary verdict.
• The judges' rubrics are not worded identically: this experiment does not isolate an architectural effect.

The metadata for this run are dated September 18.

• They name the 3 LLMs and report a modified working tree at launch. The archived code therefore does not guarantee access to every exact file used in this run.
• The hosted Jev model version is not specified; it cannot be inferred from the LangChain client version.

The cost collection script queries LangSmith. The scope of the means and totals is not clearly limited to the 500 verdicts. Luna also appears close to Jev in cost in their table. The comparison therefore does not justify claiming that all LLMs cost several orders of magnitude more.

To test Jev as a judge, vary the tasks, tools, partial failures and rubrics, then have new traces annotated. Repeating a trace helps measure stability; adding different traces is necessary to test coverage of situations.

Theo at 12:40 criticizes choosing between implementations using a judge that does not explore the repository. This objection is a reminder to test a specific criterion with the context it needs. It does not establish that a model without written reasoning can never serve as a judge. The video analysis distinguishes these 2 claims.

The JevGrep protocol added September 22 separates local performance with a mock provider from a semantic-retrieval pilot. It defines 30 questions across 3 small synthetic repositories, including 6 negatives. A published protocol does not establish a held-out final result or better agent task success.

Capital & Compute brings several results together, but their tasks, comparators and definitions of cost differ. The result on 18,514 emails is reported by Arize, which links to bitnovus. Counting it once for every article would artificially multiply the confirmations.

These trials are more useful when their exact scope is retained.

• The product test resembles Sentry matching: retrieve candidates, then judge a pair. The annotation task and the cost of an error still differ.
• Code-review scores do not predict the quality of a PR review involving multiple files and business rules.

• Capital & Compute also reports its own trial on 483 Search Console queries: Jev is called, but the other models' costs are calculated, and only 18 examples receive human review. This result therefore does not compare full runs of all competitors.
• The Every figures repeated in this chain are not included here as verified results: the article was located, but its body and linked material could not be read.

The CounterProof analysis of consensus labels finds 8 disagreements among 19 cases where both its judges respond. These cases come from a selection of 20 vendor examples, not a random sample of the full evaluation. This check invites scrutiny of the labels; it neither replaces them with human ground truth nor reruns the full benchmark.

The MindStudio article on Jev and classifiers reports, among other figures, 93.2% for a 22-million-parameter encoder. The article body reviewed provides no link to the protocol or repository for that experiment. This figure is therefore not added to the table of results verified at their original source. The article also reports the opposite result on Yelp; it does not justify a general conclusion that a trained classifier would always win.

Jev opened the category on September 15. Within 4 weeks, Cloudflare, Strands Labs, Perplexity and OpenAI released models that answer the same question types: yes or no, a choice among options, a score on a scale. They return probabilities, not text to parse.

Two request formats coexist. System One, Jev’s interface, sends a state and named questions of type noul, choice or score. Clef, Strands Decider and Ollama 0.35 adopt it. OpenAI uses /v1/decisions: an input, an array of predicate, choice or score questions, up to 400 questions, and a possible refusal answer for each question. LiteLLM has served both routes since October 14.

Cloudflare presents Clef as the leader on the Jev Decision Index; its own table leaves Jev ahead on When2Call, BRIGHT and agent traces. The launch post also claims the Clef models beat other decision models on latency, except Laya. Yet its table shows a 51.4 ms median for Kev-9B and 84.4 ms for DiffusionGemma Jev, against 209.3 ms for Clef. On October 9, Cloudflare reported a 152 ms median for Clef on about 800 tokens, from a serving-layer change, without new weights. The same post says the project was decided on a Friday evening, trained over the weekend and launched on Thursday.

The evaluations by Perplexity (pplx-decider) and Strands Labs (Strands Decider) are also vendor measurements. The Strands Decider 2B v1-2610 card reports 180 correct answers on JevBench’s 231 public tasks, with a 4,096-token window.

3 evaluations published by third parties. None compares every model on the same task, and none was reproduced for this report.

• At Astronomer, at the 0.8 threshold, Jev 1.13 stops 147 tasks including 15 recoverable ones, Clef 112 including 5, Clef-flash 133 including 11 and Strands Decider 63 including 2. Fewer errors come with fewer correct stops: 128, 105, 118 and 61 out of 222.
• Rewording category descriptions changes 5% to 14% of each model’s decisions; Jev’s confidence moves by up to 0.14 between 2 repeated calls.
• Rules placed before the model produce 25 stops, all correct.
• On the JevBench Index, Jev 1.13 and Clef are 0.4 points apart: one table is not enough to separate them.

The lab ran on an Apple M5 Max with 128 GiB, macOS 26.5.1, Python 3.12.13, Laya-MLX 0.2.0 and MLX 0.32.2, using the GPU in FP16. Model: aac6fef/laya-multilingual-mlx, revision f2b4faf51023039425946074e2cf1361d2db11d5. Compilation and prompt caching were disabled. All 11 manifest files were verified, including roughly 614 MiB of weights.

The data contains 72 synthetic development examples. The 3 tasks below use 20 each, for 60 predictions. Questions, labels and keywords were frozen before inference. The evaluator’s ordinal-score validation was strengthened before producing the reports; its final version is archived. This is neither a held-out real-world evaluation nor independent human annotation. Settings were not changed to erase the observed errors.

Timing protocol: 10 warm-up calls, then 100 sequential calls per configuration using the same short text. Timings include engine validation but exclude loading, HTTP and the browser. Measured loading: 5.45 s. Peak MLX allocation: roughly 822 MiB, not total process memory. This does not establish throughput for long documents. These timings cannot be directly compared with remote Jev calls or the M3 Max data in the local-model chapter.

The data and measurements for this revision contain synthetic examples, questions, predictions, lexical baselines, timing samples and evidence hashes. They contain no customer data or private conversations.

The question was narrow: which technical area owns the requested change? Options were tooling, backend, frontend, fullstack, multiple, other and unknown. The pilot does not assess duplicates, ticket completeness or request type. No GitHub label was changed and no automation was activated.

Precision divides matching suggestions by emitted suggestions. Coverage divides emitted suggestions by all 60 tickets, including oversized ones. 8 out of 60 is therefore not Laya’s precision. It leaves 49 tickets without a suggestion, compared with 43 for the current rules. These cases need another handling path; they are not all incorrect classifications.

The lexical variant reuses the same expressions and priority order on title and body. It is not the deployed workflow. It gives the rules more text, as Laya receives: its result must remain separate from the title-only baseline. Its 40 matching suggestions warrant human review, including examination of its 11 errors; no time saving was measured.

The model produced 52 valid structured outputs and 8 length rejections, with no other technical error observed in the test batch. Questions and options leave 896 tokens for serialized input within the 1,024-token total limit. The 8 overflows account for 13.3% of the test. All 8 matching Laya suggestions concern tooling; no suggestion for another domain matches its annotation in this sample.

These timings are not interchangeable with the previous chapter’s 8.55 ms synthetic filter: inputs, question and measurement scope differ. Other Mac activity was not controlled. No remote inference API or cloud fallback was used; each compute process verified an EPERM network denial. The GitHub export required network access. Energy use and human correction effort were not measured.

A suggestion means same for Sentry/reviews, or a code, configuration or infrastructure classification for CI. Precision divides matching suggestions by proposed suggestions. Coverage divides suggestions by all test cases, including oversized inputs: 79.2% for Sentry, 25.0% for reviews and 55.6% for CI. A comparator proposing nothing has undefined precision, not 0% precision.

Sentry: no additional matching suggestion beyond known links. The lexical zero applies after links and identifiers are masked. Of the 48 pairs, 16 already have an explicit Sentry association in GitHub provenance. Both matching Laya suggestions are in that set. A documentary link alone does not independently prove a common cause.

CI: 1 more matching suggestion in total, but 13 more wrong ones. Laya proposes 6 matching and 14 nonmatching classifications, against 5 and 1 for lexical signatures. Its 20 suggestions are all code: 6 match, 8 concern cases annotated configuration and 6 concern unknown cases. It proposes no configuration or infrastructure classification on this test.

For reviews, Laya proposes 1 matching link and 8 unsupported ones. A slightly higher macro-F1 than the comparator does not qualify the grouping system. The task is to identify the same alleged defect; it does not establish that the defect exists.

All 15 overflows among the 120 test cases stay in the denominators. Decisions of different, unknown and abstentions are not wrong matches. They remain separate from context errors. No other inference error type is reported.

Context refusals are counted separately from quantiles. Overall Mac load was not controlled; collection, annotation and human correction are excluded. These timings do not measure human time saved and are not interchangeable with measurements from other tasks.

Videos suggest tests and also show failures. The same creator can appreciate Jev's speed while criticizing its decisions. Opinions are attributed below; reported results remain those of their authors. None of these trials was reproduced for this report.

The September 17 ThursdAI episode description identifies Allie Laabs as Founding DevRel at TypeSafe. The links below use timestamps from the YouTube excerpt about Jev, which differ from those of the full episode.

The short video Jev Doom Demo SoftBed is a commented repost, with no independent experiment identified. In the original demo, TypeSafe says Jev receives a textual game state, not images. Neither this repost nor an edited video measures general gameplay ability. The timestamped links in this report were checked against the transcripts, without an audiovisual audit of the displayed counters.

Jev is a hosted service. Here, you install the Python client library that calls its API. Jev’s weights are not downloaded to your machine. The program uses the result: no chatbot conversation is needed.

1. Get a key and check Python.

• Open the TypeSafe console and create an API key if your account has access to the service.
• You need Python 3.10 or later and an Internet connection.
• This walkthrough uses TypeSafe’s direct API; a key from another provider will not work.
• Check your account’s access and billing in the console.

2. Install the client in a dedicated directory. Run the following commands in a macOS or Linux terminal. Check that the displayed Python version is at least 3.10 before continuing. The SDK is pinned to 0.7.0 so you can reproduce the example with the same client version.

mkdir jev-premier-appel
cd jev-premier-appel
python3 --version
python3 -m venv .venv
source .venv/bin/activate
python -m pip install "typesafe-sdk==0.7.0"

3. Create the script. In that directory, save the following block as premier_appel.py. The key is read from TYPESAFE_API_KEY if that variable exists; otherwise, the program asks for it in the terminal with hidden input. It appears neither in the file nor in the assistant prompt.

"""A fictional ticket, 1 API attempt and a displayed predicted category.

Requires Python 3.10+ and typesafe-sdk==0.7.0.
Running this file sends a request to TypeSafe, billable according to the account.
"""
import getpass
import json
import os
from time import perf_counter

from typesafe_sdk import (
    Choice, RetryPolicy, TypeSafeAPIConnectionError, TypeSafeAPIError, TypeSafeClient,
)


def main():
    api_key = os.environ.get("TYPESAFE_API_KEY", "").strip()
    if not api_key:
        api_key = getpass.getpass("TypeSafe API key (hidden input): ").strip()
    if not api_key:
        raise SystemExit("No key provided; no request sent.")

    try:
        with TypeSafeClient(
            api_key=api_key,
            base_url="https://api.typesafe.ai",
            retry=RetryPolicy(max_retries=0),
            timeout=30.0,
        ) as client:
            start = perf_counter()
            response = client.system_one(
                model="jev-1.13.0",
                state={
                    "ticket": "The payment failed and I can no longer access my account.",
                    "language": "en",
                },
                questions={
                    "destination": Choice(
                        instructions=(
                            "Which category best describes the main subject of the ticket? "
                            "Evaluate the categories using the supplied text."
                        ),
                        criteria={
                            "facturation": "Rejected payment, invoice or refund.",
                            "acces_compte": "Unable to log in or account access blocked.",
                            "incident_technique": "Outage unrelated to payment or account access.",
                            "autre": "None of the above topics.",
                        },
                    ),
                },
            )
            elapsed = perf_counter() - start
    except TypeSafeAPIError as error:
        raise SystemExit(f"TypeSafe API error: HTTP {error.status}.") from None
    except TypeSafeAPIConnectionError:
        raise SystemExit("Unable to connect to TypeSafe or HTTP timeout exceeded.") from None

    answer = response.choices["destination"]
    print(json.dumps({
        "model": response.model,
        "proposition": answer.choice,
        "probabilities": answer.probabilities,
        "confidence": answer.confidence,
        "duree_appel_secondes": round(elapsed, 3),
        "usage": {
            "input_tokens": response.usage.input_tokens,
            "output_tokens": response.usage.output_tokens,
        },
    }, ensure_ascii=False, indent=2))


if __name__ == "__main__":
    main()

4. Send the fictional ticket. With the virtual environment still active, run the command below.

• It makes 1 SDK attempt to TypeSafe, billable under your account’s terms.
• Automatic retries are disabled.
• The 30-second timeout applies to HTTP operations; it does not guarantee a total duration of 30 seconds.

python premier_appel.py

5. Read the predicted category. The program displays a JSON object with the Choice option and its probabilities. The ticket mentions two topics: inspect their distribution without asking the model to enforce a handling priority. The script does not move any ticket. A successful first call confirms that the connection works for this case; it does not measure classification reliability.

Code owns the decision: an offline demonstration. For a ticket mentioning a failed payment and blocked access, two Nouls can separately evaluate what the text says. These signals do not prove the actual transaction or account state. The block below declares questions without sending them and uses only fictional answers. Save it as decision_policy.py, then run python decision_policy.py with Python 3.10 or later; no key or third-party package is needed.

"""Offline demonstration: fictional values and thresholds, no Jev call."""
import json

# Two independent questions to ask Jev; this file does not send them.
questions = {
    "payment_failed": {
        "type": "noul",
        "instructions": "Does the ticket mention a declined or failed payment?",
    },
    "access_blocked": {
        "type": "noul",
        "instructions": "Does the ticket mention blocked account access?",
    },
}


def decide_route(probabilities, *, threshold=0.8, priority="facturation"):
    # Conservative teaching policy; this threshold is not validated for production.
    if not 0.5 < threshold <= 1 or priority not in ("facturation", "acces_compte"):
        raise ValueError("Invalid policy")
    payment = probabilities["payment_failed"]
    access = probabilities["access_blocked"]
    if not (0 <= payment <= 1 and 0 <= access <= 1):
        raise ValueError("Probabilities must be between 0 and 1")
    if payment >= threshold and access >= threshold:
        return priority
    if payment >= threshold and access <= 1 - threshold:
        return "facturation"
    if access >= threshold and payment <= 1 - threshold:
        return "acces_compte"
    return "relecture"


# Simulated answers to the two Nouls, reused without another call.
probabilities = {"payment_failed": 0.92, "access_blocked": 0.88}
proposals = {
    "billing_first": decide_route(probabilities, priority="facturation"),
    "access_first": decide_route(probabilities, priority="acces_compte"),
    "higher_threshold": decide_route(probabilities, threshold=0.95),
}
assert proposals == {
    "billing_first": "facturation",
    "access_first": "acces_compte",
    "higher_threshold": "relecture",
}
assert decide_route({"payment_failed": 0.52, "access_blocked": 0.48}) == "relecture"
assert decide_route({"payment_failed": 0.92, "access_blocked": 0.10}) == "facturation"
assert decide_route({"payment_failed": 0.10, "access_blocked": 0.92}) == "acces_compte"
print(json.dumps({"questions": questions, "simulated_probabilities": probabilities,
                  "application_proposals": proposals}, ensure_ascii=False, indent=2))

The same probabilities, 0.92 and 0.88, produce facturation (billing) or acces_compte (account access), depending on the priority written in code. A fictional threshold of 0.95 yields relecture (review). The function proposes handling a single issue only if the other is below the complementary threshold; ambiguous cases go to review. The assertions check this local policy, not Jev. No ticket is moved. Measure errors and choose thresholds using your own data before real use.

Prepare the same test with Claude Code or Codex. Copy the following prompt into your assistant from the project directory. It first requests a local simulation; calling the real service remains a separate command that you run. Do not paste your key into the conversation.

Create a minimal demo of TypeSafe’s Jev API in this project, using an entirely fictional ticket. Start by reading the project structure and instructions; preserve existing files.

Documentation to check before coding:
- https://docs.typesafe.ai/introduction/quickstart
- https://docs.typesafe.ai/sdk/python/usage
- https://docs.typesafe.ai/sdk/python/api/retries
- https://docs.typesafe.ai/models

Use Python 3.10+ and typesafe-sdk==0.7.0 in a dedicated virtual environment. Request model jev-1.13.0 and record the identifier actually returned. If this version is no longer documented, note the change in the README.

Case: “The payment failed and I can no longer access my account.” Create a Choice question named destination, with facturation, acces_compte, incident_technique and autre. Describe each category and ask which best describes the main subject. Do not put an operational priority in this question: choice is a predicted category, not a ticket assignment.

By default, run a local simulation using the real SDK and a simulated HTTP transport. Clearly label all simulated values. Add a --live option that I will run myself to send this fictional ticket to the service; do not run it during your work. In live mode, use 1 SDK attempt with RetryPolicy(max_retries=0), a 30-second HTTP timeout, and a key from TYPESAFE_API_KEY or hidden input in the terminal. Never ask for the key in the conversation or write it into code.

Display the predicted category, probabilities, confidence, model, token counts and complete client-call duration. Keep null if a count is missing. Explain that confidence describes the concentration of probabilities, not the probability of being right.

Add a standalone decision_policy.py file with no API client or network access. Declare two independent Noul questions about mentions of a failed payment and blocked access. Use explicitly fictional answers and a decide_route(probabilities, *, threshold=0.8, priority="facturation") function that applies a priority and returns a proposal or relecture (review). Values and thresholds are for teaching, not validated for production. Test that the same probabilities can produce a different proposal when only the policy changes, as well as coexisting issues and ambiguous cases sent to review. Display simulated probabilities and the application proposal separately; do not move a ticket. These signals concern the text, not the actual payment or account state.

Produce the script, installation and execution commands, then offline tests for a valid response, a missing key, an HTTP error and no retry on overload. Make no changes on GitHub or Sentry. State exactly what was tested; a simulated transport tests neither Jev nor access to my account.

After this first test.

• The chapter Calling the API and reading its responses also covers HTTP, TypeScript, Noul and Score.
• To move on to your own tasks, follow the GitHub and Sentry pilots, then the comparison protocol.

Replacing the fictional ticket with a real issue is not enough to validate an automation.

Tutorial references: official quickstart, Python SDK 0.7.0 on PyPI, Python client and options, retry policy. SDK documentation and type declarations reviewed on 21 September 2026.

Prerequisites: an Apple Silicon Mac, Python 3.12 already available, and space for the environment plus roughly 614 MiB of weights. This walkthrough uses the lab versions. Local inference and the command interface were checked; the full installation was not repeated on a clean Mac. Windows, Linux and Intel Macs were not evaluated here. Official setup.

1. From the directory where you keep experiments, create a new folder. These commands install packages over the Internet. If python3.12 is unavailable, install Python 3.12 first. If the folder already exists, choose a different name.

mkdir laya-demo
cd laya-demo
python3.12 -m venv .venv
.venv/bin/python -m pip install \
  'laya-mlx==0.2.0' 'mlx==0.32.2' 'tokenizers==0.23.2' 'numpy==2.5.3' 'huggingface-hub==1.33.0'
.venv/bin/python -m pip freeze > requirements-laya.txt

2. In the same directory, explicitly download the multilingual revision. This step needs Internet and is separate from prediction. Keep requirements-laya.txt to identify the resolved environment.

.venv/bin/python - <<'PY'
from huggingface_hub import snapshot_download

snapshot_download(
 \
   repo_id="aac6fef/laya-multilingual-mlx",
 \
   revision="f2b4faf51023039425946074e2cf1361d2db11d5",
 \
   local_dir="models/laya-multilingual",
)
PY

3. Save the JSON below as questions.json, next to .venv. It requests a category for a short text. The questions are in English; the selected model supports French or English input.

{
  "department": {
    "type": "choice",
    "instructions": "Which team should handle the main request?",
    "criteria": {
      "billing": "Invoices, payments and refunds.",
      "technical": "Software bugs and outages.",
      "other": "A clear request unrelated to billing or technical support."
    }
  }
}

4. Run a prediction using local weights. The JSON should contain answers.department and option probabilities. A duplicate payment suggests the billing category, but this expectation is not a guaranteed result. Valid JSON proves the call works, not that the answer is correct.

HF_HUB_OFFLINE=1 HF_HUB_DISABLE_TELEMETRY=1 HF_HUB_DISABLE_IMPLICIT_TOKEN=1 \
  .venv/bin/laya-mlx predict \
  --model ./models/laya-multilingual \
  --device gpu \
  --questions questions.json \
  --state "I was billed twice. Please refund the duplicate payment."

printf '(version 1)
(allow default)
(deny network*)
' > deny-network.sb
/usr/bin/sandbox-exec -f deny-network.sb \
  env HF_HUB_OFFLINE=1 HF_HUB_DISABLE_TELEMETRY=1 HF_HUB_DISABLE_IMPLICIT_TOKEN=1 \
  .venv/bin/laya-mlx predict \
  --model ./models/laya-multilingual --device gpu \
  --questions questions.json --state "I was billed twice. Please refund the duplicate."

The issue-triage pilot has now run: 30 development examples followed by 60 test examples. The candidate is rejected and human validation of annotations remains outstanding. For another experiment, group duplicates before splitting, have labels reviewed, freeze thresholds before testing and use a new held-out set. Measure per-category quality, coverage, abstention, errors, overflows and full-chain latency. This walkthrough installs the engine; it does not promise a quality gain.

The lab used for our measurements adds a local page, 6 forms, imports and resumable batches. It is not yet distributed: start.command, stop.command and laya-lab are not installed by pip install laya-mlx. The public walkthrough above provides a first inference; reproducing the full lab requires its code and environment.

As of October 28, the Models page lists jev-1.14.0 and jev-1.13.0, at the same price and with the same limits; jev-latest and jev-preview both point to jev-1.14.0. The direct model documented in the initial edition was jev-1.13.0.

• The 64k-token limit covers the state and all questions; the 32k limit covers the state and the longest question.
• The listed ceilings, 250,000 tokens per second and 1,200 requests per minute, are dynamic and may depend on the plan. They are not an availability commitment.

TypeSafe models and limits.

The operators describe these routes on their pages: Vercel, Cloudflare and OpenRouter.

• Their 32k listings are not enough to infer the direct API’s dual 64k/32k budget. Check how the chosen route divides the budget before sizing requests.
• No account access permissions were verified here.
• The Vercel promotion is dated; comparisons must not treat it as permanent pricing.

The adapters do not necessarily use the same names. Vercel, for example, exposes a boolean type where the native API uses Noul, and places some information in provider metadata. Changing gateways can also change billing, limits and data handling. Compare 2 routes as 2 separate configurations, even if both are named Jev.

The official Python and JavaScript SDKs inspected use the MIT license. The Python manifest reviewed specifies 0.7.0 and Python 3.10 minimum; the JavaScript tag inspected is v0.6.0 and requires Node 20 minimum. Python SDK 0.7.0 was also installed for the First call tutorial and run with a simulated transport. The JavaScript client was not run here. Python SDK, JavaScript SDK.

The Perplexity audit stated that all processing terms remained unknown. Official pages allow a more precise account. The table summarizes their statements as of 21 September; it verifies neither their technical implementation nor the compliance of any particular use.

The Trust Center and subprocessors pages provided no usable content to the reading tool. Certifications, the named list of subprocessors and the terms applicable to the account therefore remain unverified. This access limitation does not prove that the information or guarantees are absent.

The Python SDK warns that debug mode may log request and response bodies without redaction. Preparing a pilot therefore also requires checking application-side logs. Sentry messages, bug descriptions and diffs may contain more information than a title alone.

This audit found no explicit ban on publishing benchmarks in the 19 September versions of the website terms and customer agreement reviewed. The agreement still includes confidentiality provisions, imitation or distillation restrictions, and possible special terms. This documentary finding is not blanket permission to publish.

Before publishing your own results, keep the terms you actually accepted and those of the chosen platform. Promotional credits, the nominal rate and the amount paid are 3 separate facts. A free trial does not establish the ongoing cost of operating the system.

I reviewed a selection of my skill, agent and automated task definitions. These proposals build on those existing tasks. Their frequency and cost still need measuring before deciding whether adding Jev is worthwhile.

• For Sentry, Jev would intervene only when exact matches have failed. It would rank candidates already retrieved, with a review before linking anything.
• For release notes, it would help select fixes; the writing model would still write the text. A vague commit title does not establish user impact: include the PR context too.

• For the release notes pilot, the program would collect commits and PR context.
• Jev would suggest a category for each fix.
• The code would apply selection rules, then send the selected changes and their references to the LLM.

Jev could therefore provide an evaluation before code selects which items to pass to the LLM. This division of work needs testing: an LLM can also classify fixes, and simple rules may be enough. The diagram demonstrates no improvement in speed, cost or accuracy.

Another proposal appears in Syntax at 10:54: transcribe content, ask an LLM to write notes, then ask Jev to check claims. This is a suggested use presented in the video, with no accuracy measurement for that pipeline.

• For each test, record errors, the share of cases handled, p50/p95 response times and cost per correct decision.
• Specify the region, number of concurrent requests, input size, question count and version.
• Count network errors, retries and review effort.

Calculate Brier score and ECE from the probabilities relevant to the task, rather than relabeling the confidence field.

For these proposed tests, rules that calculate Sentry severity, normalize URLs or turn a verdict into an action stay in code. The tests concern semantic evaluations where a lexical rule may miss a paraphrase or incorrectly match 2 problems.

1. Describe the decision

   Define the unit being evaluated: ticket, Sentry/issue pair, fix or document. Write down the categories, ambiguities and action the suggestion might trigger.

2. Build the reference answers

   Have people familiar with the task annotate authorized real examples or synthetic ones. On a subset, 2 independent annotations help reveal disagreements. Resolve them and keep the instructions.

3. Split the data

   Reserve separate sets for any adaptation, for choosing questions and thresholds, and for the final test. Group duplicates and paraphrases of the same incident. For a historical workflow, also test a later period.

4. Set the baselines

   Existing rules, lexical search, a classifier or constrained-output LLM, and Jev receive the same relevant information. Offer a variant with decomposed questions to each model concerned.

5. Run and log

   Record the version, API route, client region, options, question count, tokens, timing, raw response and errors. Alternate the order of competing methods to limit the effect of load variations.

6. Evaluate the pipeline

   Measure errors, coverage, retries, cost and latency. Then check the effect on the final writing or investigation, using the same downstream model.

French deserves its own subset: accents, spoken phrasing, domain abbreviations, typos, negation and tickets mixing French and English. A translation of the same case must stay in the same data group. Comparing 2 sets with different topics in French and English does not let you attribute the difference to language alone.

• For many-option choices, vary the number of similar distractors while keeping the same problem.
• Permute both the positions and identifiers of the options.
• Beyond the documented limit, a preliminary filter followed by a final choice forms a new system: measure incorrectly discarded candidates and the cost of both steps.

Starting in suggestion mode lets you compare each output with the usual decision. The log can store a pseudonymized identifier, versions, the suggested decision, what happened to it, and call metrics. API keys and sensitive content do not belong in a results dataset intended for sharing.

For model routing, asking an expert which model they prefer is not enough. Run the task with the candidate models on a reference set, then compare the quality, cost and latency of the chosen model. The router adds its own cost. Agreement with an expert opinion is a separate metric, useful but insufficient to establish an improvement in completed tasks.

• Measure error and coverage for every method. For methods that produce probabilities, also assess calibration and the quality of decisions retained after applying a threshold. The confidence field is not a probability of correctness.
• Tune thresholds on the validation set, then measure error rate and coverage on the held-out test set.

Synthetic text or a rare language does not prove that the model never encountered similar data during training.

The Perplexity audit considers a check before an agent acts. The proposed test uses Jev to flag cases that deserve an additional review. Authorization remains defined by identity, permissions, allowed actions and system approvals. Comparing this signal with local rules and an LLM judge requires counting dangerous actions not flagged, safe actions blocked, latency and cost.

A strict format constrains the shape of the response. It does not prevent a malicious instruction in a ticket from changing the selected category. The test must therefore include ordinary text, instructions embedded in data, and ambiguous wording, without conflating these 3 populations. A reassuring classification does not neutralize an injection.

For GitHub and Sentry, the first output remains a suggestion. Finding an exact identifier, checking access permissions or applying a severity rule is work for code. Reserving the model for a semantic question does not remove the need to test that question: “same likely cause” may be wrong despite very similar error messages.

The official adapter makes comparisons easier for the GitHub and Sentry pilots: the same inputs and questions can go through Jev or an LLM.

• Its discrete mode asks for a decision; its probabilities mode asks for a distribution. These 2 tasks have different costs; the protocol must specify which meets the need.
• The README documents attempt traces and structural corrections.
• This adapter was not run for this report. The First call tutorial checks the official Python client with a simulated transport, without inference.

The SemIf test compares score extraction with generated JSON on an RTX 3090, agreeing on 18 of 21 decisions. Its comparison with Jev reuses the vendor’s public outputs, without a new call to Jev. These results come from the README. No project in this table was installed or run for this report.

In his video, Micah explains reading logits at 02:07, then distillation from a teacher model at 02:39. This trained demonstration differs from SemIf’s frozen Qwen 4B. At 03:55, he explains the limitations of his Doom demo and notes that Jev’s architecture is undisclosed.

The OpenSysOne model card, inaccessible during the initial collection, was read on 21 September. It separates training, calibration and testing.

• On 768 held-out Social IQA decisions, it reports 72.92% versus 70.31% for its base model.
• Its speed measurements use FP32, warm runs and serial execution on a GB10 machine: they compare its own variants, without calling Jev.

The Laya README also reports failures of its base models: on 2,000 typed decisions, they remain below the majority-class baseline, while the fine-tuned model reaches 76.6%.

• The authors reuse Jev figures published elsewhere; this is not a head-to-head test with identical inputs.
• Local latency on a T4 GPU does not include Jev’s network service. Local inference requires hardware, even without an API bill.
• The acronym RLCD used by Laya refers to its published method, without establishing that it reproduces TypeSafe’s method.

• In Pixel 0.4.0, plan routing recognizes words and chooses among 5 analyses.
• File ranking combines several signals.
• Embedding-derived leads are ranked P2 and marked as unverified.
• The concept resolver’s confidence field takes the values resolved, ranked or unresolved; this search status is not a probability of correctness.

To choose between Pixel and Jev, start with the decision you need. Pixel can calculate and rank what its rules and indexes describe; Jev uses learned inference to evaluate criteria supplied in the call. A business rule may be enough to triage tickets. Jev’s usefulness depends on the cases that rule handles poorly and the cost of maintaining it. Pixel’s source code was inspected at the cited version, without a comparative benchmark: equivalent quality or speed between these approaches has yet to be measured.

The terms of jev-agent.com specify that texts are sent to TypeSafe. TokenRa, an API aggregator, lists Jev at $0.042 per million tokens, matching its “official rate” column. These are operator statements, read on 21 September; no call or actual data transmission path was tested here.

• The description of Micah’s video supplied the links to SemIf, jevlike, open-jev and OpenSysOne.
• The description for Sam Witteveen, at 17:50, links to Laya.
• The lists AnotiaWang/awesome-jev, fatwang2/awesome-jev and Made with Jev provided further leads.

Only their relevant sections were reviewed; their selection guarantees neither completeness nor performance.

The September 24 additions are split between commands and integrations and Laya-MLX / Kev. The former use an evaluation interface; the latter serve different weights. Their results do not reveal Jev’s architecture.

The September 24 collection adds 17 repositories absent from version 0.15.0. Some existed before September 21: discovery and creation dates are kept separate. Relevant sources were read and parts of jgrep’s code inspected; none of these tools was installed or run for this report.

The commands share jevkit-core, distributed as jevkit-runtime, for transport, caching, provenance and cost accounting. They have the same author; their number does not count independent validations. Versions observed on September 23: jgrep 0.4.1, jselect 0.3.1 and runtime 0.3.1. Local-server support was introduced in the preceding September 22 versions.

The inspected jgrep code builds Noul questions and applies a default threshold of 0.5. With multiple descriptions it takes a minimum or maximum depending on the mode; these operations do not compute a joint probability. The README claims about 200 ms per call and reports processing 994 titles in 4.6 seconds for $0.012, with concurrency. These are the author’s figures.

Adapted examples, not executed. They assume jgrep is installed and a provider is configured; hosted providers may charge for calls. The first expects a CSV with an abstract column; the second evaluates hunks in the current diff. Outputs need review.

jgrep --csv --field abstract "reports a failed replication" papers.csv
git diff --no-color | jgrep --diff "removes error handling for a persistent write"

kyu1204’s test selection first keeps files by name and imports, then asks Jev about other candidates using the diff and test names. The author instructs users to run the full suite afterwards. Measure time to useful feedback and missed regressions, including selection overhead.

Laya-MLX independently ports Laya to MLX on Apple Silicon. After downloading weights, inference is local and returns Choice, Score and Noul. The README pinned to September 22 documents a 421-million-parameter English model, a 322-million multilingual model and a 421-million typed-decisions variant. It does not provide Jev’s weights. Declared version: 0.2.0, commit 0a85951.

Recalculating the 50 samples in each file gives 17.745958 and 10.9073125 ms. The data is dated September 22, on an M3 Max with 40 GPU cores and 128 GiB. Timing includes preparation, synchronized inference and formatting; loading and downloads are excluded. The README disagrees with these files at the same commit. Sources: report, English, multilingual. These M3 Max benchmarks were not rerun under the same conditions.

Agreement with the upstream runtime on 63/63 answers for each model and precision measures port fidelity. The small AG News test uses a corpus present in the upstream training mix. These 2 observations do not establish general equivalence with Jev.

The JevKit guide supplies a separate server adapter and documents jgrep --api laya. It rejects inputs that would be cropped by default. The author tested only the English 421M model; the guide pins the port to September 19, before the calibration correction. Those trials therefore do not automatically validate the September 22 revision. The jgrep comparison also shows task-dependent rankings.

In the Snake demo, code prepares board information, Laya evaluates proposals, and a safety rule can correct the action. Survival results concern this combined system; they do not demonstrate unaided board reasoning.

Kev provides weights and training recipes for a Qwen3.5 family of 0.8B, 4B and 9B models. Its API follows System One; the reviewed README documents CUDA, ROCm and Apple Silicon through MLX. Its reference to an analysis titled “Architecture Unmasked” does not establish that Jev’s proprietary architecture is known.

On September 24, Kev-4B received a specialization on financial complaints: the reported gain stays within the same domain and question templates. Commit 60c8d39 adds batching and GPU optimizations. Because Jev’s training data is unknown, the comparison cannot isolate architectures. The repository also retains external evaluations where Kev trails Jev.

The Perplexity audit provides research leads. Its review articles, videos and commercial retellings are not counted as separate replications. Primary sources were preferred for the API, access terms and experiments; secondary pages remain useful for finding an author or a test.

Monitoring flags useful changes; it neither publishes the report nor launches paid tests. Each revision preserves source dates, raw results and earlier versions.

Further research on Jev / TypeSafe. Baseline report: version 0.16.0, collection dated September 24, 2026. Set a new end date and find subsequent changes; separately mark older projects missing from the report.

Already covered: primitives, calibration, incompletely disclosed RLCD, earlier benchmarks and 17 additional repositories: keltokhy/jgrep, jsort, jlink, jcol, jselect and jevkit-core; nassim-arifette/jevgrep; sglenon/jev-semantic-reviewer; supercorp-ai/supercov; simonw/llm-typesafe; edwardyen724-g/jev-compactor; uehaj/jev-semgrep; kyu1204/jgrep; NiazMorshed2007/jev-review; devagrawal09/jev-review; mizorewww/laya-mlx; jaredpalmer/kev. Do not present them as new without a dated change. The jgrep/jevgrep projects are distinct.

Check TypeSafe releases (Python 0.7.1 observed, report tutorial tested on 0.7.0; JS 0.6.0), models, pricing and limits. Distinguish hosted Jev from local models. For Laya-MLX, examine the README/raw-data discrepancy, truncation and corrected temperatures; the JevKit guide pins an earlier revision. For Kev, distinguish development, test and domain specialization.

Explore composable tools, context selection, entity resolution, extraction of existing values, citations, supervised features and test selection. Read the skill_suggestion, autoresearch_feature_discovery, pre_parsed_value_extraction_cookbook and citation_check cookbooks at docs.typesafe.ai. Try to retrieve autoformat, listed in llms.txt but inaccessible during collection. Their Jev 1.12 examples do not become new September measurements.

Prioritize primary sources, commits, protocols, failures and raw responses. Jev evaluates; code applies thresholds, permissions and actions. Confidence is not a probability of correctness. Do not generalize 100× factors, per-email costs or 300 ms loops. Do not infer proprietary architecture from a reimplementation.

Deliver 5 useful discoveries, a dated matrix of changes and sources, limitations, and experiments with comparators and acceptance criteria. Separate documented features, authors’ figures, reproduced results and proposals. Use UNKNOWN for unverifiable claims. Do not install anything, call the API or publish. Previous Perplexity syntheses contained errors; check every claim at its source.

A publication detailing RLCD would allow its novelty to be examined. A comparison with competing training or calibration methods should control the base model, data and settings. That publication depends on TypeSafe; it is something to watch for, not a test a reader can run with the API alone.

The official index also lists an autoformat cookbook for reconstructing Markdown structure. Its detailed page was inaccessible during collection: this remains a lead, without a retained result.

• Portable Claude Code and Codex configuration (EN): organize instructions, skills, hooks and checks in a shared configuration.
• Back Market and OpenRouter (FR): practical experience with budgets and provider selection behind a shared API.
• Designing an agent’s loops and steps (EN): define steps, stopping conditions and handoff to a person.
• Evaluating an agent (EN): choose quality criteria and compare runs with their tools and rules.
• Calculating cost per accepted task (EN): count attempts, retries and review effort in the cost of an accepted result.
• Securing agents, hooks and tools (EN): address permissions, dependencies and untrusted inputs.

Abstention

A case that the application leaves to a person or another method because it cannot decide under its rules.

Calibration

Agreement between predicted probabilities and outcomes across a set of cases. Among predictions at 80%, roughly 80% successes are expected; this guarantees no individual answer.

Typed answer

Answer whose shape is defined before the call: an option and probabilities with Choice, a score and probabilities with Score, or the probability of yes with Noul. Application code determines how to use it to decide what happens next.

Conclusion

As of September 27, 2026, this report separates Jev, published benchmarks, our synthetic Laya experiments and the Aristote issue pilot. Local inference works. The pilot nevertheless misses the required gain: 8 matching suggestions out of 11 emitted by Laya, versus 12 out of 17 for title-only rules, across 60 tickets. No Jev API call or direct Jev/Laya comparison was performed.

Keep rules and exact links. The area pilot and 3 Sentry, review and CI POCs reject the tested Laya configurations. Across the 120 cases in the latest 3 POCs, matching suggestions are 2/38, 1/9 and 6/20; CI rules reach 5/6. Agent annotations without human validation and diagnostic corpora limit generalisation. Any new configuration requires another held-out set. The first pilot’s lexical variant remains undeployed.

As of October 28, the question is no longer only “Jev or an LLM?”. Clef, Strands Decider, pplx-decider and the Decisions API offer the same family of answers; third-party rankings place several models within a few points of each other, and their authors name no overall winner. My Laya POCs remain negative. The next local candidate is Strands Decider 2B, on the same held-out sets and with recalibrated thresholds.

Dated sources

• Introducing System One Models & Jev · TypeSafe AI / Diogo Almeida · accessed September 21, 2026 · 15 September 2026 · Source documentaire
• Introduction · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Choice · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Score · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Noul · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Confidence · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• AI primer · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Models · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• API reference · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Jev 1.13 jaggedness · TypeSafe AI · accessed September 21, 2026 · 17 September 2026 · Source documentaire
• Workflow evals · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• System One Adapter · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Python SDK manifest · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• JavaScript SDK manifest · TypeSafe AI · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• InstructGPT paper · OpenAI authors / arXiv · accessed September 21, 2026 · 4 March 2022 · Source documentaire
• Privacy policy · TypeSafe AI · accessed September 21, 2026 · 19 November 2025 · Source documentaire
• Phishing benchmark · anisselbd · accessed September 21, 2026 · 17 September 2026 · Source documentaire
• BTZSC pilot · AbdelStark · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Jev measured · WallerChen · accessed September 21, 2026 · 20 September 2026 · Source documentaire
• JevBench · Benchmark Heaven / fstandhartinger · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• TypeSafe Jev on AI Gateway · Vercel · accessed September 21, 2026 · 16 September 2026 · Source documentaire
• Jev reranking benchmark, MIRACL French · anessbelbati · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Jev 1.13.0 behavior study, arithmetic permutations · RINNECODER · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Calibration, Guo et al. · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• SelectiveNet · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Uncertainty under shift, Ovadia et al. · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• BERT · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• ModernBERT · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• RouteLLM · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• SCX Router, abstract only · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Text generation · Hugging Face · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• ASSAY-001, report and data · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• ASSAY-001, protocol · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• ASSAY-001, first protocol amendment · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Vercel AI SDK, TypeSafe AI: rounding and confidence · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Agent Journal, LLM judge vs feature extraction · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• LangChain, Building a Harness with Jev · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Jev as a judge, experiment repository · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Rewarding Doubt, v6 · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• DeepSeek-R1, abstract · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Tülu 3, abstract · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• TechCrunch, interview with Almeida · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• TypeSafe team · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• DCVC funding announcement · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• State, shared context · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Python SDK, usage · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Python SDK, response types inspected · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• JavaScript SDK v0.6.0, response types and options · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• JavaScript SDK v0.6.0, question helpers · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• JavaScript SDK v0.6.0 · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Vercel, Jev listing and promotion · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Cloudflare, Jev · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• OpenRouter, Jev catalog · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• TypeSafe, DPA · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• TypeSafe, MCA customer agreement · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• TypeSafe, legal documents and ZDR · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• TypeSafe, website terms · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Capital & Compute, compilation of costs and use cases · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Near Here, event validation · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• gemanor, code-review benchmark · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• paddo, Thirty-Cent Judge · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• bitnovus, email-filtering experiments · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• SemIf, decision interface on Qwen · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• jevlike, community prototype · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• daseinlabs, open-jev on MLX · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• razorback16, openjev with DiffusionGemma · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• AnotiaWang, awesome-jev catalog · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Made with Jev, project catalog · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• jevtypesafeai.com, third-party service documentation · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• jev-agent.com, third-party website terms of use · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• CounterProof, review of consensus labels · accessed September 21, 2026 · 21 September 2026 · Source documentaire
• Every · article by Mike Taylor (title and URL; body not read) · 21 September 2026 · Source documentaire
• Arize / Laurie Voss · evaluation of Jev as a judge · 21 September 2026 · Source documentaire
• OpenSysOne · model card published by andyshu · 21 September 2026 · Source documentaire
• TokenRa · advertised offer and prices · 21 September 2026 · Source documentaire
• NandhaKishorM / Laya · README and comparison limitations · 21 September 2026 · Source documentaire
• LivioGama / Pixel · code intelligence and classification · 21 September 2026 · Source documentaire
• TypeSafe · official Python SDK · 21 September 2026 · Source documentaire
• Browser Use · Jev Ultrafast demonstration · 21 September 2026 · Source documentaire
• Aness Belbati · reranking benchmark · 21 September 2026 · Source documentaire
• ThursdAI · September 17 episode and Allie Laabs’s presentation · 21 September 2026 · Source documentaire
• ExplainX · republication of Jev-as-a-judge benchmark figures · 21 September 2026 · Source documentaire
• Yin et al., 2019 · zero-shot classification through NLI · 21 September 2026 · Source documentaire
• Hugging Face · ZeroShotClassification pipeline · 21 September 2026 · Source documentaire
• Anthropic · structured outputs and schema limitations · 21 September 2026 · Source documentaire
• simonmesmith · Jev on Banking77 with examples, README consulted · 21 September 2026 · Source documentaire
• MindStudio · overview of classifiers, original protocol not found · 21 September 2026 · Source documentaire
• Hacker News · Jev launch, public discussion · 21 September 2026 · Source documentaire
• Hacker News · zero-shot description by petesergeant · 21 September 2026 · Source documentaire
• Hacker News · CompleteSkeptic’s response to the zero-shot description · 21 September 2026 · Source documentaire
• Hacker News · CompleteSkeptic identifies as CEO · 21 September 2026 · Source documentaire
• Hacker News · architecture confidentiality and discussion of a paper · 21 September 2026 · Source documentaire
• Hacker News · valid type and factual accuracy · 21 September 2026 · Source documentaire
• Reddit / LocalLLaMA · hypotheses about Jev’s architecture · 21 September 2026 · Source documentaire
• Reddit / LocalLLaMA · comparison with BERT · 21 September 2026 · Source documentaire
• Reddit / accelerate · summary and reactions · 21 September 2026 · Source documentaire
• Reddit / accelerate · successive letter choices · 21 September 2026 · Source documentaire
• Reddit / accelerate · discussion of email classification · 21 September 2026 · Source documentaire
• Nandakishor M · claim of prior work, author’s post · 21 September 2026 · Source documentaire
• SalesRLAgent · record and abstract, PDF not audited · 21 September 2026 · Source documentaire
• Confidence-based routing · record and abstract, PDF not audited · 21 September 2026 · Source documentaire
• Laya · pinned README, calibration scopes · 21 September 2026 · Source documentaire
• Laya typed-decisions · fine-tuned model and calibration settings · 21 September 2026 · Source documentaire
• TypeSafe · official quickstart · 21 September 2026 · Source documentaire
• PyPI · typesafe-sdk 0.7.0 and Python requirements · 21 September 2026 · Source documentaire
• TypeSafe · synchronous Python client and transport · 21 September 2026 · Source documentaire
• Theo · Jev is incredible · full transcript analyzed on September 22, 2026 · 22 September 2026 · Source documentaire
• Fireship · An ex-OpenAI researcher just deleted language from the LLM... · full transcript analyzed on September 22, 2026 · 22 September 2026 · Source documentaire
• Jev Engineering 101 · Daniel Moka / Craft Better Software · educational inspiration, accessed September 23, 2026 · 22 September 2026 · Source documentaire
• keltokhy/jgrep · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• keltokhy/jsort · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• keltokhy/jlink · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• keltokhy/jcol · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• keltokhy/jselect · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• keltokhy/jevkit-core · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• nassim-arifette/jevgrep · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• sglenon/jev-semantic-reviewer · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• supercorp-ai/supercov · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• mizorewww/laya-mlx · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• simonw/llm-typesafe · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• edwardyen724-g/jev-compactor · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• uehaj/jev-semgrep · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• kyu1204/jgrep · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• NiazMorshed2007/jev-review · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• devagrawal09/jev-review · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• jaredpalmer/kev · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• jgrep · Noul questions and filtering · 5727fe3 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• jgrep · code-review diagnostic · September 19 run · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• jgrep · local model comparison · September 22 runs · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• JevGrep · retrieval pilot protocol · 2495212 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Laya-MLX · README · 0a85951 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Laya-MLX · detailed local benchmarks · 0a85951 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Laya-MLX · English FP16 raw timings · September 22 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Laya-MLX · multilingual FP16 raw timings · September 22 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Laya-MLX · input preparation and truncation · 0a85951 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Laya-MLX · calibration temperature handling · 0a85951 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• JevKit · Laya adapter and pinned environment · 3a672d7 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Laya-MLX · Snake planner and safety interventions · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Kev · model family and evaluation boundaries · 30c619b · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Kev · batching and GPU optimizations · September 24 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Kiln · Auto Optimizing Jev with Autoresearch · September 21 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• TypeSafe Python SDK · v0.7.1 release · September 21 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• TypeSafe · skill suggestion cookbook · Jev 1.12 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• TypeSafe · AutoResearch feature discovery · Jev 1.12 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• TypeSafe · pre-parsed value extraction · Jev 1.12 · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• TypeSafe · citation checking · August 16 example · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• TypeSafe · documentation index and unresolved autoformat lead · accessed September 24, 2026 · 24 September 2026 · Source documentaire
• Local Laya experiments: accompanying synthetic data and evidence · 26 September 2026 · Observation vérifiée
• Multilingual weights: revision used in our experiments · 26 September 2026 · Source documentaire
• Laya-MLX 0.2.0: installed version · 26 September 2026 · Source documentaire
• Laya / Aristote pilot: aggregates and configuration, private corpus excluded · 27 September 2026 · Observation vérifiée
• Laya Sentry, review and CI POCs: aggregates and configurations, private corpus excluded · 27 September 2026 · Observation vérifiée
• Cloudflare, Clef and Clef-flash launch · accessed October 28, 2026 · 1 October 2026 · Source documentaire
• Cloudflare, Workers AI changelog: Clef · accessed October 28, 2026 · 1 October 2026 · Source documentaire
• Cloudflare, Clef-omni and Clef-flash price cut · accessed October 28, 2026 · 9 October 2026 · Source documentaire
• Hugging Face, Clef model card · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• Hugging Face, Clef-omni model card · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• Cloudflare, Clef-omni Workers AI listing · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• Cloudflare, Jev 1.14 Workers AI listing · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• TypeSafe, Jev 1.14 and experimental features · accessed October 28, 2026 · 14 October 2026 · Source documentaire
• TypeSafe, Series A announcement · accessed October 28, 2026 · 9 October 2026 · Source documentaire
• Andreessen Horowitz, investing in TypeSafe · accessed October 28, 2026 · 9 October 2026 · Source documentaire
• TypeSafe, Jack & Jill case study · accessed October 28, 2026 · 7 October 2026 · Source documentaire
• TypeSafe, Box question type · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• TypeSafe, image inputs · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• TypeSafe, Python SDK changelog · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• OpenAI, Decisions API guide · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• OpenAI, API changelog · accessed October 28, 2026 · 22 October 2026 · Source documentaire
• Strands Labs, strands-decider repository · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• Hugging Face, Strands Decider 2B v1-2610 card · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• Astronomer, how to evaluate a decision model · accessed October 28, 2026 · 8 October 2026 · Source documentaire
• AIMultiple, AIM-Decision: Jev, Clef and LLMs · accessed October 28, 2026 · 22 October 2026 · Source documentaire
• Leanmcp, JevBench Index · accessed October 28, 2026 · 11 October 2026 · Source documentaire
• Ollama, release 0.35.0 · accessed October 28, 2026 · 28 September 2026 · Source documentaire
• Hugging Face, pplx-decider v1.1 card · accessed October 28, 2026 · 28 October 2026 · Source documentaire
• LiteLLM, /v1/decisions and /v1/systemone routes · accessed October 28, 2026 · 14 October 2026 · Source documentaire

“Accessed on” gives the date the source was read. A source may have been published earlier and later updated.

Update history