Classifier models in Glove: decide what matters without reading it

Most of what an agent reads, it reads to answer a small question. Is this email a refund request? Which team owns this ticket? Did the checkout page load? glove-classifier lets those questions go to a model built to answer them, and only the answer comes back.

A language model is a general tool, and agents built on one tend to push every judgement through the main loop. To learn which of four hundred emails ask for a refund, the planner reads four hundred emails. To learn whether a page shows a login wall, it reads the DOM. The conversation fills with material the agent needed for one bit of information and will never look at again.

Glove has spent several releases moving work out of the context window. The scratchpad and the REPLs let a program touch data the model never sees, and the working environment gives that program a filesystem. glove-egress makes the boundary enforceable. The missing piece was the judgement itself. When the program meets an email, what decides whether the email matters?

A different kind of model

TypeSafe recently released Jev, the first of what it calls System One models. Jev does not generate text. You send a state and a set of typed questions, and it returns typed answers. There are three kinds of question: a noul returns the probability that a statement is true, a choice picks a label, and a score places the state on a rubric. Each answer carries its full probability distribution and a confidence value, and all the questions are evaluated in parallel in one pass.

import { jev, noul, choice, score } from "glove-classifier";

const { answers, model } = await jev().classify({
  state: "Help! My payouts have been failing for 3 days and I'm losing sales.",
  questions: {
    urgent: noul("Does this convey urgency?"),
    team: choice("Which team should handle this?", {
      billing: "Payments, invoices, refunds",
      technical: "Bugs, outages, integrations",
      sales: "Pricing, upgrades",
    }),
    frustration: score("How frustrated is the customer?", ["Calm", "Frustrated", "Very angry"]),
  },
});

answers.urgent.noul;        // 0.97
answers.team.choice;        // "billing" (typed: "billing" | "technical" | "sales")
answers.team.probabilities; // { billing: 0.91, technical: 0.09, sales: 0 }
answers.team.confidence;    // 0.86
answers.frustration.score;  // 1.18, between "Frustrated" and "Very angry"
model;                      // "jev-1.13.0", answered in 227 ms

Those values are from a live call. Jev cannot sit behind Glove's ModelAdapter, because it does not chat, stream or call tools, so glove-classifier gives classifier models their own small contract, ClassifierAdapter. Jev implements it over plain fetch. llmClassifier() implements it over any existing ModelAdapter, so the same questions run on the model you already use. cascade() composes the two.

Jev is not the only model of its kind. Within weeks, open models appeared that serve the same /v1/systemone API on your own hardware: Kev, Laya, Von, Rizzo Flow and Decider. Glove has a preset for each (kev(), laya(), and so on), and systemOne() covers any other compatible server. Older zero-shot classifiers such as NLI models and GLiClass plug in through labelScorer(). On our inbox, Laya running on four CPU cores with no GPU scored 75–89% at 2.85 s per message, against Jev's 97.5–100% at 18 ms. A self-hosted model is a fair choice when data can't leave your network, and a cascade can send its uncertain answers onward.

Less context: judge the data where it lives

Where the question is asked decides what the agent has to read. We measured it on a labelled support inbox of 80 messages, about 24k tokens with the signatures, quoted threads and footers real mail carries. Every agent got the same request: “which messages ask for a refund, and how many are urgent?” One refund request contains a planted customer account number.

The agent on the right never read a message. The host registered the inbox as a classifier source. The agent asked both of its questions in one call and got back ids, subject lines and answers:

import { mountClassifier, jev, noul, choice } from "glove-classifier";

mountClassifier(agent, {
  classifier: jev(),
  sources: {
    inbox: {
      description: "The support inbox",
      load: async () => (await mail.unread()).map((m) => ({ id: m.id, label: m.subject, state: m })),
    },
  },
});

// What the agent called, verbatim from a benchmark run:
// glove_classify_source({
//   source: "inbox",
//   questions: { refund: "Does the sender ask for a refund or their money back?",
//                urgent: "Is the message urgent?" },
//   where: { question: "refund" }
// })

Five places to put the question

A source is one pattern. The same idea applies everywhere data flows into an agent: keep the data where it is, ask the classifier there, and pass on only the answer.

In a REPL program

An agent writing code can move records it never reads. classifierFns() adds classifier.many() to the JavaScript, Python and Lisp REPLs. REPL programs call host functions one at a time, so many judges a whole batch in parallel inside a single call and returns plain values that a program can compare directly:

const judged = classifier.many({
  items: inbox.list().map(m => ({ id: m.id, label: m.subject, state: m })),
  questions: {
    refund: "Does the sender ask for a refund, a chargeback reversal, or their money back?",
    urgent: "Does the sender need this handled today?",
  },
  where: { question: "refund" },
});
({
  refunds: judged.map(j => j.id),
  urgent: judged.filter(j => j.answers.urgent > 0.5).map(j => j.id + " — " + j.label),
})
// 80 judgements in 1.46 s · returned 592 characters instead of 96,980

In a working environment

import { glob, readFile } from 'env:fs';
import { describe } from 'env:email';
import { many } from 'env:classifier';   // classifierEnv(jev()) in the host's stdlib

export default async function () {
  const items = [];
  for (const path of await glob('/inbox/*.eml')) {
    const meta = await describe(path);
    items.push({ id: path, label: meta.subject, state: (await readFile(path)).slice(0, 20000) });
  }
  const hits = await many({ items, questions: { refund: 'Does the sender ask for a refund?' }, where: { question: 'refund', min: 0.7 } });
  return hits.map(h => ({ path: h.id, subject: h.label, p: h.answers.refund }));
}

In a browser

// host: mountBrowser(glove, { adapter: withClassifier(stationBrowser({ client }), { classifier: jev() }) })
browser.navigate({ sessionId, url: "https://shop.example/orders/latest" });
const { answers } = browser.judge({
  sessionId,
  questions: {
    placed: "Did the order go through?",
    wall: "Is the page asking the user to sign in?",
  },
});
answers   // the page itself never enters the agent's context

Before an agent starts, in Foundry

Inbound transmissions pass through a classify step and each playbook's predicates before any agent runs. A classifier there decides which event a message is and which playbooks wake for it, so an agent starts only for the messages that need one:

import { defineTransmissionPredicate } from "glove-foundry";
import { classifierPredicate } from "glove-classifier/foundry";

export default defineTransmissionPredicate(classifierPredicate({
  classifier: jev(),
  questions: { urgent: noul("Does the sender need help today?") },
  where: { question: "urgent", min: 0.7 },       // a playbook can pass { min: 0.9 }
  state: (event: Ticket) => ({ subject: event.subject, body: event.body }),
}));

Speed: one parallel pass, not a turn per item

A language model answers one token at a time, so judging each item means a model turn per item. A System One model evaluates every question in one parallel pass. Here are the same 80 messages and the same three questions on each classifier, with Glove running eight requests at a time:

That is roughly 12× faster and 11× cheaper than the LLM on this set, and at least as accurate. The inbox is templated, so it measures the mechanism rather than serving as a leaderboard. TypeSafe's own figures are 70–500 ms per call, with only input tokens priced.

Privacy by depending on assertions

Glove's egress work began with an uncomfortable measurement. We gave cheap models a task over records that contained planted secrets. With a raw tool surface, secrets leaked in 75% of runs. Telling the model to return only decisions reduced that to 33%, and no further. Only an enforced boundary reached 0%: the program may return an assertion, a count, or a choice from a short list, and never a raw record. Our conclusion was that a privacy boundary that depends on the model's goodwill is not a boundary.

An assertion-only boundary needs something to produce the assertions. For judgements (“is this feedback negative?”), the same study delegated each document to a small classifier inside the sandbox. Judging accuracy went from 25% to 75%, and leakage from 25% to 0%, because the documents reached the judge but never the planner. The inbox benchmark repeats that result. The agents that read the inbox saw the planted account number in 6 of 6 runs. The agents that classified it saw it in 0 of 12.

Classifier answers are bounded by construction. A noul is a probability, a choice is one label out of k, and a score is a level on a rubric you wrote. That is exactly the shape glove-egress budgets for: a yes/no crossing carries at most one bit, and a k-way choice at most log₂ k. The boundary lies between the data and the agent's context. The classifier itself does see the content, so choose it with that in mind. TypeSafe states that Jev is not trained on customer requests. For data that must stay in your infrastructure, llmClassifier() runs the same questions on a model you host.

Knowing when not to act

import { cascade, gate, jev, llmClassifier } from "glove-classifier";

const classifier = cascade({
  primary: jev(),
  fallback: llmClassifier({ model: reasoningModel }),
  threshold: 0.6,                          // re-ask only the uncertain answers
});

const { answers, escalated } = await classifier.classify({ state: ticket, questions });
switch (gate(answers.team, { act: 0.8, review: 0.5 })) {
  case "act":      return route(answers.team.choice);
  case "review":   return confirmWithUser(answers.team.choice);
  case "escalate": return handToHuman(ticket);
}

Keep questions atomic, one quick expert judgement each, and combine them in code. Weighting then lives in a coefficient you can change instead of a prompt you have to rewrite.

What the benchmark taught the library

The first version of these tools did not get these numbers. The runs showed us where the design was wrong:

• A strict schema is a wall. The first source runs spent their whole turn budget on validation errors. The model wrote questions as bare strings, left out instructions, and wrote type: "yes_no". Each has one obvious meaning, so the tools now accept them.
• Programs want plain values. REPL agents wrote answers.refund > 0.5 and answers.team === "billing" against nested objects, or skipped the classifier and matched on keywords instead. Keyword matching caught phishing (“claim your refund!”) and a thank-you note, and missed “money back”. classifier.many now returns scalars, and REPL F1 rose from 0.65 to 0.97.
• Truncation needs to be said, not flagged. An agent ignored a truncated: true flag and reported 18 of 24 refunds. Truncated results now carry a note that tells the agent how to narrow or widen them.

Try it

pnpm add glove-classifier
# the benchmark and demos:
pnpm --filter glove-classifier-inbox-example repl
pnpm --filter glove-classifier-inbox-example agent "Which messages ask for a refund?"

The classifier guide covers every integration. The egress and code execution guides explain the boundary this work builds on.

Also in this release: glove-core now includes the Vercel AI Gateway as a provider. Use createAdapter({ provider: "vercel" }) with AI_GATEWAY_API_KEY. Inside a Vercel deployment it falls back to the OIDC token Vercel provides automatically.