A request to correct a spelling mistake and a request to design a distributed ledger need different kinds of work. An agent can send both to the same model, but it can also make a small decision first: which model is sufficient for this task?
TypeSafe's Jev gives us a way to make that decision in Elixir. It evaluates supplied state and returns typed answers. BeamWeaver 0.1.28 adds Jev as a first-class decision model, with direct invocation, batching, async calls, caching, and tracing. Model-routing middleware is one application of that integration.
We will build two things: a support-ticket classifier that returns a team and priority, and an agent that uses Jev to choose between OpenAI's Luna and Sol. The code will own the thresholds and fallback behavior. The resulting model calls can be inspected in WeaveScope.
What Jev returns
Jev evaluates a state against named questions. State can be text or structured JSON-compatible data. Each question uses one of three primitives:
| Primitive | Use it for | Returned fields |
|---|---|---|
| Choice | Selecting a team, category, or model | choice, probabilities, confidence |
| Score | Rating something against ordered levels | score, legend, probabilities, confidence |
| Noul | A yes/no judgment | noul, the probability of yes |
A Score with three levels ranges from 0 to 2 and can fall between levels. A Noul of 0.5 means yes and no have similar probability; it does not mean medium severity. Noul has no separate confidence field. The TypeSafe primitive guide explains these distinctions.
This also explains the BeamWeaver API: initialize Jev with init_decision_model/2. Your generative agent still uses a chat model through init_chat_model/2.
Set up BeamWeaver and credentials
Add the released BeamWeaver package to your application's dependencies:
# mix.exs
defp deps do
[
{:beam_weaver, "~> 0.1.28"}
]
end
Then run mix deps.get. In config/runtime.exs, configure TypeSafe and, for the routing example, OpenAI:
import Config
config :beam_weaver,
typesafe: [
api_key:
System.get_env("TYPESAFE_API_KEY") ||
System.fetch_env!("TYPESAFE_API")
],
openai: [api_key: System.fetch_env!("OPENAI_API_KEY")]
Export the credentials in the process running your application. BeamWeaver does not automatically load .env files. If you only want classification, omit the OpenAI configuration.
The remaining examples can run in iex -S mix inside that application. Start by creating a decision model:
alias BeamWeaver.Models
alias BeamWeaver.Core.DecisionModel
alias BeamWeaver.TypeSafe.Question
jev = Models.init_decision_model!("typesafe:jev-1.13.0")
Pinning the model makes prompt evaluation easier to reproduce. The aliases typesafe:jev-latest and typesafe:jev-preview are also supported, and the response reports the version that actually answered.
Classify a support ticket in one request
Suppose we need three independent judgments: the responsible team, whether the situation is urgent, and the functional impact. Give all three questions the same state in one call:
input = %{
state: %{
ticket: %{
subject: "Checkout is failing",
message: "Customers get an error when paying. Nobody can complete an order."
}
},
questions: %{
team: Question.choice(
instructions: "Which team should investigate `ticket`?",
criteria: %{
engineering: %{
covers: "Software defects, outages, and integration failures",
excludes: "An invoice question when the service works"
},
billing: %{
covers: "Invoices, subscriptions, and charges when the service works",
excludes: "A checkout failure preventing purchases"
},
other: "The issue does not fit engineering or billing"
}
),
urgent: Question.noul(
instructions: "Does `ticket` describe an ongoing disruption blocking customers?"
),
impact: Question.score(
instructions: "How severe is the functional impact described in `ticket`?",
criteria: [
"Cosmetic issue; functionality works",
"Degraded functionality with a usable workaround",
"Customers cannot complete the affected task"
]
)
}
}
classification = DecisionModel.invoke(jev, input)
Answers come back under string keys, even though we used atom keys in the request. We can now apply an explicit application policy without parsing generated prose:
decision =
case classification do
{:ok, response} ->
team = response.answers["team"]
urgent = response.answers["urgent"]
impact = response.answers["impact"]
priority =
if urgent.noul >= 0.9 and impact.confidence >= 0.8 and impact.score >= 1.5,
do: :high,
else: :normal
assignment =
if team.confidence < 0.8 or team.choice == "other",
do: {:manual_review, :uncertain_team},
else: {:queue, team.choice, priority}
IO.inspect(response.usage, label: "Jev token usage")
IO.inspect(response.metadata.cost, label: "Estimated cost")
%{assignment: assignment, answers: response.answers}
{:error, error} ->
%{assignment: {:manual_review, :classifier_error}, error: error}
end
IO.inspect(decision, label: "Ticket decision")
This example only computes an assignment; it does not create a ticket or page an engineer. The thresholds are example policy values to evaluate on your own tickets. Keeping the individual answers lets you inspect why the policy chose a queue or sent the ticket for review.
Use Jev as model-routing middleware
Now use the same decision model to select the chat model that handles an agent run. We will offer Luna for direct tasks and Sol for tasks that need deeper reasoning.
Describe the work each route is intended to handle. A label such as fast gives your code a convenient key, but the criteria should explain the actual boundary.
alias BeamWeaver.Agent
alias BeamWeaver.Agent.Middleware.TypeSafeModelRouter
alias BeamWeaver.Core.{Message, Tool}
fast = Models.init_chat_model!("openai:gpt-5.6-luna", timeout: 60_000)
powerful = Models.init_chat_model!("openai:gpt-5.6-sol", timeout: 60_000)
router = TypeSafeModelRouter.new(
classifier: jev,
min_confidence: 0.8,
choices: %{
"fast" => %{
model: fast,
criteria: %{
covers: "Direct lookup, extraction, localized changes, or a specified tool with known inputs",
excludes: "Architecture design, novel diagnosis, or consequential tradeoffs",
examples: ["Call a calculator with supplied operands", "Correct a spelling error"]
}
},
"powerful" => %{
model: powerful,
criteria: %{
covers: "Architecture design, novel root-cause analysis, and complex correctness reasoning",
excludes: "Mechanical execution of a supplied operation",
examples: ["Design distributed consistency and recovery", "Diagnose an intermittent failure"]
}
}
}
)
Add a small tool so we can see that routing holds across a model → tool → model turn:
multiply = Tool.from_function!(
name: "multiply",
description: "Multiply two numbers exactly.",
input_schema: %{
type: :object,
properties: %{a: %{type: :number}, b: %{type: :number}},
required: ["a", "b"]
},
handler: fn %{"a" => a, "b" => b}, _opts ->
to_string(a * b)
end
)
{:ok, agent} = Agent.build(
model: fast,
tools: [multiply],
middleware: [router],
system_prompt: "Use multiply for arithmetic. Keep explanations concise.",
model_opts: [max_output_tokens: 1200],
recursion_limit: 12
)
prompts = [
"Use multiply to calculate 7 times 8 and report the result.",
"Design a multi-region ledger. Explain consistency, duplicate events, and partition recovery."
]
for prompt <- prompts do
case Agent.invoke(agent, %{messages: [Message.user(prompt)]}) do
{:ok, state} ->
IO.inspect(state.model_route, label: "Routing decision")
IO.puts(Message.text(List.last(state.messages)))
{:error, error} ->
IO.inspect(error, label: "Agent failed")
end
end
The router runs in before_agent, classifies the latest user message, and stores the result in state.model_route. Its wrap_model_call hook applies the selected model to the following chat calls. Tool turns and a separate structured-output call keep the route; the middleware does not call Jev again for every tool result.
In our live smoke test, the arithmetic request selected Luna, executed multiply once, and returned 56. The architecture request selected Sol. Those are observed examples, not guaranteed classifications for every wording or future model release.
Make uncertainty and failure behavior explicit
The middleware accepts a choice when its confidence is at least 0.8. Otherwise, it keeps the agent's base model. In this example the base is Luna, so uncertain routing favors lower cost. An application that needs a more capable default can set model: powerful in Agent.build/1.
Classifier errors also retain the base model. Missing or unsupported user input follows the same fallback path. The route record includes accepted, reason, choice, confidence, probabilities, the resolved classifier model, request ID, latency, and usage when available.
The threshold applies to the provider's reported Choice confidence. It is not a promise that accepted requests are classified correctly 80% of the time. Measure the policy on representative requests, including ambiguous cases and misleading instructions. TypeSafe's confidence guidance explains what the statistic measures.
There are three integration details worth planning for:
- Context-dependent follow-ups: the built-in router examines the latest user message. A message such as “fix the issue we discussed” may need a custom classifier input containing selected conversation context.
- Model overrides: a runtime model selector that conflicts with an accepted Jev route returns a configuration error. Put provider-specific settings on the corresponding model values.
- Fallback middleware: place
TypeSafeModelRouterbeforeModelFallbackso a fallback model can replace a failed selected model. Classifier-error fallback and chat-model failure recovery happen at different boundaries.
Routes survive checkpoint resume. A new agent invocation recomputes the route, while a crash before a result is checkpointed can cause inference to repeat under the graph's normal retry behavior.
Inspect decisions and costs in WeaveScope
Configure the exporter in your application's config/runtime.exs:
config :beam_weaver,
weave_scope: [
endpoint: "https://app.weavescope.com",
api_key: System.fetch_env!("WEAVESCOPE_API_KEY")
]
Jev classifications are recorded as model spans. Their payloads include the supplied state and questions, typed answers, token usage, resolved model version, request ID, and timing. Routing calls contribute classifier usage once; they do not add a synthetic assistant message to conversation history.
WeaveScope's Jev support renders Choice, Score, and Noul answers with probability distributions and preserves the raw response. That gives you a useful inspection sequence: read the request, inspect Jev's decision, see which chat model ran, and follow the tool results through to the answer. Treat the state you supply to classification as traceable application data; avoid including secrets or irrelevant private material.
As checked on September 21, 2026, TypeSafe lists Jev 1.13 at $0.042 per million input tokens, with free output tokens. A 1,000-input-token classification therefore costs approximately $0.000042, before any downstream chat-model calls. BeamWeaver reports usage and a profile-based cost estimate, and WeaveScope recognizes the nested cost metadata and managed Jev prices. Check the current provider pricing before estimating production spend.
Evaluate the whole request path. A classifier adds latency and cost of its own; whether routing saves enough depends on which chat calls it changes and whether the resulting answers remain acceptable.
Write prompts you can evaluate
Start with specific questions about supplied evidence. For ticket routing, “which team handles this issue?” is more useful when every team has clear inclusion and exclusion criteria. For model routing, define observable task requirements rather than asking Jev to infer quality from a model's name.
Use a small development set to refine those boundaries, then keep a separate evaluation set untouched. Include direct tasks, difficult tasks, missing context, and text that tries to choose its own route. Track raw-choice errors, fallback frequency, effective-route errors, latency, and total spend. Keep the raw decisions so an apparent saving can be checked against the work the agent actually performed.
Our initial ten-case synthetic evaluation matched the expected route on eight labeled cases; the two ambiguous cases fell below the threshold and used the base model. The labels were assistant-authored, and the sample is too small to establish production accuracy. It was a check of the integration and policy behavior, not a general model benchmark.
Jev also has documented limitations, including numerical precision, indirect instructions, irrelevant context, and adversarial input. Keep exact arithmetic in tools—as we did with multiply—and use Jev for the semantic decision about which path to take.
Run the complete example
From the BeamWeaver repository, with TypeSafe and OpenAI credentials exported:
mix deps.get
mix run examples/typesafe_dynamic_routing.exs
The example uses real Jev and OpenAI calls by default. For an explicit fixture run:
mix run examples/typesafe_dynamic_routing.exs --offline
Opt-in live contract tests and the separate routing evaluation are available with:
mix test test/beam_weaver/typesafe/live_test.exs --only typesafe_live
Start with one bounded decision whose outcome you can inspect. Keep the rubric, raw answers, fallback behavior, and downstream result together. That makes it possible to improve routing based on what happened in your application.