Decisions with Jev

Use the standalone decision SDK to route a request, classify a document, or review an answer. It wraps TypeSafe Jev's Choice, Score, and Noul primitives in typed HarnessX requests and results.

Import it from harnessx.decisions. Decisions have their own async provider and usage accounting. Call the provider before selecting an agent or after receiving an answer. The agent loop does not call them on its own.

Live calls need the jev extra and a TypeSafe key. See Installation for credentials and Examples for the runnable commands.

Choose a question type

HarnessX question	Jev primitive	Result	Used in the examples
`ChoiceQuestion`	`Choice`	Selected label, probabilities for every option, and confidence	Agent routing and document classification
`ScoreQuestion`	`Score`	Unrounded expected score, ordered probabilities, rubric levels, and confidence	Answer coverage
`BooleanQuestion`	`Noul`	Probability that the proposition is true	Support from supplied evidence

Choice requires at least two named options with descriptions. Score accepts two to ten ordered descriptions, indexed from zero. A four-level rubric produces a score between 0 and 3, which may be fractional.

BooleanCriteria describes the true and false outcomes, and either description can be omitted. Noul returns a probability with no separate confidence value. Compare answer.probability with your own threshold; bool(answer) raises to prevent implicit conversion.

Evaluate typed questions

This example sends an answer and its evidence in one request. Each question assesses that state independently. It makes a live Jev call when run.

import asyncio
from harnessx.decisions import BooleanCriteria, BooleanQuestion, DecisionRequest
from harnessx.decisions import JevDecisionProvider, ScoreQuestion

async def main():
    request = DecisionRequest(
        check_id="answer_review",
        version="v1",
        state={"answer": "Sales were $100 in January.", "evidence": "January: $100."},
        questions={
            "coverage": ScoreQuestion(
                instructions="How completely does the answer address the query?",
                levels=(
                    "Does not answer the request.",
                    "Addresses most of the request, with minor omissions.",
                    "Fully addresses every part of the request.",
                ),
            ),
            "supported": BooleanQuestion(
                instructions="Does the evidence support every factual claim?",
                criteria=BooleanCriteria(true="Every claim follows from the evidence."),
            ),
        },
    )
    async with JevDecisionProvider() as decisions:
        batch = await decisions.evaluate(request)
    print(batch.scores["coverage"].score, batch.booleans["supported"].probability)

asyncio.run(main())

The adapter translates those questions to the official SDK's Choice, Score, and Noul, then calls AsyncTypeSafeClient.system_one(). The result keeps the resolved model and the service request ID. The default endpoint is the direct TypeSafe API.

Route to one agent

The routing example source uses Choice to select from an application-defined mapping of agent factories:

Decision	Application behavior
`sql`, `research`, or `general`, with sufficient confidence	Construct and run that one agent
`unknown` or an unexpected label	Use the general agent
Missing confidence or confidence below `min_confidence`	Use the general agent
Unavailable decision	Use the general agent
Selected agent fails	Return its failed result; do not start a second agent

min_confidence is required and must be a finite number between zero and one. The example's 0.8 is illustrative, not a calibrated default. The model selects a label; application code owns the factories. The example's specialists draft answers without a database or live sources.

Run it offline with uv run python examples/07-quality/decisions_routing.py --min-confidence 0.8.

Classify documents and review answers

The routing example classifies a document before it routes. It asks a Choice question with five labels: W2, Deposit, Insurance, Payroll, and unknown. The offline fixture always returns W2; pass --live --text ... with --min-confidence 0.8 and --model to assess your own document text.

The answer-review example source makes a separate decision request after a successful Agent.run() result:

Score evaluates coverage against four ordered levels, from unanswered to fully addressed.
Noul evaluates whether every factual claim follows from the supplied evidence, using explicit true and false criteria.

The review skips unsuccessful runs and leaves the original run unchanged. It reports probabilities and scores, not an automatic pass/fail gate.

Handle results and failures

DecisionBatch.status is either completed or unavailable. A completed batch requires every requested answer to validate. An uncertain but valid answer is still completed; confidence thresholds belong to the application.

Missing or incompatible answers, mismatched option labels or rubric levels, nonfinite probabilities, invalid distributions, and inconsistent scores produce an unavailable batch with empty answer maps. Probability distributions are never silently renormalized. Extra answer IDs are ignored.

Condition	Outcome
Missing optional dependency	`ImportError` with installation guidance
Invalid local configuration, question, or oversized state	Local exception before I/O
Authentication, authorization, or request rejection	Unavailable, without retry
Connection failure, transport timeout, HTTP 429 or 500/502/503/504	Bounded retry, then unavailable if unresolved
Invalid or missing requested answers	Unavailable, without retry
Caller cancellation	`asyncio.CancelledError` propagates
Programming error	Exception propagates

DecisionError exposes a stable code, sanitized message, optional HTTP status, and whether the cause is retryable. Codes are authentication, forbidden, invalid_request, rate_limited, timeout, connection, server, invalid_response, and unexpected_http. Raw server bodies are excluded, though the upstream SDK's own debug logging can include them.

Configure limits and ownership

Provider option	Default	Meaning
`model`	`jev-1.13.0`	Explicit model sent on every request
`timeout_seconds`	`2.0`	Total evaluation deadline, including retry delays
`max_retries`	`1`	At most two HTTP attempts by default
`max_state_bytes`	`16384`	Limit on compact UTF-8 JSON state
`api_key`	`TYPESAFE_API_KEY`	Credential used when constructing a client
`base_url`	`https://api.typesafe.ai`	Explicit API root; does not inherit `TYPESAFE_BASE_URL`
`client`	`None`	Optional borrowed `AsyncTypeSafeClient`

evaluate(request, timeout_seconds=..., max_attempts=...) shortens the deadline and caps attempts for one call. SDK retries are disabled so they cannot multiply the adapter's retry budget. A Retry-After delay is honored when it fits the remaining deadline; otherwise the evaluation returns unavailable.

Use async with JevDecisionProvider(...) to close resources reliably. The adapter owns and closes the clients it creates. Injected clients remain caller-owned, and per-call overrides do not change their defaults; combining client with api_key or base_url raises.

The provider supports concurrent evaluations. aclose() is idempotent, cancels outstanding evaluations, and closes owned resources. Evaluating after close raises RuntimeError.

Preserve state, usage, and results

State

Request state must be text, a JSON object, or a JSON array holding only finite JSON values and string keys. State is copied at construction and again before dispatch. Arbitrary objects are never converted with str().

Serialization

Requests and results expose to_dict() and from_dict() with schema_version=1. Decoding validates the data and rejects unknown schema versions and question or answer kinds, while accepting additive metadata fields. Use those codecs before json.dumps.

Usage

A batch preserves check_id, version, requested and resolved model, request ID, elapsed time, and attempt count. Token counts describe the successful response only. After a retry, earlier attempts may have consumed unreported tokens, so usage.incomplete stays true. Absent measurements stay None, cost included. Decision usage is separate from agent model usage.