Skip to main content
POST
Submit an LLM Judge evaluation

​
What it does

Submits an answer plus your judge prompt and the sources you want it graded against, and returns an evaluation_id immediately. The judge verifies the sources within the selected source scope, then a deterministic LLM grades the answer on a fixed rubric — Factuality / Completeness / Relevance / Grounding, each 1–5 — and returns a structured verdict (dimension scores plus a short reasoning, with a server-computed overall_score mean). Unlike Execute Agent Judge, which returns a written assessment against primary sources, this endpoint returns numeric scores you can compare across runs. The rubric is the same for every run so scores are comparable across callers and across time. That still takes minutes, not seconds, so this endpoint does not return the verdict. 202 means the submission is recorded and will be evaluated — never that it has been.
status is pending for a new submission, but read it rather than assuming it. An Idempotency-Key retry returns the evaluation that key already names — which may have finished in the meantime — so it can come back completed or failed. The value set is the same one Get LLM Judge Evaluation returns.
Retrieve the result with Get LLM Judge Evaluation.

​
Example

​
The fields

These are the fields this endpoint reads; anything else in the body is rejected.
Built for organization-bound API keys. The organization and user are taken from your key. The evaluation is filed under that organization, and RMS verification uses that user’s document visibility.Platform-wide keys are not a supported configuration for this endpoint — it has not been designed or tested against them, and the behaviour you get is whatever the shared authentication layer does rather than something this endpoint guarantees. If you hold a platform key, talk to your LinqAlpha contact before integrating.
Fixed rubric. Every LLM Judge run is scored on the same four dimensions (Factuality / Completeness / Relevance / Grounding, each 1–5), with a server-computed overall_score (arithmetic mean). Previously supported scoring_rubric and response_schema fields were removed so that scores from any two runs are directly comparable; sending either now returns a 400.
start_time and end_time are inclusive bounds applied to the selected search_type. When search_type is all, the same window applies to both RMS and external search. Omit start_time to search everything up to and including end_time.The offset is required, and that is deliberate. 2026-08-19T14:32:11 without one is ambiguous, and reading it as UTC would move a Seoul timestamp nine hours. The result of that is not an error you would see: it is a plausible verdict judged against the wrong instant. Send Z or your own offset — both name the same instant and both are accepted.start_time must be earlier than or equal to end_time. A future end_time is rejected; a few minutes of clock skew is tolerated.

​
How the judgement is produced

1

Verify references

The verifier runs each reference against Linq’s tool set for the selected search_type, producing a per-reference verdict (supported / contradicted / unresolved) with evidence. When references: [] this step is skipped and no cost is incurred here.
2

Grade with the fixed rubric

A deterministic model (temperature=0, seed=7) reads your prompt, the query, answer, references, and verification verdicts, and returns integer scores for each of the four fixed dimensions plus a short reasoning. The server then computes overall_score as the arithmetic mean of the four scores.
Partial verification is preserved, not hidden. If a verifier chunk failed, only the surviving verdicts reach the judge, and the response reports verified_reference_count less than input_reference_count so you can tell — see Get LLM Judge Evaluation.

​
Idempotency

Send an Idempotency-Key header to make retries safe. Within your organization:
  • Same key, same body → the original evaluation_id, no second judge run.
  • Same key, different body → 409 Conflict.
Use it whenever a network error leaves you unsure whether a submission landed. Without it, a retry starts a second run and you are billed for both.
Idempotency is scoped per endpoint. Using the same key on Execute Agent Judge and this endpoint returns two distinct evaluations — the two endpoints do not share the key space.
A retry still answers 202, but the status it carries is the original evaluation’s current status — not necessarily pending. If that evaluation already finished, you get completed or failed straight from the retry and there is nothing left to poll.

​
Limits

Oversized submissions are rejected with 400 at submission time — nothing is queued and nothing is billed, so a request that is too large costs only the round trip.
Two of these are easy to trip without noticing.The character counts are UTF-16 code units, which is what "…".length returns in JavaScript. Characters outside the Basic Multilingual Plane — emoji, some rarer CJK — count as two. If your text is plain prose the distinction never comes up.The token cap is separate from the byte cap, and applies to the request as a whole. Dense CJK text can pass 200,000 code units and still exceed 100,000 tokens, so a long Korean or Japanese answer may be refused while a longer English one is not.
These may be raised as we see real usage. A raise never breaks a client that was within the old figure, so code against them as minimums. If you are running close to one, tell us rather than splitting a submission.

Authorizations

​
X-API-KEY
string
header
required

Headers

​
Idempotency-Key
string

Makes a retry safe. Same key + same body returns the original evaluation_id; same key + different body is a 409. Scope is per-endpoint — the same key on POST /v2/judge/agent yields a distinct evaluation.

Maximum string length: 255

Body

application/json

Body for POST /v2/judge/llm. Unknown top-level fields are rejected (public-api zod is .strict()). The previously supported scoring_rubric and response_schema fields were removed to keep every run comparable on the fixed default rubric; a caller still sending either now gets a 400. Whole serialized body must be at most 1,000,000 bytes (UTF-8).

​
answer
string
required

The text being judged.

Required string length: 1 - 200000
​
query
string
required

The original user prompt that produced the answer. Lets the judge check whether the answer followed the instructions.

Required string length: 1 - 10000
​
time_window
object
required

Inclusive bounds applied to the selected search_type. For all, the same window applies to both RMS and external search. A bare date, missing timezone offset, or nonexistent instant is rejected.

​
prompt
string

Optional. Delivered verbatim as the judge's system message when supplied. Omit the field to fall back to Linq's default judge prompt. If supplied it must contain non-whitespace text; an empty string is rejected.

Required string length: 1 - 50000
​
references
object[]

Structured references the caller wants verified. At most 50 items. Each element is { title?, content, url?, metadata? }; content is required, while title, an HTTP(S) url, and free-form JSON-object metadata are optional. Bare URL strings and undeclared reference-level fields are not accepted. references: [] is allowed — the verifier is skipped and the judge grades on prompt / query / answer only.

Maximum array length: 50
​
search_type
enum<string>
default:external

Verifier source scope: rms, external, or all. Defaults to external; use all when the answer combines RMS and external sources.

Available options:
rms,
external,
all

Response

Accepted and durably recorded. Not evaluated yet — unless this was an Idempotency-Key retry naming an evaluation that has since finished, in which case status reports that evaluation's current state. Read status; do not assume pending.

​
error
unknown
​
payload
object