{
"error": {
"code": "INVALID_REQUEST_BODY",
"msg": "evaluation_id must be a valid UUID",
"message": "evaluation_id must be a valid UUID"
},
"payload": null
}
{
"error": {
"code": "UNAUTHORIZED",
"msg": "Invalid API key",
"message": "Invalid API key"
},
"payload": null
}
{
"error": {
"code": "JUDGE_EVALUATION_NOT_FOUND",
"msg": "Evaluation not found",
"message": "Evaluation not found"
},
"payload": null
}
Returns HTTP 200 in every state, including while still running. Branch on status, not on the status code. The response shape is identical across statuses — the same seven fields are always present.
GET
/
v2
/
judge
/
llm
/
{evaluation_id}
Retrieve an LLM Judge evaluation
curl --request GET \ --url https://api.linqalpha.com/v2/judge/llm/{evaluation_id} \ --header 'X-API-KEY: <api-key>'
{
"error": {
"code": "INVALID_REQUEST_BODY",
"msg": "evaluation_id must be a valid UUID",
"message": "evaluation_id must be a valid UUID"
},
"payload": null
}
{
"error": {
"code": "UNAUTHORIZED",
"msg": "Invalid API key",
"message": "Invalid API key"
},
"payload": null
}
{
"error": {
"code": "JUDGE_EVALUATION_NOT_FOUND",
"msg": "Evaluation not found",
"message": "Evaluation not found"
},
"payload": null
}
Returns the current state of one LLM Judge run, using the evaluation_id from
Execute LLM Judge. The response shape is identical in
every state — the same six fields are always present — so a client reads status and never has
to branch on the body’s shape.
This returns HTTP 200 while the judge is still running. Branch on the status field, not on the
status code.
Same shape as completed, but scores / overall_score are null and reasoning carries a customer-safe sentence
null
null
Yes
A GET immediately after a POST returns pending. That is expected, not an error.
Failure detection: verdict !== null && verdict.overall_score === null is the failure
marker. reasoning on such a verdict is the customer-safe sentence that used to be
surfaced as a top-level reason field — that field was removed to keep the shape uniform.
Unlike Get Agent Judge Evaluation, this endpoint does not
currently return excluded. The LLM Judge has no exclusion pre-filter — the same status enum is
exposed for consistency, but only pending / completed / failed are produced today.
Poll no more than once every 10 seconds. Typical runs settle in a few minutes; a sensible
client gives up after around 30 minutes and treats the run as failed.
import timeimport requestsurl = f"https://api.linqalpha.com/v2/judge/llm/{evaluation_id}"headers = {"X-API-KEY": "<your-api-key>"}deadline = time.time() + 30 * 60 # give up after ~30 minuteswhile time.time() < deadline: payload = requests.get(url, headers=headers).json()["payload"] if payload["status"] == "completed": print(payload["verdict"]) # Coverage signal — see the note below. print(f"verified {payload['verified_reference_count']}/{payload['input_reference_count']}") break if payload["status"] == "failed": # `verdict.overall_score is None` is the failure marker; `reasoning` is the # customer-safe reason that used to live under a separate `reason` field. print("failed:", payload["verdict"]["reasoning"]) # safe to retry break time.sleep(10) # still pending
verdict uses one shape on every non-null run — four dimensions on a 1–5 scale, plus reasoning
and a server-computed overall_score. completed populates all three; failed uses the same
shape with scores / overall_score set to null and the customer-safe reason on reasoning.
Every successful LLM Judge run therefore returns scores directly comparable across callers and
time, and a caller reads one field to tell success from failure.
Non-null on completedandfailed. See the two response examples above.
judge_model
string | null
The model that produced the verdict. Non-null only on completed.
input_reference_count
int | null
How many references you submitted. Non-null only on completed.
verified_reference_count
int | null
How many references the verifier actually produced a verdict for. Non-null only on completed. See below.
verified_reference_count vs input_reference_count — read them together
On a clean run the two are equal.When they differ (verified_reference_count < input_reference_count), a chunk of the verifier
failed and only the surviving verdicts reached the judge — the verdict is still valid,
but partly graded on incomplete verification. Downstream you may want to weigh those
runs differently or resubmit.On non-completed statuses both are null.
Empty references — both counts are 0
Submitting references: [] is a valid request; the verifier is skipped and the judge grades
on prompt / query / answer only. In this case input_reference_count and
verified_reference_count are both 0, not null.
Per-reference verification verdicts, the evidence behind them, and internal cost/latency accounting
are used to produce the verdict but are not part of this response.
Judge runs are scoped to the organization that submitted them. An evaluation_id belonging to
another organization returns 404, exactly as an id that does not exist — the two are
indistinguishable by design.The two endpoints — this one and Get Agent Judge Evaluation —
read from disjoint id spaces. An id from POST /v2/judge/agent returns 404 here, and vice versa.
Idempotency keys are scoped the same way, so the same key on the two endpoints yields two separate
evaluations.A malformed evaluation_id is rejected with 400 before any lookup.
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"Linq Alpha API Docs","colors":{"primary":"#1767FE","light":"#1767FE","dark":"#1767FE"},"logo":{"light":"https://mintcdn.com/linqalpha/9BNJqkVKHb4kSYyJ/logo/light.svg?fit=max\u0026auto=format\u0026n=9BNJqkVKHb4kSYyJ\u0026q=85\u0026s=d7dd093386c0527ef6e6c9ef5f8130ad","dark":"https://mintcdn.com/linqalpha/9BNJqkVKHb4kSYyJ/logo/dark.svg?fit=max\u0026auto=format\u0026n=9BNJqkVKHb4kSYyJ\u0026q=85\u0026s=03c8b4a884d6b9c2deb87606bd15f0c7"},"favicon":"https://mintcdn.com/linqalpha/9BNJqkVKHb4kSYyJ/favicon.svg?fit=max\u0026auto=format\u0026n=9BNJqkVKHb4kSYyJ\u0026q=85\u0026s=384de566c2f8fe8b6d627bd19c014f2c","navbar":{"links":[{"href":"mailto:support@linqalpha.com","label":"Support"}]},"footer":{"socials":{"github":"https://github.com/linq-rag","linkedin":"https://www.linkedin.com/company/linqalpha/posts?feedView=all"},"links":[{"header":"Legal","items":[{"label":"AI Disclaimer","href":"https://linqalpha.com/disclaimer"},{"label":"Terms of Use","href":"https://linqalpha.com/terms-of-use"},{"label":"Privacy Policy","href":"https://linqalpha.com/privacy-policy"}]}]},"styling":{"latex":false},"navigation":{"global":{"anchors":[{"anchor":"Website","href":"https://linqalpha.com"}]},"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"linqalpha","hasAudioNarration":false,"actualSubdomain":"linqalpha","customDomains":["docs.linqalpha.com"],"gitSource":{"type":"github","owner":"linq-rag","repo":"linq-public-api","deployBranch":"main","contentDirectory":"docs","isPrivate":true,"mountPath":"","label":""},"chroma":{"liveDeploymentHistoryId":"6adbbd2a01ae999ce782ca39","collectionId":"00c88381-983e-45c7-9553-efed77ef87e8","schemaVersion":"v3","latestBuiltDeploymentHistoryId":"6adbbd2a01ae999ce782ca39"},"feedback":{"thumbs":true,"issues":false,"edits":false},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6adbbd2a01ae999ce782ca39","clientVersion":"0.0.3971","searchFilterCounts":[{"language":"default","tag":"API Reference","count":53},{"language":"default","tag":"Basic","count":16},{"language":"default","tag":"Briefing","count":9},{"language":"default","tag":"RMS (On request)","count":7},{"language":"default","tag":"Analytics V2","count":6},{"language":"default","tag":"Evaluations","count":4},{"language":"default","tag":"Transcriptions","count":4},{"language":"default","tag":"Meeting Agent","count":3},{"language":"default","tag":"Sync Status","count":3},{"language":"default","tag":"MCP","count":1}]},"assistantConfig":{},"basePath":"","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-v7-file-tree","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"pause-universal-search-reconcile","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"per-site-search-delta","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-private-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-nav-generations","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"byok-sep-25","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"vite-origin","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-agent-model-picker","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-updated-9-15","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"isAtlas":false,"googleFontsInlined":false},"customCss":[],"page":{"pagePath":"api-reference/evaluations/get_llm","slug":"/api-reference/evaluations/get_llm","pageMetadata":{"title":"Get LLM Judge Evaluation","openapi":"GET /v2/judge/llm/{evaluation_id}","description":"Returns HTTP 200 in every state, including while still running. Branch on `status`, not on the status code. The response shape is identical across statuses — the same seven fields are always present.","href":"/api-reference/evaluations/get_llm"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx, jsxs: _jsxs} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n code: \"code\",\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsxs(_components.p, {\n children: [\"Returns HTTP 200 in every state, including while still running. Branch on \", _jsx(_components.code, {\n children: \"status\"\n }), \", not on the status code. The response shape is identical across statuses — the same seven fields are always present.\"]\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,..._provideComponents(),...props.components},{Accordion,AccordionGroup,CodeBlock,Heading,Note,Table,Warning}=_components;if(!Accordion)_missingMdxReference(`Accordion`,true);if(!AccordionGroup)_missingMdxReference(`AccordionGroup`,true);if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);if(!Table)_missingMdxReference(`Table`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsx(Heading,{level:`2`,id:`what-it-does`,children:`What it does`}),`\n`,_jsxs(_components.p,{children:[`Returns the current state of one LLM Judge run, using the `,_jsx(_components.code,{children:`evaluation_id`}),` from\n`,_jsx(_components.a,{href:`/api-reference/evaluations/create_llm`,children:`Execute LLM Judge`}),`. The response shape is `,_jsx(_components.strong,{children:`identical in\nevery state`}),` — the same six fields are always present — so a client reads `,_jsx(_components.code,{children:`status`}),` and never has\nto branch on the body’s shape.`]}),`\n`,_jsx(Warning,{children:_jsxs(_components.p,{children:[`This returns HTTP `,_jsx(_components.code,{children:`200`}),` while the judge is still running. Branch on the `,_jsx(_components.code,{children:`status`}),` field, not on the\nstatus code.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`statuses`,children:`Statuses`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:_jsx(_components.code,{children:`status`})}),_jsx(_components.th,{children:`Meaning`}),_jsx(_components.th,{children:_jsx(_components.code,{children:`verdict`})}),_jsx(_components.th,{children:_jsx(_components.code,{children:`judge_model`})}),_jsxs(_components.th,{children:[_jsx(_components.code,{children:`input_reference_count`}),` / `,_jsx(_components.code,{children:`verified_reference_count`})]}),_jsx(_components.th,{children:`Retry?`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`pending`})}),_jsx(_components.td,{children:`Queued or running.`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`null`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`null`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`null`})}),_jsx(_components.td,{children:`Keep polling`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`completed`})}),_jsx(_components.td,{children:`Finished.`}),_jsx(_components.td,{children:`Fixed-rubric verdict — see below`}),_jsx(_components.td,{children:`The model that produced it`}),_jsx(_components.td,{children:`Both non-null`}),_jsx(_components.td,{children:`No`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`failed`})}),_jsx(_components.td,{children:`Something broke on our side.`}),_jsxs(_components.td,{children:[`Same shape as `,_jsx(_components.code,{children:`completed`}),`, but `,_jsx(_components.code,{children:`scores`}),` / `,_jsx(_components.code,{children:`overall_score`}),` are `,_jsx(_components.code,{children:`null`}),` and `,_jsx(_components.code,{children:`reasoning`}),` carries a customer-safe sentence`]}),_jsx(_components.td,{children:_jsx(_components.code,{children:`null`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`null`})}),_jsx(_components.td,{children:`Yes`})]})]})]}),`\n`,_jsxs(_components.p,{children:[`A `,_jsx(_components.code,{children:`GET`}),` immediately after a `,_jsx(_components.code,{children:`POST`}),` returns `,_jsx(_components.code,{children:`pending`}),`. That is expected, not an error.`]}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[`Failure detection: `,_jsx(_components.code,{children:`verdict !== null \u0026\u0026 verdict.overall_score === null`}),` is the failure\nmarker. `,_jsx(_components.code,{children:`reasoning`}),` on such a verdict is the customer-safe sentence that used to be\nsurfaced as a top-level `,_jsx(_components.code,{children:`reason`}),` field — that field was removed to keep the shape uniform.`]})}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[`Unlike `,_jsx(_components.a,{href:`/api-reference/evaluations/get`,children:`Get Agent Judge Evaluation`}),`, this endpoint does `,_jsx(_components.strong,{children:`not`}),`\ncurrently return `,_jsx(_components.code,{children:`excluded`}),`. The LLM Judge has no exclusion pre-filter — the same status enum is\nexposed for consistency, but only `,_jsx(_components.code,{children:`pending`}),` / `,_jsx(_components.code,{children:`completed`}),` / `,_jsx(_components.code,{children:`failed`}),` are produced today.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`polling`,children:`Polling`}),`\n`,_jsxs(_components.p,{children:[`Poll no more than `,_jsx(_components.strong,{children:`once every 10 seconds`}),`. Typical runs settle in a few minutes; a sensible\nclient gives up after around 30 minutes and treats the run as failed.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`22`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`22`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` time`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` requests`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`url `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"https://api.linqalpha.com/v2/judge/llm/`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`evaluation_id`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`headers `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"X-API-KEY\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"\u003cyour-api-key>\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`deadline `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` time.time() `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`+`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:` 30`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` *`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:` 60`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` # give up after ~30 minutes`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`while`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` time.time() `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`\u003c`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` deadline:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` payload `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` requests.get(url, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`headers`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`headers).json()[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"payload\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` payload[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"status\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"completed\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(payload[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"verdict\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`])`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` # Coverage signal — see the note below.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"verified `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`payload[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'verified_reference_count'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`/`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`payload[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'input_reference_count'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` break`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` payload[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"status\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"failed\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:\" # `verdict.overall_score is None` is the failure marker; `reasoning` is the\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:\" # customer-safe reason that used to live under a separate `reason` field.\"})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"failed:\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, payload[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"verdict\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`][`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"reasoning\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]) `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# safe to retry`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` break`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` time.sleep(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`10`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# still pending`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`response`,children:`Response`}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`verdict`}),` uses one shape on every non-null run — four dimensions on a 1–5 scale, plus `,_jsx(_components.code,{children:`reasoning`}),`\nand a server-computed `,_jsx(_components.code,{children:`overall_score`}),`. `,_jsx(_components.code,{children:`completed`}),` populates all three; `,_jsx(_components.code,{children:`failed`}),` uses the same\nshape with `,_jsx(_components.code,{children:`scores`}),` / `,_jsx(_components.code,{children:`overall_score`}),` set to `,_jsx(_components.code,{children:`null`}),` and the customer-safe reason on `,_jsx(_components.code,{children:`reasoning`}),`.\nEvery successful LLM Judge run therefore returns scores directly comparable across callers and\ntime, and a caller reads one field to tell success from failure.`]}),`\n`,_jsx(Heading,{level:`3`,id:`completed`,children:_jsx(_components.code,{children:`completed`})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`20`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`20`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"error\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"payload\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"evaluation_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"2fde560a-e7eb-45e0-8cd3-04e57c50d1d3\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"status\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"completed\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"verdict\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"scores\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"Factuality\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`4`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"Completeness\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`4`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"Relevance\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`5`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"Grounding\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`5`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"reasoning\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"The answer's core claims align with the supplied sources...\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"overall_score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`4.5`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"judge_model\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"gpt-5.4-mini\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"input_reference_count\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`2`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"verified_reference_count\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`2`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`failed`,children:_jsx(_components.code,{children:`failed`})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`15`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`15`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"error\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"payload\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"evaluation_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"2fde560a-e7eb-45e0-8cd3-04e57c50d1d3\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"status\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"failed\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"verdict\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"scores\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"reasoning\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"The evaluation could not be completed.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"overall_score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"judge_model\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"input_reference_count\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"verified_reference_count\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`the-response-fields`,children:`The response fields`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Field`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Notes`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`evaluation_id`})}),_jsx(_components.td,{children:`UUID`}),_jsxs(_components.td,{children:[`The id from `,_jsx(_components.code,{children:`POST /v2/judge/llm`}),`.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`status`})}),_jsx(_components.td,{children:`enum`}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`pending`}),` / `,_jsx(_components.code,{children:`completed`}),` / `,_jsx(_components.code,{children:`failed`}),`.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`verdict`})}),_jsx(_components.td,{children:`object | null`}),_jsxs(_components.td,{children:[`Non-null on `,_jsx(_components.code,{children:`completed`}),` `,_jsx(_components.strong,{children:`and`}),` `,_jsx(_components.code,{children:`failed`}),`. See the two response examples above.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`judge_model`})}),_jsx(_components.td,{children:`string | null`}),_jsxs(_components.td,{children:[`The model that produced the verdict. Non-null only on `,_jsx(_components.code,{children:`completed`}),`.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`input_reference_count`})}),_jsx(_components.td,{children:`int | null`}),_jsxs(_components.td,{children:[`How many references you submitted. Non-null only on `,_jsx(_components.code,{children:`completed`}),`.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`verified_reference_count`})}),_jsx(_components.td,{children:`int | null`}),_jsxs(_components.td,{children:[`How many references the verifier actually produced a verdict for. Non-null only on `,_jsx(_components.code,{children:`completed`}),`. See below.`]})]})]})]}),`\n`,_jsxs(AccordionGroup,{children:[_jsxs(Accordion,{title:`verified_reference_count vs input_reference_count — read them together`,id:`verified-reference-count-vs-input-reference-count-read-them-together`,children:[_jsx(_components.p,{children:`On a clean run the two are equal.`}),_jsxs(_components.p,{children:[`When they differ (`,_jsx(_components.code,{children:`verified_reference_count \u003c input_reference_count`}),`), a chunk of the verifier\nfailed and only the surviving verdicts reached the judge — the verdict is still valid,\nbut partly graded on incomplete verification. Downstream you may want to weigh those\nruns differently or resubmit.`]}),_jsxs(_components.p,{children:[`On non-`,_jsx(_components.code,{children:`completed`}),` statuses both are `,_jsx(_components.code,{children:`null`}),`.`]})]}),_jsx(Accordion,{title:`Empty references — both counts are 0`,id:`empty-references-both-counts-are-0`,children:_jsxs(_components.p,{children:[`Submitting `,_jsx(_components.code,{children:`references: []`}),` is a valid request; the verifier is skipped and the judge grades\non `,_jsx(_components.code,{children:`prompt`}),` / `,_jsx(_components.code,{children:`query`}),` / `,_jsx(_components.code,{children:`answer`}),` only. In this case `,_jsx(_components.code,{children:`input_reference_count`}),` and\n`,_jsx(_components.code,{children:`verified_reference_count`}),` are both `,_jsx(_components.code,{children:`0`}),`, not `,_jsx(_components.code,{children:`null`}),`.`]})})]}),`\n`,_jsx(Note,{children:_jsx(_components.p,{children:`Per-reference verification verdicts, the evidence behind them, and internal cost/latency accounting\nare used to produce the verdict but are not part of this response.`})}),`\n`,_jsx(Heading,{level:`2`,id:`isolation`,children:`Isolation`}),`\n`,_jsxs(_components.p,{children:[`Judge runs are scoped to the organization that submitted them. An `,_jsx(_components.code,{children:`evaluation_id`}),` belonging to\nanother organization returns `,_jsx(_components.code,{children:`404`}),`, exactly as an id that does not exist — the two are\nindistinguishable by design.`]}),`\n`,_jsxs(_components.p,{children:[`The two endpoints — this one and `,_jsx(_components.a,{href:`/api-reference/evaluations/get`,children:`Get Agent Judge Evaluation`}),` —\nread from `,_jsx(_components.strong,{children:`disjoint`}),` id spaces. An id from `,_jsx(_components.code,{children:`POST /v2/judge/agent`}),` returns `,_jsx(_components.code,{children:`404`}),` here, and vice versa.\nIdempotency keys are scoped the same way, so the same key on the two endpoints yields two separate\nevaluations.`]}),`\n`,_jsxs(_components.p,{children:[`A malformed `,_jsx(_components.code,{children:`evaluation_id`}),` is rejected with `,_jsx(_components.code,{children:`400`}),` before any lookup.`]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Get LLM Judge Evaluation","openapi":"GET /v2/judge/llm/{evaluation_id}","description":"Returns HTTP 200 in every state, including while still running. Branch on `status`, not on the status code. The response shape is identical across statuses — the same seven fields are always present.","href":"/api-reference/evaluations/get_llm"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"What it does","slug":"what-it-does","depth":2,"children":[]},{"title":"Statuses","slug":"statuses","depth":2,"children":[]},{"title":"Polling","slug":"polling","depth":2,"children":[]},{"title":"Response","slug":"response","depth":2,"children":[{"title":"completed","slug":"completed","depth":3,"children":[]},{"title":"failed","slug":"failed","depth":3,"children":[]}]},{"title":"The response fields","slug":"the-response-fields","depth":2,"children":[]},{"title":"Isolation","slug":"isolation","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":[],"openApiReferenceData":{"operation":{"tags":["Evaluations"],"summary":"Retrieve an LLM Judge evaluation","description":"Returns HTTP 200 in every state, including while still running. Branch on `status`, not on the status code. The response shape is identical across statuses — the same seven fields are always present.","parameters":["ce58edb8-7efa-4415-88ca-ab74001e97d8"],"responses":{"200":"7caf5066-0348-45c8-9031-3abe546411d5","400":"8903bb22-9485-4c04-9917-1948c4da50d3","401":"88b683b0-2db1-4289-b74f-b99f18985883","404":"ec7acf8d-2aaf-48f0-83a2-6de1d9915cb5","429":"80081f2d-a913-41c0-b700-a0c0b1bf4a23","502":"3bd2e506-3a83-4b6d-89bc-733542bc87ad","504":"d1f6931e-5de2-4d6f-9eef-3d3dbef06e5c"},"method":"get","type":"path","servers":["9cdf8e58-8f32-4db6-a144-4d35b9596918"],"security":["8aa7599b-2dcb-4258-a16e-03ea68ad1736"],"path":"/v2/judge/llm/{evaluation_id}","baseUrlOptions":["https://api.linqalpha.com"],"title":"Retrieve an LLM Judge evaluation","codeSamples":[],"requiredOnlyExamples":false,"requestExampleType":"generated","responseExampleType":"generated"},"dependencies":{"parameters":{"ce58edb8-7efa-4415-88ca-ab74001e97d8":{"name":"evaluation_id","in":"path","required":true,"description":"The id returned by `POST /v2/judge/llm`. Ids from `POST /v2/judge/agent` return `404` here — the two endpoints read from disjoint id spaces.","schema":{"type":"string","format":"uuid","isRequired":true,"description":"The id returned by `POST /v2/judge/llm`. Ids from `POST /v2/judge/agent` return `404` here — the two endpoints read from disjoint id spaces.","uniqueKey":"","typeLabel":"string\u003cuuid>"},"example":"2fde560a-e7eb-45e0-8cd3-04e57c50d1d3"}},"responses":{"7caf5066-0348-45c8-9031-3abe546411d5":{"description":"The evaluation's current state. Same shape in every status.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"oneOf":[{"example":null},{"example":null,"type":"null","uniqueKey":"error.oneOf.1","typeLabel":"null"}],"uniqueKey":"error","typeLabel":"null"},"payload":{"type":"object","required":["evaluation_id","status","verdict","judge_model","input_reference_count","verified_reference_count"],"properties":{"evaluation_id":{"type":"string","format":"uuid","isRequired":true,"uniqueKey":"payload.evaluation_id","typeLabel":"string\u003cuuid>"},"status":{"type":"string","enum":["pending","completed","excluded","failed"],"description":"`pending` means queued or running. The LLM judge does not currently produce `excluded`; the enum is shared with `POST /v2/judge/agent` for consistency.","isRequired":true,"uniqueKey":"payload.status","typeLabel":"enum\u003cstring>"},"verdict":{"oneOf":[{"type":"object","description":"The structured verdict. Non-null on both `completed` and `failed`. `completed` populates `scores` / `overall_score`; `failed` sets both to `null` and carries the customer-safe reason on `reasoning`. `pending` and `excluded` return `null`.","required":["scores","reasoning","overall_score"],"properties":{"scores":{"oneOf":[{"type":"object","properties":{"Factuality":{"type":"integer","minimum":1,"maximum":5,"uniqueKey":"payload.verdict.oneOf.0.scores.oneOf.0.Factuality","typeLabel":"integer"},"Completeness":{"type":"integer","minimum":1,"maximum":5,"uniqueKey":"payload.verdict.oneOf.0.scores.oneOf.0.Completeness","typeLabel":"integer"},"Relevance":{"type":"integer","minimum":1,"maximum":5,"uniqueKey":"payload.verdict.oneOf.0.scores.oneOf.0.Relevance","typeLabel":"integer"},"Grounding":{"type":"integer","minimum":1,"maximum":5,"uniqueKey":"payload.verdict.oneOf.0.scores.oneOf.0.Grounding","typeLabel":"integer"}},"additionalProperties":false,"description":"Per-dimension integer scores 1-5. `null` on `failed`.","isRequired":true,"uniqueKey":"payload.verdict.oneOf.0.scores.oneOf.0","typeLabel":"object"},{"type":"null","properties":{"Factuality":{"type":"integer","minimum":1,"maximum":5},"Completeness":{"type":"integer","minimum":1,"maximum":5},"Relevance":{"type":"integer","minimum":1,"maximum":5},"Grounding":{"type":"integer","minimum":1,"maximum":5}},"additionalProperties":false,"description":"Per-dimension integer scores 1-5. `null` on `failed`.","isRequired":true,"uniqueKey":"payload.verdict.oneOf.0.scores.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload.verdict.oneOf.0.scores","typeLabel":"object | null"},"reasoning":{"type":"string","description":"Short justification for the dimension scores on `completed`; on `failed` this is the customer-safe reason the evaluation did not complete.","isRequired":true,"uniqueKey":"payload.verdict.oneOf.0.reasoning","typeLabel":"string"},"overall_score":{"oneOf":[{"type":"number","description":"Arithmetic mean of the dimension scores. Computed server-side on `completed`; `null` on `failed` — a caller can treat `overall_score === null` as the failure marker.","isRequired":true,"uniqueKey":"payload.verdict.oneOf.0.overall_score.oneOf.0","typeLabel":"number"},{"type":"null","description":"Arithmetic mean of the dimension scores. Computed server-side on `completed`; `null` on `failed` — a caller can treat `overall_score === null` as the failure marker.","isRequired":true,"uniqueKey":"payload.verdict.oneOf.0.overall_score.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload.verdict.oneOf.0.overall_score","typeLabel":"number | null"}},"isRequired":true,"uniqueKey":"payload.verdict.oneOf.0","typeLabel":"object"},{"type":"null","isRequired":true,"uniqueKey":"payload.verdict.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload.verdict","typeLabel":"object | null"},"judge_model":{"oneOf":[{"type":"string","description":"The model that produced the verdict. Non-null only on `completed`.","isRequired":true,"uniqueKey":"payload.judge_model.oneOf.0","typeLabel":"string"},{"type":"null","description":"The model that produced the verdict. Non-null only on `completed`.","isRequired":true,"uniqueKey":"payload.judge_model.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload.judge_model","typeLabel":"string | null"},"input_reference_count":{"oneOf":[{"type":"integer","description":"How many references the request submitted. Non-null only on `completed`. `0` when `references: []` was sent.","isRequired":true,"uniqueKey":"payload.input_reference_count.oneOf.0","typeLabel":"integer"},{"type":"null","description":"How many references the request submitted. Non-null only on `completed`. `0` when `references: []` was sent.","isRequired":true,"uniqueKey":"payload.input_reference_count.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload.input_reference_count","typeLabel":"integer | null"},"verified_reference_count":{"oneOf":[{"type":"integer","description":"How many references the verifier actually produced a verdict for. Equal to `input_reference_count` on a clean run; strictly less when a verifier chunk failed and only the surviving verdicts reached the judge. Non-null only on `completed`.","isRequired":true,"uniqueKey":"payload.verified_reference_count.oneOf.0","typeLabel":"integer"},{"type":"null","description":"How many references the verifier actually produced a verdict for. Equal to `input_reference_count` on a clean run; strictly less when a verifier chunk failed and only the surviving verdicts reached the judge. Non-null only on `completed`.","isRequired":true,"uniqueKey":"payload.verified_reference_count.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload.verified_reference_count","typeLabel":"integer | null"}},"uniqueKey":"payload","typeLabel":"object"}},"uniqueKey":"","typeLabel":"object"},"example":{"error":null,"payload":{"evaluation_id":"2fde560a-e7eb-45e0-8cd3-04e57c50d1d3","status":"completed","verdict":{"scores":{"Factuality":4,"Completeness":4,"Relevance":5,"Grounding":5},"reasoning":"The answer's core claims align with the supplied sources...","overall_score":4.5},"judge_model":"gpt-5.4-mini","input_reference_count":2,"verified_reference_count":2}},"examples":{"Example":{"title":"Example","value":{"error":null,"payload":{"evaluation_id":"2fde560a-e7eb-45e0-8cd3-04e57c50d1d3","status":"completed","verdict":{"scores":{"Factuality":4,"Completeness":4,"Relevance":5,"Grounding":5},"reasoning":"The answer's core claims align with the supplied sources...","overall_score":4.5},"judge_model":"gpt-5.4-mini","input_reference_count":2,"verified_reference_count":2}}}},"exampleType":"examples"}},"exampleType":"examples"},"8903bb22-9485-4c04-9917-1948c4da50d3":{"description":"Malformed `evaluation_id`.","content":{"application/json":{"schema":{"type":"object","description":"Standard error response wrapper","properties":{"error":{"type":"object","properties":{"code":{"type":"string","description":"Error code indicating the type of error. Common codes:\n- Authentication: `API_KEY_MISSING`, `INVALID_API_KEY`\n- Validation: `INVALID_REQUEST_BODY`, `ORGANIZATION_ID_MISSING`, `TICKERS_MISSING`, `CHAT_MESSAGE_ID_MISSING`, `DOCUMENT_ID_MISSING`\n- Service: `SEARCH_FAIL`, `TTS_FAIL`, `CREATE_CONV_FAIL`, `GET_STOCK_FAIL`, `CREATE_MSG_FAIL`, `ALPHA_COMP_FAIL`, `GET_REF_FAIL`\n- Connectors: `CONNECTOR_LIST_FAIL`, `CONNECTOR_NOT_FOUND`, `CONNECTOR_CREATE_FAIL`, `CONNECTOR_UPDATE_FAIL`, `CONNECTOR_DELETE_FAIL`, `CONNECTOR_TEST_FAIL`\n- Not Found: `NOT_FOUND`\n- Meeting cancellation: `MEETING_NOT_FOUND`, `MEETING_NOT_CANCELABLE`, `MEETING_CANCEL_REASON_REQUIRED`, `MEETING_CANCEL_UNAVAILABLE`, `MEETING_CANCEL_FAIL`\n- Meeting compliance review: `MEETING_NOT_FOUND`, `MEETING_REVIEW_NOT_PERMITTED`, `MEETING_NOT_REVIEWABLE`, `MEETING_REVIEW_UNAVAILABLE`, `MEETING_REVIEW_FAIL`\n- Transcriptions: `TRANSCRIPTION_UPLOAD_FAIL`, `TRANSCRIPTION_SUBMIT_FAIL`, `TRANSCRIPTION_GET_FAIL`, `TRANSCRIPTION_NOT_FOUND`, `TRANSCRIPTION_REQUEST_INVALID`, `TRANSCRIPTION_RESULT_UNAVAILABLE` (503, poll again), `TRANSCRIPTION_SUBMIT_UNAVAILABLE` (503, submit again with the same Idempotency-Key)","example":"INVALID_REQUEST_BODY","isRequired":true,"uniqueKey":"error.code","typeLabel":"string"},"msg":{"type":"string","description":"Error message (deprecated, use `message` instead)","example":"query is required and must be a string","uniqueKey":"error.msg","typeLabel":"string"},"message":{"type":"string","description":"Error message providing more details about the error","example":"query is required and must be a string","uniqueKey":"error.message","typeLabel":"string"}},"required":["code"],"isRequired":true,"uniqueKey":"error","typeLabel":"object"},"payload":{"oneOf":[{"description":"Always null for error responses","isRequired":true,"uniqueKey":"payload.oneOf.0","typeLabel":"any"},{"description":"Always null for error responses","type":"null","isRequired":true,"uniqueKey":"payload.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload","typeLabel":"any | null"}},"required":["error","payload"],"uniqueKey":"","typeLabel":"object"},"example":{"error":{"code":"INVALID_REQUEST_BODY","msg":"evaluation_id must be a valid UUID","message":"evaluation_id must be a valid UUID"},"payload":null},"examples":{"Example":{"title":"Example","value":{"error":{"code":"INVALID_REQUEST_BODY","msg":"evaluation_id must be a valid UUID","message":"evaluation_id must be a valid UUID"},"payload":null}}},"exampleType":"examples"}},"exampleType":"examples"},"88b683b0-2db1-4289-b74f-b99f18985883":{"description":"Missing or invalid API key.","content":{"application/json":{"schema":{"type":"object","description":"Standard error response wrapper","properties":{"error":{"type":"object","properties":{"code":{"type":"string","description":"Error code indicating the type of error. Common codes:\n- Authentication: `API_KEY_MISSING`, `INVALID_API_KEY`\n- Validation: `INVALID_REQUEST_BODY`, `ORGANIZATION_ID_MISSING`, `TICKERS_MISSING`, `CHAT_MESSAGE_ID_MISSING`, `DOCUMENT_ID_MISSING`\n- Service: `SEARCH_FAIL`, `TTS_FAIL`, `CREATE_CONV_FAIL`, `GET_STOCK_FAIL`, `CREATE_MSG_FAIL`, `ALPHA_COMP_FAIL`, `GET_REF_FAIL`\n- Connectors: `CONNECTOR_LIST_FAIL`, `CONNECTOR_NOT_FOUND`, `CONNECTOR_CREATE_FAIL`, `CONNECTOR_UPDATE_FAIL`, `CONNECTOR_DELETE_FAIL`, `CONNECTOR_TEST_FAIL`\n- Not Found: `NOT_FOUND`\n- Meeting cancellation: `MEETING_NOT_FOUND`, `MEETING_NOT_CANCELABLE`, `MEETING_CANCEL_REASON_REQUIRED`, `MEETING_CANCEL_UNAVAILABLE`, `MEETING_CANCEL_FAIL`\n- Meeting compliance review: `MEETING_NOT_FOUND`, `MEETING_REVIEW_NOT_PERMITTED`, `MEETING_NOT_REVIEWABLE`, `MEETING_REVIEW_UNAVAILABLE`, `MEETING_REVIEW_FAIL`\n- Transcriptions: `TRANSCRIPTION_UPLOAD_FAIL`, `TRANSCRIPTION_SUBMIT_FAIL`, `TRANSCRIPTION_GET_FAIL`, `TRANSCRIPTION_NOT_FOUND`, `TRANSCRIPTION_REQUEST_INVALID`, `TRANSCRIPTION_RESULT_UNAVAILABLE` (503, poll again), `TRANSCRIPTION_SUBMIT_UNAVAILABLE` (503, submit again with the same Idempotency-Key)","example":"INVALID_REQUEST_BODY","isRequired":true,"uniqueKey":"error.code","typeLabel":"string"},"msg":{"type":"string","description":"Error message (deprecated, use `message` instead)","example":"query is required and must be a string","uniqueKey":"error.msg","typeLabel":"string"},"message":{"type":"string","description":"Error message providing more details about the error","example":"query is required and must be a string","uniqueKey":"error.message","typeLabel":"string"}},"required":["code"],"isRequired":true,"uniqueKey":"error","typeLabel":"object"},"payload":{"oneOf":[{"description":"Always null for error responses","isRequired":true,"uniqueKey":"payload.oneOf.0","typeLabel":"any"},{"description":"Always null for error responses","type":"null","isRequired":true,"uniqueKey":"payload.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload","typeLabel":"any | null"}},"required":["error","payload"],"uniqueKey":"","typeLabel":"object"},"example":{"error":{"code":"UNAUTHORIZED","msg":"Invalid API key","message":"Invalid API key"},"payload":null},"examples":{"Example":{"title":"Example","value":{"error":{"code":"UNAUTHORIZED","msg":"Invalid API key","message":"Invalid API key"},"payload":null}}},"exampleType":"examples"}},"exampleType":"examples"},"ec7acf8d-2aaf-48f0-83a2-6de1d9915cb5":{"description":"No such evaluation in your organization. An id owned by another organization is indistinguishable from one that does not exist.","content":{"application/json":{"schema":{"type":"object","description":"Standard error response wrapper","properties":{"error":{"type":"object","properties":{"code":{"type":"string","description":"Error code indicating the type of error. Common codes:\n- Authentication: `API_KEY_MISSING`, `INVALID_API_KEY`\n- Validation: `INVALID_REQUEST_BODY`, `ORGANIZATION_ID_MISSING`, `TICKERS_MISSING`, `CHAT_MESSAGE_ID_MISSING`, `DOCUMENT_ID_MISSING`\n- Service: `SEARCH_FAIL`, `TTS_FAIL`, `CREATE_CONV_FAIL`, `GET_STOCK_FAIL`, `CREATE_MSG_FAIL`, `ALPHA_COMP_FAIL`, `GET_REF_FAIL`\n- Connectors: `CONNECTOR_LIST_FAIL`, `CONNECTOR_NOT_FOUND`, `CONNECTOR_CREATE_FAIL`, `CONNECTOR_UPDATE_FAIL`, `CONNECTOR_DELETE_FAIL`, `CONNECTOR_TEST_FAIL`\n- Not Found: `NOT_FOUND`\n- Meeting cancellation: `MEETING_NOT_FOUND`, `MEETING_NOT_CANCELABLE`, `MEETING_CANCEL_REASON_REQUIRED`, `MEETING_CANCEL_UNAVAILABLE`, `MEETING_CANCEL_FAIL`\n- Meeting compliance review: `MEETING_NOT_FOUND`, `MEETING_REVIEW_NOT_PERMITTED`, `MEETING_NOT_REVIEWABLE`, `MEETING_REVIEW_UNAVAILABLE`, `MEETING_REVIEW_FAIL`\n- Transcriptions: `TRANSCRIPTION_UPLOAD_FAIL`, `TRANSCRIPTION_SUBMIT_FAIL`, `TRANSCRIPTION_GET_FAIL`, `TRANSCRIPTION_NOT_FOUND`, `TRANSCRIPTION_REQUEST_INVALID`, `TRANSCRIPTION_RESULT_UNAVAILABLE` (503, poll again), `TRANSCRIPTION_SUBMIT_UNAVAILABLE` (503, submit again with the same Idempotency-Key)","example":"INVALID_REQUEST_BODY","isRequired":true,"uniqueKey":"error.code","typeLabel":"string"},"msg":{"type":"string","description":"Error message (deprecated, use `message` instead)","example":"query is required and must be a string","uniqueKey":"error.msg","typeLabel":"string"},"message":{"type":"string","description":"Error message providing more details about the error","example":"query is required and must be a string","uniqueKey":"error.message","typeLabel":"string"}},"required":["code"],"isRequired":true,"uniqueKey":"error","typeLabel":"object"},"payload":{"oneOf":[{"description":"Always null for error responses","isRequired":true,"uniqueKey":"payload.oneOf.0","typeLabel":"any"},{"description":"Always null for error responses","type":"null","isRequired":true,"uniqueKey":"payload.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload","typeLabel":"any | null"}},"required":["error","payload"],"uniqueKey":"","typeLabel":"object"},"example":{"error":{"code":"JUDGE_EVALUATION_NOT_FOUND","msg":"Evaluation not found","message":"Evaluation not found"},"payload":null},"examples":{"Example":{"title":"Example","value":{"error":{"code":"JUDGE_EVALUATION_NOT_FOUND","msg":"Evaluation not found","message":"Evaluation not found"},"payload":null}}},"exampleType":"examples"}},"exampleType":"examples"},"80081f2d-a913-41c0-b700-a0c0b1bf4a23":{"description":"Organization rate limit exceeded.","content":{"application/json":{"schema":{"type":"object","description":"Standard error response wrapper","properties":{"error":{"type":"object","properties":{"code":{"type":"string","description":"Error code indicating the type of error. Common codes:\n- Authentication: `API_KEY_MISSING`, `INVALID_API_KEY`\n- Validation: `INVALID_REQUEST_BODY`, `ORGANIZATION_ID_MISSING`, `TICKERS_MISSING`, `CHAT_MESSAGE_ID_MISSING`, `DOCUMENT_ID_MISSING`\n- Service: `SEARCH_FAIL`, `TTS_FAIL`, `CREATE_CONV_FAIL`, `GET_STOCK_FAIL`, `CREATE_MSG_FAIL`, `ALPHA_COMP_FAIL`, `GET_REF_FAIL`\n- Connectors: `CONNECTOR_LIST_FAIL`, `CONNECTOR_NOT_FOUND`, `CONNECTOR_CREATE_FAIL`, `CONNECTOR_UPDATE_FAIL`, `CONNECTOR_DELETE_FAIL`, `CONNECTOR_TEST_FAIL`\n- Not Found: `NOT_FOUND`\n- Meeting cancellation: `MEETING_NOT_FOUND`, `MEETING_NOT_CANCELABLE`, `MEETING_CANCEL_REASON_REQUIRED`, `MEETING_CANCEL_UNAVAILABLE`, `MEETING_CANCEL_FAIL`\n- Meeting compliance review: `MEETING_NOT_FOUND`, `MEETING_REVIEW_NOT_PERMITTED`, `MEETING_NOT_REVIEWABLE`, `MEETING_REVIEW_UNAVAILABLE`, `MEETING_REVIEW_FAIL`\n- Transcriptions: `TRANSCRIPTION_UPLOAD_FAIL`, `TRANSCRIPTION_SUBMIT_FAIL`, `TRANSCRIPTION_GET_FAIL`, `TRANSCRIPTION_NOT_FOUND`, `TRANSCRIPTION_REQUEST_INVALID`, `TRANSCRIPTION_RESULT_UNAVAILABLE` (503, poll again), `TRANSCRIPTION_SUBMIT_UNAVAILABLE` (503, submit again with the same Idempotency-Key)","example":"INVALID_REQUEST_BODY","isRequired":true,"uniqueKey":"error.code","typeLabel":"string"},"msg":{"type":"string","description":"Error message (deprecated, use `message` instead)","example":"query is required and must be a string","uniqueKey":"error.msg","typeLabel":"string"},"message":{"type":"string","description":"Error message providing more details about the error","example":"query is required and must be a string","uniqueKey":"error.message","typeLabel":"string"}},"required":["code"],"isRequired":true,"uniqueKey":"error","typeLabel":"object"},"payload":{"oneOf":[{"description":"Always null for error responses","isRequired":true,"uniqueKey":"payload.oneOf.0","typeLabel":"any"},{"description":"Always null for error responses","type":"null","isRequired":true,"uniqueKey":"payload.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload","typeLabel":"any | null"}},"required":["error","payload"],"uniqueKey":"","typeLabel":"object"},"example":{"error":{"code":"RATE_LIMIT_EXCEEDED","msg":"Rate limit exceeded","message":"Rate limit exceeded"},"payload":null},"examples":{"Example":{"title":"Example","value":{"error":{"code":"RATE_LIMIT_EXCEEDED","msg":"Rate limit exceeded","message":"Rate limit exceeded"},"payload":null}}},"exampleType":"examples"}},"exampleType":"examples"},"3bd2e506-3a83-4b6d-89bc-733542bc87ad":{"description":"Upstream failure while loading the evaluation.","content":{"application/json":{"schema":{"type":"object","description":"Standard error response wrapper","properties":{"error":{"type":"object","properties":{"code":{"type":"string","description":"Error code indicating the type of error. Common codes:\n- Authentication: `API_KEY_MISSING`, `INVALID_API_KEY`\n- Validation: `INVALID_REQUEST_BODY`, `ORGANIZATION_ID_MISSING`, `TICKERS_MISSING`, `CHAT_MESSAGE_ID_MISSING`, `DOCUMENT_ID_MISSING`\n- Service: `SEARCH_FAIL`, `TTS_FAIL`, `CREATE_CONV_FAIL`, `GET_STOCK_FAIL`, `CREATE_MSG_FAIL`, `ALPHA_COMP_FAIL`, `GET_REF_FAIL`\n- Connectors: `CONNECTOR_LIST_FAIL`, `CONNECTOR_NOT_FOUND`, `CONNECTOR_CREATE_FAIL`, `CONNECTOR_UPDATE_FAIL`, `CONNECTOR_DELETE_FAIL`, `CONNECTOR_TEST_FAIL`\n- Not Found: `NOT_FOUND`\n- Meeting cancellation: `MEETING_NOT_FOUND`, `MEETING_NOT_CANCELABLE`, `MEETING_CANCEL_REASON_REQUIRED`, `MEETING_CANCEL_UNAVAILABLE`, `MEETING_CANCEL_FAIL`\n- Meeting compliance review: `MEETING_NOT_FOUND`, `MEETING_REVIEW_NOT_PERMITTED`, `MEETING_NOT_REVIEWABLE`, `MEETING_REVIEW_UNAVAILABLE`, `MEETING_REVIEW_FAIL`\n- Transcriptions: `TRANSCRIPTION_UPLOAD_FAIL`, `TRANSCRIPTION_SUBMIT_FAIL`, `TRANSCRIPTION_GET_FAIL`, `TRANSCRIPTION_NOT_FOUND`, `TRANSCRIPTION_REQUEST_INVALID`, `TRANSCRIPTION_RESULT_UNAVAILABLE` (503, poll again), `TRANSCRIPTION_SUBMIT_UNAVAILABLE` (503, submit again with the same Idempotency-Key)","example":"INVALID_REQUEST_BODY","isRequired":true,"uniqueKey":"error.code","typeLabel":"string"},"msg":{"type":"string","description":"Error message (deprecated, use `message` instead)","example":"query is required and must be a string","uniqueKey":"error.msg","typeLabel":"string"},"message":{"type":"string","description":"Error message providing more details about the error","example":"query is required and must be a string","uniqueKey":"error.message","typeLabel":"string"}},"required":["code"],"isRequired":true,"uniqueKey":"error","typeLabel":"object"},"payload":{"oneOf":[{"description":"Always null for error responses","isRequired":true,"uniqueKey":"payload.oneOf.0","typeLabel":"any"},{"description":"Always null for error responses","type":"null","isRequired":true,"uniqueKey":"payload.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload","typeLabel":"any | null"}},"required":["error","payload"],"uniqueKey":"","typeLabel":"object"},"example":{"error":{"code":"JUDGE_EVALUATION_FAIL","msg":"Failed to load the evaluation","message":"Failed to load the evaluation"},"payload":null},"examples":{"Example":{"title":"Example","value":{"error":{"code":"JUDGE_EVALUATION_FAIL","msg":"Failed to load the evaluation","message":"Failed to load the evaluation"},"payload":null}}},"exampleType":"examples"}},"exampleType":"examples"},"d1f6931e-5de2-4d6f-9eef-3d3dbef06e5c":{"description":"Timed out loading the evaluation.","content":{"application/json":{"schema":{"type":"object","description":"Standard error response wrapper","properties":{"error":{"type":"object","properties":{"code":{"type":"string","description":"Error code indicating the type of error. Common codes:\n- Authentication: `API_KEY_MISSING`, `INVALID_API_KEY`\n- Validation: `INVALID_REQUEST_BODY`, `ORGANIZATION_ID_MISSING`, `TICKERS_MISSING`, `CHAT_MESSAGE_ID_MISSING`, `DOCUMENT_ID_MISSING`\n- Service: `SEARCH_FAIL`, `TTS_FAIL`, `CREATE_CONV_FAIL`, `GET_STOCK_FAIL`, `CREATE_MSG_FAIL`, `ALPHA_COMP_FAIL`, `GET_REF_FAIL`\n- Connectors: `CONNECTOR_LIST_FAIL`, `CONNECTOR_NOT_FOUND`, `CONNECTOR_CREATE_FAIL`, `CONNECTOR_UPDATE_FAIL`, `CONNECTOR_DELETE_FAIL`, `CONNECTOR_TEST_FAIL`\n- Not Found: `NOT_FOUND`\n- Meeting cancellation: `MEETING_NOT_FOUND`, `MEETING_NOT_CANCELABLE`, `MEETING_CANCEL_REASON_REQUIRED`, `MEETING_CANCEL_UNAVAILABLE`, `MEETING_CANCEL_FAIL`\n- Meeting compliance review: `MEETING_NOT_FOUND`, `MEETING_REVIEW_NOT_PERMITTED`, `MEETING_NOT_REVIEWABLE`, `MEETING_REVIEW_UNAVAILABLE`, `MEETING_REVIEW_FAIL`\n- Transcriptions: `TRANSCRIPTION_UPLOAD_FAIL`, `TRANSCRIPTION_SUBMIT_FAIL`, `TRANSCRIPTION_GET_FAIL`, `TRANSCRIPTION_NOT_FOUND`, `TRANSCRIPTION_REQUEST_INVALID`, `TRANSCRIPTION_RESULT_UNAVAILABLE` (503, poll again), `TRANSCRIPTION_SUBMIT_UNAVAILABLE` (503, submit again with the same Idempotency-Key)","example":"INVALID_REQUEST_BODY","isRequired":true,"uniqueKey":"error.code","typeLabel":"string"},"msg":{"type":"string","description":"Error message (deprecated, use `message` instead)","example":"query is required and must be a string","uniqueKey":"error.msg","typeLabel":"string"},"message":{"type":"string","description":"Error message providing more details about the error","example":"query is required and must be a string","uniqueKey":"error.message","typeLabel":"string"}},"required":["code"],"isRequired":true,"uniqueKey":"error","typeLabel":"object"},"payload":{"oneOf":[{"description":"Always null for error responses","isRequired":true,"uniqueKey":"payload.oneOf.0","typeLabel":"any"},{"description":"Always null for error responses","type":"null","isRequired":true,"uniqueKey":"payload.oneOf.1","typeLabel":"null"}],"isRequired":true,"uniqueKey":"payload","typeLabel":"any | null"}},"required":["error","payload"],"uniqueKey":"","typeLabel":"object"},"example":{"error":{"code":"JUDGE_EVALUATION_FAIL","msg":"Evaluation lookup timed out","message":"Evaluation lookup timed out"},"payload":null},"examples":{"Example":{"title":"Example","value":{"error":{"code":"JUDGE_EVALUATION_FAIL","msg":"Evaluation lookup timed out","message":"Evaluation lookup timed out"},"payload":null}}},"exampleType":"examples"}},"exampleType":"examples"}},"security":{"8aa7599b-2dcb-4258-a16e-03ea68ad1736":{"ApiKeyAuth":{"type":"apiKey","in":"header","name":"X-API-KEY","scopes":[],"schemeName":"ApiKeyAuth"}}},"servers":{"9cdf8e58-8f32-4db6-a144-4d35b9596918":{"url":"https://api.linqalpha.com"}},"processedSecurityOptions":[{"schemes":[{"scheme":{"type":"apiKey","in":"header","name":"X-API-KEY","scopes":[],"schemeName":"ApiKeyAuth"},"schema":{"title":"X-API-KEY","type":"string","isRequired":true,"typeLabel":"string","deprecated":false},"schemeName":"ApiKeyAuth","name":"X-API-KEY"}],"title":"ApiKeyAuth"}]},"originalFileLocation":"api-reference/openapi.json"},"subdomain":"linqalpha","siteName":"Linq Alpha API Docs","title":"Get LLM Judge Evaluation - Linq Alpha API Docs","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-09-01T01:51:08.777Z","originalFileLocation":"api-reference/evaluations/get_llm.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Returns HTTP 200 in every state, including while still running. Branch on status, not on the status code. The response shape is identical across statuses — the same seven fields are always present.\">\u003cmeta data-mint-head name=\"application-name\" content=\"Linq Alpha API Docs\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Get LLM Judge Evaluation - Linq Alpha API Docs\">\u003cmeta data-mint-head property=\"og:description\" content=\"Returns HTTP 200 in every state, including while still running. Branch on status, not on the status code. The response shape is identical across statuses — the same seven fields are always present.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Linq Alpha API Docs\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://docs.linqalpha.com/api-reference/evaluations/get_llm\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://linqalpha.mintlify.app/_mintlify/api/og/v2?division=Evaluations\u0026amp;title=Get+LLM+Judge+Evaluation\u0026amp;description=Returns+HTTP+200+in+every+state%2C+including+while+still+running.+Branch+on+%60status%60%2C+not+on+the+status+code.+The+response+shape+is+identical+across+statuses+%E2%80%94+th\u0026amp;theme=90ffa347c1039bd70010bd6c\u0026amp;signature=36a5d2aa12eeeda65217e4b233ee2043870973da15836603fac26fcc44af4e9a\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Get LLM Judge Evaluation - Linq Alpha API Docs\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Returns HTTP 200 in every state, including while still running. Branch on status, not on the status code. The response shape is identical across statuses — the same seven fields are always present.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://linqalpha.mintlify.app/_mintlify/api/og/v2?division=Evaluations\u0026amp;title=Get+LLM+Judge+Evaluation\u0026amp;description=Returns+HTTP+200+in+every+state%2C+including+while+still+running.+Branch+on+%60status%60%2C+not+on+the+status+code.+The+response+shape+is+identical+across+statuses+%E2%80%94+th\u0026amp;theme=90ffa347c1039bd70010bd6c\u0026amp;signature=36a5d2aa12eeeda65217e4b233ee2043870973da15836603fac26fcc44af4e9a\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/linqalpha/lrWql8nYb9-314d0/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Linq Alpha API Docs\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#1767FE\">\u003clink data-mint-head rel=\"canonical\" href=\"https://docs.linqalpha.com/api-reference/evaluations/get_llm\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/api-reference/evaluations/get_llm.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://docs.linqalpha.com/#organization\",\"name\":\"Linq Alpha API Docs\",\"url\":\"https://docs.linqalpha.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/linqalpha/9BNJqkVKHb4kSYyJ/logo/light.svg?fit=max\u0026auto=format\u0026n=9BNJqkVKHb4kSYyJ\u0026q=85\u0026s=d7dd093386c0527ef6e6c9ef5f8130ad\"}},{\"@type\":\"WebSite\",\"@id\":\"https://docs.linqalpha.com#website\",\"name\":\"Linq Alpha API Docs\",\"url\":\"https://docs.linqalpha.com\",\"publisher\":{\"@id\":\"https://docs.linqalpha.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://docs.linqalpha.com/api-reference/evaluations/get_llm#webpage\",\"url\":\"https://docs.linqalpha.com/api-reference/evaluations/get_llm\",\"name\":\"Get LLM Judge Evaluation\",\"description\":\"Returns HTTP 200 in every state, including while still running. Branch on status, not on the status code. The response shape is identical across statuses — the same seven fields are always present.\",\"dateModified\":\"2026-09-01T01:51:08.777Z\",\"isPartOf\":{\"@id\":\"https://docs.linqalpha.com#website\"},\"breadcrumb\":{\"@id\":\"https://docs.linqalpha.com/api-reference/evaluations/get_llm#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://docs.linqalpha.com/api-reference/evaluations/get_llm#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Evaluations\",\"item\":\"https://docs.linqalpha.com/api-reference/evaluations/create\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Get LLM Judge Evaluation\",\"item\":\"https://docs.linqalpha.com/api-reference/evaluations/get_llm\"}]},{\"@type\":[\"TechArticle\",\"APIReference\"],\"@id\":\"https://docs.linqalpha.com/api-reference/evaluations/get_llm#apireference\",\"headline\":\"Get LLM Judge Evaluation\",\"name\":\"Get LLM Judge Evaluation\",\"description\":\"Returns HTTP 200 in every state, including while still running. Branch on status, not on the status code. The response shape is identical across statuses — the same seven fields are always present.\",\"url\":\"https://docs.linqalpha.com/api-reference/evaluations/get_llm\",\"mainEntityOfPage\":{\"@id\":\"https://docs.linqalpha.com/api-reference/evaluations/get_llm#webpage\"},\"dateModified\":\"2026-09-01T01:51:08.777Z\",\"publisher\":{\"@id\":\"https://docs.linqalpha.com/#organization\"},\"isPartOf\":{\"@id\":\"https://docs.linqalpha.com#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"Linq Alpha API Docs\",\"creator\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","siteName":"Linq Alpha API Docs","nav":{"global":{"anchors":[{"anchor":"Website","href":"https://linqalpha.com"}]},"tabs":[{"tab":"Quickstart","pages":[{"title":"Quickstart","href":"/quickstart"}]},{"tab":"API Reference","groups":[{"group":"MCP","pages":[{"title":"LinqAlpha MCP","openapi":"POST /v1/mcp","href":"/api-reference/basic/mcp"}]},{"group":"Basic","pages":[{"title":"Map Tickers","openapi":"GET /v1/map_tickers","href":"/api-reference/basic/map_tickers"},{"title":"Chat Message","openapi":"GET /v1/chat_messages/{chat_message_id}","href":"/api-reference/basic/chat_message"},{"title":"References","openapi":"GET /v1/references","href":"/api-reference/basic/references"},{"title":"References (v2)","openapi":"GET /v2/conversations/{conversation_id}/references","href":"/api-reference/basic/references_v2"},{"title":"Presigned URL","openapi":"GET /v1/documents/{document_id}/presigned_url","href":"/api-reference/basic/presigned_url"},{"title":"Batch Presigned URLs","openapi":"GET /v1/documents/batch_presigned_urls","href":"/api-reference/basic/batch_presigned_urls"},{"title":"Presigned URL V2","openapi":"GET /v2/documents/{document_id}/presigned_url","href":"/api-reference/basic/presigned_url_v2"},{"title":"Batch Presigned URLs V2","openapi":"GET /v2/documents/presigned_urls","href":"/api-reference/basic/batch_presigned_urls_v2"},{"title":"Customer Connectors","openapi":"GET /v2/connectors","href":"/api-reference/basic/connectors"},{"title":"Search","openapi":"POST /v1/search","href":"/api-reference/basic/search"},{"title":"TTSQL","openapi":"POST /v1/ttsql","href":"/api-reference/basic/ttsql"},{"title":"Chat","openapi":"POST /v2/chat/sse","href":"/api-reference/basic/chat_v2"},{"title":"Analytics","openapi":"POST /v1/analytics/sse","href":"/api-reference/basic/analytics-sse"},{"title":"Stop Stream","openapi":"POST /v1/stop_stream","href":"/api-reference/basic/stop_stream"},{"title":"Conversation Feedback","openapi":"POST /v1/conversations/{conversation_id}/feedback","href":"/api-reference/basic/conversation_feedback"},{"title":"Delete Conversation Feedback","openapi":"DELETE /v1/conversations/{conversation_id}/feedback","href":"/api-reference/basic/delete_conversation_feedback"}]},{"group":"Analytics V2","pages":[{"title":"Analytics V2","openapi":"POST /v2/analytics/sse","href":"/api-reference/basic/analytics-sse-v2"},{"title":"Analytics V2 References","openapi":"GET /v2/analytics/conversations/{conversation_id}/references","href":"/api-reference/basic/analytics_references"},{"title":"Analytics V2 Judge","openapi":"POST /v2/analytics/messages/{chat_message_id}/judge","href":"/api-reference/basic/analytics_judge"},{"title":"Vault — Get Upload URL","openapi":"POST /v2/vault/presigned_url","href":"/api-reference/vault/presigned_url"},{"title":"Vault — Confirm Upload","openapi":"POST /v2/vault/confirm","href":"/api-reference/vault/confirm"},{"title":"Vault — Document Status","openapi":"GET /v2/vault/status","href":"/api-reference/vault/status"}]},{"group":"Transcriptions","tag":"NEW","pages":[{"title":"Transcriptions — Get Upload URL","openapi":"POST /v2/transcriptions/uploads","href":"/api-reference/transcriptions/upload"},{"title":"Transcriptions — Submit Job","openapi":"POST /v2/transcriptions","href":"/api-reference/transcriptions/create"},{"title":"Transcriptions — Poll / Retrieve","openapi":"GET /v2/transcriptions/{job_id}","href":"/api-reference/transcriptions/get"},{"title":"Transcriptions — Supported Languages","href":"/api-reference/transcriptions/languages"}]},{"group":"Evaluations","pages":[{"title":"Execute Agent Judge","openapi":"POST /v2/judge/agent","href":"/api-reference/evaluations/create"},{"title":"Get Agent Judge Evaluation","openapi":"GET /v2/judge/agent/{evaluation_id}","href":"/api-reference/evaluations/get"},{"title":"Execute LLM Judge","openapi":"POST /v2/judge/llm","description":"Records the submission and queues it. Returns immediately; the judgement itself runs asynchronously and is retrieved with `GET /v2/judge/llm/{evaluation_id}`. Unlike `POST /v2/judge/agent` (Agent Judge), which returns a written assessment, this endpoint produces a **structured verdict** — dimension scores plus reasoning — on a fixed rubric (Factuality / Completeness / Relevance / Grounding, each 1-5) with a server-computed `overall_score`. Every run uses the same rubric so runs are comparable across callers and time.","href":"/api-reference/evaluations/create_llm"},{"title":"Get LLM Judge Evaluation","openapi":"GET /v2/judge/llm/{evaluation_id}","href":"/api-reference/evaluations/get_llm"}]},{"group":"RMS (On request)","pages":[{"title":"Source","openapi":"GET /v1/sources/{source_id}","tag":["New"],"description":"Retrieves the current processing status of an uploaded source. Use this endpoint to poll source readiness before including it in chat or search requests.\n\n**Status Values:**\n- `processing`: Source is being parsed and indexed (typically takes 10-60 seconds depending on file size)\n- `success`: Source is ready for use in conversations\n- `failed`: Processing failed (check file format, size, or content validity)\n\n**Best Practice:** Poll this endpoint with exponential backoff until status is `success` before initiating chat/search requests.","href":"/api-reference/rms/source"},{"title":"Create Organization","openapi":"POST /v1/organizations","href":"/api-reference/rms/create_organization"},{"title":"Create Source Batch","openapi":"POST /v1/source_batches","tag":["New"],"href":"/api-reference/rms/create_source_batch"},{"title":"Get Upload URL","openapi":"POST /v1/sources/upload_url","tag":["New"],"href":"/api-reference/rms/upload_url"},{"title":"Create Source","openapi":"POST /v1/sources","tag":["New"],"href":"/api-reference/rms/create_source"},{"title":"RMS Chat","openapi":"POST /v1/rms_chat","href":"/api-reference/rms/rms_chat"},{"title":"RMS Deep Research","openapi":"POST /v1/rms_deep_research","href":"/api-reference/rms/rms_deep_research"}]},{"group":"Briefing","tag":"NEW","pages":[{"title":"Create Briefing Schedule","openapi":"POST /v1/briefings","href":"/api-reference/briefing/create"},{"title":"List Briefing Schedules","openapi":"GET /v1/briefings","href":"/api-reference/briefing/list"},{"title":"Update Briefing Schedule","openapi":"PATCH /v1/briefings/{id}","href":"/api-reference/briefing/update"},{"title":"Delete Briefing Schedule","openapi":"DELETE /v1/briefings/{id}","href":"/api-reference/briefing/delete"},{"title":"Preview Briefing On-Demand","openapi":"POST /v1/briefings/preview","href":"/api-reference/briefing/preview"},{"title":"List Deliveries","openapi":"GET /v1/briefings/{id}/deliveries","href":"/api-reference/briefing/deliveries"},{"title":"Get Delivery Detail","openapi":"GET /v1/briefings/deliveries/{delivery_id}","href":"/api-reference/briefing/delivery-detail"},{"title":"Get Delivery References","openapi":"GET /v1/briefings/deliveries/{delivery_id}/references","href":"/api-reference/briefing/delivery-references"},{"title":"Refresh External Content PDF Upload URL","openapi":"POST /v1/external_content/refresh_upload_url","href":"/api-reference/briefing/external-content-refresh-upload-url"}]},{"group":"Sync Status","tag":"NEW","pages":[{"title":"Organization Sync Status","openapi":"GET /v1/status/sync","href":"/api-reference/status/sync"},{"title":"Document Sync Status","openapi":"GET /v1/status/documents","href":"/api-reference/status/documents"},{"title":"Container Sync Status","openapi":"GET /v1/status/containers","href":"/api-reference/status/containers"}]},{"group":"Meeting Agent","pages":[{"title":"Cancel Meeting Agent","openapi":"POST /v1/meetings/{meeting_id}/cancel","href":"/api-reference/meetings/cancel"},{"title":"Approve Meeting Agent Results","openapi":"POST /v1/meetings/{meeting_id}/approve","href":"/api-reference/meetings/approve"},{"title":"Reject Meeting Agent Results","openapi":"POST /v1/meetings/{meeting_id}/reject","href":"/api-reference/meetings/reject"}]}]}]},"title":"Get LLM Judge Evaluation - Linq Alpha API Docs","buildId":"dpl_DFJrtvUi8HS5F9nkooKuDJsuD5BM","deploymentId":"dpl_DFJrtvUi8HS5F9nkooKuDJsuD5BM","dynamics":["c2aa737661","2b790eda92","b0f84f53ff","a9bb46feb3"],"chunks":["OpenApiReferenceProvider-rfhihCWU.js","utils-BmH1XFxk.js","cloneDeep-izs3Hzxb.js","_Uint8Array-BVIqJ90l.js","isObjectLike-C8vmzPEN.js","isArray-Dcu1dQ5t.js","utils-BcNsxPKB.js","toString-BX-iyh8G.js","isSymbol-CSH9U2Gm.js","addUniqueKeysToSchema-BAES3tkd.js","snippetPresets-DCMQBKLP.js","style-to-object-BY3JEAdk.js","v4-DIgz8zy6.js","ApiExamples-D6l_bzE9.js","isEqual-BfaiBbcl.js","usePlaygroundInputsStore-BOktbnNh.js","shallow-BCs8SgM7.js","getApiPlaygroundDisplay-DEmW5bJB.js","CodeSnippet-CpZOpB9C.js","CodeSnippets-BOBDqibl.js","Page-D5fEM6aW.js","getPastedJson-DH2JBvmr.js","icon-BBJH-kKh.js","MethodPill-D2Sd5NLC.js","DuplicateIcon-CGgfL2qN.js","createIcon-BKudBBzi.js","DynamicMarkdown-85clUA0i.js","SectionHeading-D0JiMgDx.js","axios-vUo_bRtp.js","Markdown-Cfiy0xZO.js","clsx-rL8ZCCw4.js","lib--scc4DXi.js","lib-B8HbPgcC.js","html-void-elements-mCSPHezU.js","dist-aeztFa9U.js","katex-YMM-FMeD.js","marked.esm-CZWythHk.js","AssistantPageLink-BywkCr5C.js"]}