{"id":14941,"date":"2026-09-24T19:30:00","date_gmt":"2026-09-24T14:00:00","guid":{"rendered":"https:\/\/www.allerin.com\/blog\/?p=14941"},"modified":"2026-09-20T19:36:50","modified_gmt":"2026-09-20T14:06:50","slug":"ai-evaluation-critical-failures-release-decisions","status":"publish","type":"post","link":"https:\/\/www.allerin.com\/blog\/ai-evaluation-critical-failures-release-decisions\/","title":{"rendered":"When a better AI evaluation score should still block a release"},"content":{"rendered":"<style>.allerin-ai-example{max-width:100%;overflow-wrap:anywhere}.allerin-ai-example h2{scroll-margin-top:125px}.allerin-ai-example nav{border-left:3px solid #17665b;padding:10px 18px;margin:24px 0}.allerin-ai-example nav ol{padding-left:22px}.allerin-ai-example nav a{display:inline-block}.allerin-ai-example code{white-space:normal;overflow-wrap:anywhere}.allerin-ai-example pre{max-width:100%;overflow:auto}.allerin-ai-example pre code{white-space:pre;overflow-wrap:normal}.allerin-ai-example .table-wrap{max-width:100%;overflow-x:auto;margin:22px 0}.allerin-ai-example table{width:100%;min-width:540px;border-collapse:collapse;line-height:1.5}.allerin-ai-example th,.allerin-ai-example td{border:1px solid #d1d9d5;padding:10px;text-align:left;vertical-align:top}.allerin-ai-example th{background:#edf2ef}.allerin-ai-example .scroll-hint{font-size:.85em}.allerin-ai-example .example-disclosure{margin-top:28px;border-top:1px solid #d1d9d5;padding-top:18px}.allerin-ai-example .table-wrap:focus-visible,.allerin-ai-example pre:focus-visible{outline:3px solid #17665b;outline-offset:3px}<\/style>\n<div class=\"allerin-ai-example\" data-example=\"AI03\">\n<nav aria-label=\"In this article\"><strong>In this article<\/strong><\/p>\n<ol>\n<li><a href=\"#define-what-completing-the-task-means\">Define what completing the task means<\/a><\/li>\n<li><a href=\"#check-the-answer-key-and-the-grader\">Check the answer key and the grader<\/a><\/li>\n<li><a href=\"#keep-critical-failures-visible\">Keep critical failures visible<\/a><\/li>\n<li><a href=\"#preserve-the-run-before-summarizing-it\">Preserve the run before summarizing it<\/a><\/li>\n<li><a href=\"#use-the-fixture-to-prepare-a-real-decision\">Prepare an AI release evaluation with the fixture<\/a><\/li>\n<li><a href=\"#about-this-example\">About this example<\/a><\/li>\n<li><a href=\"#download-example\">Download the example<\/a><\/li>\n<\/ol>\n<\/nav>\n<p>An evaluation summary improves from six passing cases to eight. That sounds encouraging until one of the newly failing answers uses a finance-only policy for a caller who cannot access it. The average improved while an access boundary regressed.<\/p>\n<p>The accompanying example makes that tradeoff visible with deliberately injected results. It uses ten fictional policy questions, a fixed source snapshot and a separately prepared answer key. No model was called. The experiment tests whether an evaluation harness can distinguish acceptable work from known failures before anybody trusts it to judge a real AI workflow.<\/p>\n<h2 id=\"define-what-completing-the-task-means\">Define what completing the task means<\/h2>\n<p>The task is to answer a current policy question using evidence available to the caller&#8217;s tenant and role. An answer needs the requested facts, their units and supporting source versions. If the permitted sources cannot establish the answer, the correct result is an explicit abstention without disclosing restricted information.<\/p>\n<p>This definition makes several outcomes different. A correct abstention completes the task. In this fixture, an injected timeout record represents a known call failure. An unknown outcome means no final result was established. A missing record means the run is incomplete. Combining these into one \u201cdid not answer\u201d bucket would lose information needed to fix the workflow.<\/p>\n<p><a href=\"https:\/\/www.anthropic.com\/engineering\/demystifying-evals-for-ai-agents\" target=\"_blank\" rel=\"noopener\">Anthropic&#8217;s evaluation guide<\/a> distinguishes the task outcome from the transcript and discusses the strengths and limits of different graders. That supports inspecting what the system accomplished and how it was judged. It does not supply a universal score at which an application becomes ready.<\/p>\n<h2 id=\"check-the-answer-key-and-the-grader\">Check the answer key and the grader<\/h2>\n<p>A separate AI reviewer authored the fictional corpus and expected outcomes before the implementation. The corpus contains two invented organizations, current and retired policies, and one finance-restricted source. The ten cases and sixteen control expectations were retained independently from the program&#8217;s output.<\/p>\n<p>The reference implementation reads the question, trusted fixture identity and corpus. It uses a few scripted question patterns to find permitted current facts; it never receives the expected answers. This is a small reference path, not a natural-language model or a real sign-in system.<\/p>\n<p>Before grading results, the harness checks each expected answer&#8217;s source references, permissions and supporting facts. That can catch a broken fixture. It cannot establish that a policy is appropriate or that every abstention label is correct. Those decisions still require independent task and domain review.<\/p>\n<p>The grader also needs positive controls. Two current documents support the same meal limit, so either should pass. A finance user may legitimately answer the restricted lodging question. Rejecting every alternative source or every restricted document would make the evaluator look strict while judging correct work incorrectly.<\/p>\n<h2 id=\"keep-critical-failures-visible\">Keep critical failures visible<\/h2>\n<p>The local rubric treats unsupported facts and forbidden, stale or unsupported evidence as critical failures. A partial correct answer or an unnecessary abstention is an ordinary task failure. These categories describe this fixture&#8217;s contract; responsible owners must define the consequences for an actual workflow.<\/p>\n<div class=\"table-wrap\" tabindex=\"0\" role=\"region\" aria-label=\"Executed comparison results\">\n<table>\n<thead>\n<tr>\n<th>Executed run<\/th>\n<th align=\"right\">Passing cases<\/th>\n<th align=\"right\">Ordinary failures<\/th>\n<th align=\"right\">Critical failures<\/th>\n<th>Local decision<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Scripted reference<\/td>\n<td align=\"right\">10\/10<\/td>\n<td align=\"right\">0<\/td>\n<td align=\"right\">0<\/td>\n<td>Accept this fixture run<\/td>\n<\/tr>\n<tr>\n<td>Prior injected control<\/td>\n<td align=\"right\">6\/10<\/td>\n<td align=\"right\">4<\/td>\n<td align=\"right\">0<\/td>\n<td>Hold<\/td>\n<\/tr>\n<tr>\n<td>Changed injected control<\/td>\n<td align=\"right\">8\/10<\/td>\n<td align=\"right\">1<\/td>\n<td align=\"right\">1<\/td>\n<td>Hold<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p class=\"scroll-hint\">On a narrow screen, scroll the table sideways to read every column.<\/p>\n<p>The changed control repairs three unnecessary abstentions and introduces one forbidden-source answer. Both controls are held because this example requires every case to pass. The comparison does not show a real model improving, a previously approved release becoming unsafe, or a general readiness threshold.<\/p>\n<p>The controls also check a correct number supported by an obsolete source, a correct number with an irrelevant citation and a confident answer to a question absent from the corpus. Value-only matching would miss distinctions that matter to the task.<\/p>\n<h2 id=\"preserve-the-run-before-summarizing-it\">Preserve the run before summarizing it<\/h2>\n<p>Each required case must appear exactly once in a trial. The harness retains duplicate records for diagnosis but withholds an acceptance fraction. Missing, unexpected and malformed records also hold the run. It never quietly selects the better duplicate or changes the denominator to the number of returned answers.<\/p>\n<p>Results use structured facts and evidence coordinates. Equivalent finite numbers can match, but units must agree and Boolean values cannot be substituted with numbers. Evidence order is irrelevant; every supplied citation must be current, permitted and relevant. Arbitrary answer prose is outside the accepted schema, so a correct fact map cannot silently validate a contradictory paragraph.<\/p>\n<p>This boundary matters when adding a real model. Its raw output, any conversion into structured facts and the semantic grader need separate review. Repeated trials must remain identifiable rather than being flattened into a single favorable result.<\/p>\n<h2 id=\"use-the-fixture-to-prepare-a-real-decision\">Prepare an AI release evaluation with the fixture<\/h2>\n<p>Run the <a href=\"#download-example\">companion<\/a> from its directory:<\/p>\n<pre tabindex=\"0\" aria-label=\"Reproduction commands\"><code class=\"language-sh\">python3 evaluation_demo.py\r\npython3 -m unittest -v\r\n<\/code><\/pre>\n<p>The <a href=\"#download-example\">execution record<\/a> identifies the tested files and environment. Research and execution took place on 20 September 2026 with Python 3.14.7. The visible cases are not a held-out benchmark, and the example measures no latency, token usage, monetary cost or production ROI.<\/p>\n<p>For a real release, start with one workflow and agree its accepted outcomes with the product, domain and security owners. Add representative cases, preserved failures and independently reviewed labels before comparing configurations. Allerin&#8217;s <a href=\"https:\/\/www.allerin.com\/services\/genai-accelerator\">GenAI engineering<\/a> and <a href=\"https:\/\/www.allerin.com\/services\/ai-orchestration\">AI orchestration<\/a> work can begin with that bounded evaluation review, followed by implementation of the gaps it establishes.<\/p>\n<section class=\"example-disclosure\" aria-labelledby=\"about-this-example\">\n<h2 id=\"about-this-example\">About this example<\/h2>\n<p>AI tools assisted the research, writing and code. Independent technical and editorial reviews were performed by AI agents. This example uses synthetic data and does not describe a customer deployment. Results apply only to the recorded versions and test conditions. Review and test the companion code against your own requirements before production use.<\/p>\n<p>This example grades fixed structured results against a fictional policy corpus. No LLM was run, and the reported scores are not model-quality measurements.<\/p>\n<\/section>\n<section aria-labelledby=\"download-example\">\n<h2 id=\"download-example\">Download the example<\/h2>\n<p><a href=\"https:\/\/www.allerin.com\/blog\/wp-content\/uploads\/2026\/09\/allerin-ai-release-evaluation-2026-09-20.zip\">Download the runnable release evaluation example and execution record (ZIP)<\/a>. Extract it and open <code>AI03-release-evaluation\/README.md<\/code>. The included <code>EXECUTION.json<\/code> records the tested environment and file identities; <code>expected-demo.json<\/code> preserves the demonstration output. The public package replay used the same implementation and tests as the original 20 September 2026 experiment. The included README identifies this public documentation adaptation.<\/p>\n<\/section>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Use a runnable evaluation example to separate critical failures, correct abstentions and incomplete runs before trusting an AI release score.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[5],"tags":[2078,260,2081],"class_list":["post-14941","post","type-post","status-publish","format-standard","hentry","category-ai","tag-ai-engineering","tag-artificial-intelligence","tag-release-evaluation"],"_links":{"self":[{"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/posts\/14941","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/comments?post=14941"}],"version-history":[{"count":3,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/posts\/14941\/revisions"}],"predecessor-version":[{"id":14963,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/posts\/14941\/revisions\/14963"}],"wp:attachment":[{"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/media?parent=14941"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/categories?post=14941"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.allerin.com\/blog\/wp-json\/wp\/v2\/tags?post=14941"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}