Performance-improvement review: validation

Start at the delivery audit and sprint-11 final acceptance. The RED record below is preserved apart from this navigation note. Earlier results are in GREEN and post-refactor replay; C5 and its complete 33-case result retain the actual execution history. Earlier not run, fallback-only discovery and sprint-10 residual obligations describe their historical freezes, not current status. Historical "verified human" labels are superseded by the source-attribution audit, not by changed case expectations.

RED creation recorded: 2026-09-22T11:04:34Z, against repository revision ae294143e5890033fcb717956050a91a6f2ed6e8. Freeze version RED-1 includes the independently verified pre-freeze corrections recorded below. This freezes source-derived expectations before a new reviewer exists. It is corpus validation, not a failed candidate execution. No candidate reviewer, benchmark, compiler build or compiler test is run in this sprint. Source acquisition remains bounded by corpus.md's 2026-04-22 through 2026-09-22 window and 2026-09-22T09:28:34Z cutoff; later capture times do not extend it.

Outcomes

• pass: the applicable claim/check is supported by the available evidence; not a universal certification.
• needs evidence: a specific missing measurement, reproducibility detail, or behavior proof prevents a conclusion; not proof of a defect.
• risk/blocker: concrete code, observed regression, or a substantiated failure scenario establishes a material risk. Cite the mechanism and evidence.
• not applicable: this check does not apply to the actual claim/change; give a short reason.

G1-G8 mean the exact eight groups in evidence.md. Outcomes below apply only to named checks. A structural/test-shape pass is not an overall PR approval or an assertion that the tests were executed here.

Packet boundary and immutable inputs

Give the candidate only the selected fenced text packet below, the PR/family/code coordinates/capture/hash fields of its source-pin row (not corpus links), and the diff/source files reconstructed at those pins. Do not give it this document's oracle matrix, coverage/audit sections, evidence rules' source discussion, corpus source notes, PR decisions, review threads, later fixes, or live GitHub discussion access. The runner may fetch pinned Git objects but not mutable PR pages. This keeps the held-back objection/answer out of the reviewer input. Full diffs are reconstructible, not copied into the matrix:

git --no-pager diff --no-ext-diff --no-textconv --full-index --binary $mergeBase $head
git --no-pager show "${head}:src/Compiler/AbstractIL/ilwrite.fs"

Git object syntax uses repository-relative /; filesystem paths use \. Use the merge base for that historical head, not today's main or a final head's merge base. Descriptions are captured final-stage snapshots, with original body hashes/capture times in the corpus. Earlier edited descriptions cannot be recovered: historical code-only pairs deliberately contain no PR description or benchmark table. They score only the designated code/behavior/scope check; missing descriptions in those curated slices are not claims that the real PR lacked a description. Never combine a final description with old code and pretend it was available then.

Source-pin keys are evaluator bookkeeping, not outcome hints. For final-stage packets, the exact corpus description/diff hashes remain the raw-snapshot references; the compact packet is a faithful extraction, not a verbatim historical body. SHA-256 packet hashes below cover the fenced text's UTF-8 bytes with LF line endings and a final LF. Future changes must create a new case/version rather than quietly overwrite these oracles.

P01

Pin A. Final captured description; no review decisions supplied.
Claim: fuse copyExpr and remarkExpr to avoid the second full tree rebuild.
65,880 LOC / 120 files, --optimize+: allocated MB/compile 10,673 -> 10,352.
Description reports byte-identical emitted IL/PDBs on all checked workloads.
No repetition count, variability, machine/tool versions, workload revision or runnable command supplied.
Diff: optimizer call sites plus pervasive optional range/debug-point handling in the remapper and an exported internal signature; no added tests.

P02

Pin B. Final captured description.
Claim: fuse generic inlining copy and type instantiation using remapExpr/mkInstRemap.
Release net11.0, 65,880 LOC, GC.GetTotalAllocatedBytes(true), 3 runs x 4 iterations.
build | allocated MB/compile
base | 10,662
changed | 10,105
Description reports 671 EmittedIL/Optimizations passes, six PrintFunction environment failures also on base, and differential IL/PDB identity for listed generic/loop/closure/byref/SRTP cases.
Reflected-definition resource-name tag changes with the unique-number count; payload hash reportedly unchanged.
Two diff hunks in one localized optimizer function plus release note; remapping implementation unchanged.
No raw runs, dispersion, hardware/SDK identity, workload revision or runnable benchmark command supplied.

P03

Pin C. Historical code-only scope check, not a reconstructed PR description.
SortTableRows packs Val/Tag/original position into int64 keys to preserve stable ordering.
It fills zero-created key and result arrays with indexed loops; System.Array.Sort sorts the keys.
Inspect the complete pinned diff and SortTableRows callers. No timing evidence is supplied in this slice.

P04

Pin D. Final captured description and code.
Four self-profiled compiler hot-path changes: metadata sorting, postponed target emission, nullness reuse, type-hierarchy common-subexpression elimination.
Metadata sort: 65,536 rows, 10.6 -> 2.5 ms and 6.55 -> 1.05 MB allocated; 64 rows, 2.99 -> 0.43 us.
Nullness application: about 153,000 allocations removed per 65k-LOC compile; sampled own CPU 0.52 -> 0.38 s.
Hierarchy loop exclusive CPU 214 -> 160 ms. Target-emission microbench claims 2.6-7x speed and about half allocations.
Description claims output identity. Repetitions/error, environment, exact workload/compiler benchmark commits and runnable commands absent.
SortTableRows uses array comprehensions for keys/results at this head.

P05

Pin E. Final captured description. Claim: less retained memory after ParseAndCheckProject.
Mean of 3 runs in fresh processes per project.
Project | before MB | after MB
consoleapp | 33.25 | 31.11
Oxpecker | 69.65 | 65.77
IcedTasks | 47.10 | 44.93
FsToolkit | 69.97 | 67.86
Fantomas.Benchmarks | 64.26 | 61.33
Prime | 100.99 | 99.44
Fantomas.Core | 107.74 | 106.21
Fantomas.Core.Tests | 140.20 | 137.61
FSharp.Common | 261.81 | 251.68
fcs | 1226.73 | 1223.99
FSharp.Common has 489 references. Description reports total allocation down 1-34 MB/project and analysis time unchanged within noise.
Two tables per reader: string-heap cache grows from zero; computed-name interning removed.
Author reports about 3.0 MB fixed buckets versus at most 0.24 MB duplicated names (2,077 strings).
No machine/runtime/tool versions, spread/raw runs, executable command, subject revisions, benchmark binary commit mapping or precise retained-heap collection protocol supplied.

P06

Pin F. Historical code-only behavior/test check. No historical description recovered.
GetParamAttribs memoizes IL parameter attributes per ImportMap with WeakMap/MemoizationTable.
Key equality uses physical ILMethodDef and extension-view flag, ignoring the supplied range; instantiated declaring types bypass memoization.
Added test compares instance/static Enumerable.Select results using "if a <> b then failwith".
The test pipeline ends with compile |> shouldSucceed.
Inspect OptionalArgInfo.FromILParameter, including metadata import and error location.

P07

Pin G. Final description/code.
Claim: eliminate repeated IL parameter-attribute computation on overload-heavy compilation.
ComputeILMethodParamAttribs allocated 477 MB -> 0; description also says "~810 MB total".
fsc --times table: 99.75% hits; 5,341 adds; 0 updates; 2,167,099 hits; 5,341 misses; 0 evictions.
Output reportedly byte-identical. No elapsed-time table, variability, environment or executable workload command supplied.
Cache uses physical ILMethodDef plus extension flag and ignores range; instantiated declaring types bypass memoization.
Instance/static extension-call test now ends with compileExeAndRun |> shouldSucceed.

P08

Pin H. Historical code-only behavior check; historical description not recovered.
applyBrFixups replaces sorting by offset with List.rev.
RecordReqdBrFixups prepends the current code position, then emits an instruction byte and placeholders.
No execution result is supplied in this slice.

P09

Pin I. Final captured description/code.
Branch fixups: replace sorting by offset with List.rev.
65,880 LOC / 120 files, net11 Release; about 46,465 calls/compile, 88% empty, longest list 25.
step | before | after
time ms/compile | 0.595 | 0.240
allocated KB/compile | 1444 | 634
Author reports 139,370 method-body order comparisons, zero mismatches and zero ties.
RecordReqdBrFixups now documents strictly increasing recorded positions and asserts them in Debug.
No raw measurements, variability, machine/SDK details or reproduction command supplied.

P10

Pin J. Final captured description/code.
Centralizes CE builder-method lookup in a per-expression dictionary, routes method-presence checks through it and adds missing-member diagnostics tests.
Lookup changes AtMostOneResult to AllResults and substitutes the builder range for per-call lookup ranges.
Description includes a Release net10.0/osx-arm64 build log and two passing component tests.
No before/after performance measurements, workload or repetitions are supplied; build duration is not a comparative benchmark.
No explicit elapsed-time speedup is claimed.

P11

Pin K. Final captured description/code.
Claim: share imported project CCUs rather than pickle/unpickle each consumer.
Solution | projects | before retained MB | after retained MB
ReSharper.FSharp | 10 | 385.94 | 304.64
Fantomas | 8 | 350.42 | 275.79
FSharp.Compiler.Service | 14 | 879.43 | 818.39
FsToolkit.ErrorHandling | 8 | 87.77 | 69.36
Oxpecker | 16 | 131.67 | 114.24
Prime | 5 | 112.22 | 97.93
IcedTasks | 7 | 96.97 | 92.37
consoleapp | 1 | 29.38 | 29.37
Author says handover alone moves the FCS solution by 2 MB, not 61 MB; most saving comes from dropping language version from the import reuse key.
Checker creation flags changed relative to earlier numbers; exact flags, repetitions/error, environment and commands absent.
Final code creates unresolved thunks for names absent from the consumer; captured description still says missing names keep the producer's CCU.
Final diff includes sharing/language-setting/race changes and tests for concurrent consumers, diagnostics and imported surfaces.

P12

Pin L. Final captured description/code. Claim: avoid guarded-or-pattern compilation blowup.
End-to-end compilation with --optimize+; author states about 2.4 s fixed startup.
N | before time / image size | after time / image size
6 | 3.7 s / 8 KB | 2.6 s / 8 KB, identical IL
8 | 2.5 s / 17 KB | 3.8 s / 12 KB
12 | 3.2 s / 183 KB | 2.5 s / 25 KB
16 | 9.6 s / 2.9 MB | 2.8 s / 61 KB
20 | >90 s, did not finish | 3.3 s / 126 KB
24 | stack overflow | 3.9 s / 221 KB
Emitted-runtime shape: promoted --optimize+ uses static calls/no allocation; --optimize- uses a local FSharpFunc closure and one call/allocation per shared residual.
Byref-like results are not promoted. Ordinary nonpromoted matches reportedly keep identical deterministic IL; promoted quotations intentionally expose joinThunk.
Author reports matching active-pattern evaluation counts for two inputs at N=6/8/16. Tests assert execution, guard bindings, byref/rethrow and quotation behavior.
No generated-program elapsed-runtime benchmark, repetitions/error, environment or exact benchmark compiler SHAs supplied.

P13

Pin M. Final captured description/code. Claim: share imported assembly CCUs across projects.
Real whole-solution options graph; all projects checked and held.
Solution | projects | retained off MB | retained on MB
ReSharper.FSharp | 10 | 1547.2 | 1002.2
Oxpecker | 16 | 226.8 | 190.4
FsToolkit.ErrorHandling | 8 | 218.3 | 206.7
IcedTasks | 7 | 194.6 | 184.5
Fantomas | 8 | 665.7 | 658.3
Prime | 5 | 215.1 | 212.9
FCS.slnx | 14 | 2444.9 | 2443.7
consoleapp | 1 | 30.9 | 30.9
Warm ParseAndCheckProject time: median 9 samples across processes, rotated order.
ReSharper 13.28 -> 12.10 s (-8.9%); Fantomas 3.88 -> 3.90 s (+0.7%, author says inside spread).
Retained repetition count, numerical time spread, raw runs, hardware/SDK and exact benchmark commit/command mapping not supplied.
Final code uses typed dependency-closure/configuration keys, weak entries and per-entry strong dependency ownership; default on with opt-out.

S01

Request-required synthetic control. No real PR, commits or human source.
Description: "Faster."
No benchmark numbers, workload, configuration, environment or code evidence supplied.

S02

Request-required synthetic control. No real PR or human source.
Claim limited to generated closure count for one unchanged input/flags.
Supplied before/after IL inventories show 12 -> 0 generated closure classes.
Supplied executed assertions cover the input's values, evaluation order and exceptions.
No runtime-speed claim or runtime benchmark; no new retained state.

S03

Request-required synthetic control, not real measured data.
Repeated identical requests: median time 100 -> 80 ms, 30 runs, spread 1 ms on each side.
Changed code appends every request graph to a process-rooted list; no removal, cap or release path.
Post-full-GC retained MB for 100/200/400 requests: baseline 20/20/20; changed 120/220/420.
All request objects are otherwise dead; heap-root trace points to the list.

S04

Request-required synthetic control. No real PR.
Only a spelling correction in a Markdown guide; diff contains no code, configuration, generated files or performance claim.

S05

Request-required synthetic control, not real measured data.
Claim: faster compilation, same input and Release binaries.
30 comparable fresh-process runs per side: baseline mean 100.0 ms, 95% CI [98,102]; changed mean 99.8 ms, 95% CI [97.8,101.8].
No further distribution or practical-workload benefit evidence is supplied.

S06

Request-required synthetic control, not real measured data.
Claim: allocation improvement, not speed.
Same input/flags and machine; 30 runs on identified baseline/changed binaries, executable command and raw runs supplied.
allocated B/op | baseline 256 (all runs) | changed 128 (all runs)
elapsed ns/op | baseline 100 +/- 2 | changed 100 +/- 2
post-release retained bytes and GC pause distribution do not regress.
Supplied behavior assertions pass; no ordering/API/generated-output change.

S07

Request-required synthetic control for generated-program runtime, not a real benchmark run.
Fixture binaries runtime-base/runtime-changed use identical F# input, Release optimization and runtime.
The stipulated benchmark report identifies Windows x64, .NET 11, the compiler binary hashes and "fixture-run --input range-32767 --repeat 30" with all raw results.
Same 32,767-element input: baseline 280 us mean (95% CI 275-285); changed 80 us (95% CI 78-82); allocated 393,680 -> 131,088 B/op.
Generated IL intentionally differs. Supplied runtime assertions preserve result, evaluation order and exceptions, including empty and singleton input.
The supplied memory/GC checks show no relevant regression. No compiler-build speedup is claimed.

Frozen expectation matrix

Human URLs below are evaluator-only. Context from unverified user-account prose supplies a request-required control grounded in actual code, not a human-derived oracle. No PR lifecycle state is passed to the candidate as a shortcut.

Coverage and limits

The cases separate reported real improvements from reproduced results: none of these benchmarks was rerun here. Paired versions of #20348, #20349 and #20353 stay within FAMILY-self-profile; #20352/#20363 stay within FAMILY-inline-copy. Their pairs are not independent family votes or holdout coverage.

No verified human-derived precedent is claimed for a numerical significance threshold, thread-pool/starvation diagnosis, peak-memory protocol or comprehensive API/compatibility matrix. These remain conditional request-required rules, not omissions. No unverified/model-associated review is used to manufacture real regression/rejection coverage. Synthetic controls cover the specified decision boundaries, not untouched real-world holdout performance.

Additional runtime-source search: on 2026-09-22, gh api --method GET search/issues --paginate --slurp exhausted these exact queries, all with per_page=100 and incomplete_results=false: repo:dotnet/fsharp is:pr created:2026-04-22..2026-09-22 "ns" in:body (4), the same with "BenchmarkDotNet" (1), and with "runtime" (77), one page each. New non-bot descriptions #20500/#20007/#19672/#19713/#19714 concern correctness/docs/platform support; #20605 fixes concurrency in a test-only runtime builder but supplies no comparative runtime measurements. Dependency-update bots supply no human measurement evidence. Previously inventoried/reserved descriptions were not used for new answers, and no reserved feedback was opened. No new unreserved measured-runtime case was established; DEV-20 fills that positive boundary honestly rather than consuming a holdout.

Local audit

Candidate results remain not run (RED expectation frozen) regardless of corpus-audit success. The creation commit is the commit first adding this file (git log --diff-filter=A -- docs\performance-improvement-review\validation.md); the parent revision and RED recording time above deliberately do not masquerade as a self-referential commit hash.

Packet hashes

Historical code-only diff hashes: C e363eb3317a2d35e1cbf18ac207fd4f018f0febd81029fd8ea3f5487c1fec3cd; F 4586d191ad39d6e3c676e747acdf42b2bcb2632b4f857381f0e42ca86cc72351; H 6a117c9153590f23f4edbf86031b0f0de58842bda227efe5d4bd223de8d13da2. The remaining final diff hashes are in the corresponding corpus records/supplement. These were recomputed from Git, not inferred from final PR file lists.

Executed corpus checks

Checks below concern sources and frozen fixtures, not candidate performance. Temporary collectors/checkers and raw responses remain in session 28a5997c-c817-48a9-8f00-4a45e40edbf0, files, not in the repository.

Pre-freeze correction trail (no candidate executed): P02 v2 corrects the author's one-hunk shorthand to two diff hunks in one function; its superseded draft hash was b1db180a161b316c68cccadbcba9ad77e6c26ac355e10b28d3f5c3bf3f36d392. DEV-03 v2 replaces the draft's conditional needs evidence oracle with the named not applicable subcheck: P03 never defended manual loops as necessary for speed, and the actual reviewer made a tentative non-gating suggestion. P03's input/hash is unchanged. These are source-validation corrections before initial commit, not adjustments to make a candidate pass. Future input/oracle changes require another explicit version.

GREEN and post-refactor replay

Executed 2026-09-22, from prerequisite commit 1eeae24016d (RED creation revision/time above unchanged). The implemented agent supplements the existing reviewer; no compiler, benchmark, build, API or test implementation changed. No benchmark or compiler test was executed. The input remains RED-1, P02 v2, DEV-03 v2: all 20 packet hashes, 13 historical base/head pairs and 20 oracle rows were checked against the prerequisite commit and remain unchanged.

Invocation and isolation

The session's callable custom-agent catalog did not contain performance-improvement-review. All runs used the explicitly permitted isolated general-purpose task fallback, loading the complete saved candidate instructions; named-agent discovery is not claimed. Runtime identity: GitHub Copilot CLI 1.0.87, GPT-6 Astra / gpt-6-astra, inherited runtime preference with no model override. Each invocation had a fresh context and one case, never a batch of cases or a prior review.

Each reviewer received the exact candidate snapshot, a rule/metric extract retaining rule IDs, conditional requirements and human/request provenance but excluding historical discussion/resolutions, its exact fenced packet, sanitized source-pin coordinates, and the complete pinned Git diff. Reviewers inspected surrounding implementation/tests/signatures with git show at those pins. Synthetic artifacts were explicitly stipulated inputs, not invented real executions. Reviewers were prohibited from reading this matrix, the corpus, full evidence source discussion, other cases/results, later fixes, live PR pages or holdout feedback. Their only write was the review transcript in session storage; there were no GitHub writes or repository changes by reviewers.

Verbose transcripts, candidate snapshots, extracted inputs, full diffs, packet/diff hashes and audit JSON remain under session cf1b8168-aaf3-4d7a-93ad-058713c498d3, files\<run>\DEV-XX\. Task names are <run>-NN; the single output-format retry is refactor-final-09-retry. Repository summaries below preserve the actual conclusions and evidence requests without depending solely on that archive.

There were 79 actual invocations, including failed/partial calibration and the formatting retry, not 79 independent cases. Run starts are packet/candidate freeze timestamps; ends are saved-output timestamps, not benchmark durations. G below means C1's complete green-final run; R means C3's complete refactor-final run, with the valid DEV-09 retry. The C3 hash is the candidate reserved for subsequent blind evaluation; no reserved feedback was opened to tune it.

Calibration changes and evaluator decisions

Additional risks were evaluated against code, not against approval/closure status or later fixes:

• S1, DEV-06/07: infos.fs's optional/default-less parameter path imports with the current range, but the new comparer ignores range. MemoizationTable stores a lazy before forcing it (Utilities/illib.fs:984-994); DiagnosticsLogger.fs:481-498 reports then throws ReportedError, which recovery suppresses. Base import caching stores only successful values (Checking/import.fs:161-167). The reviewers traced call-site lookup before argument processing and the second same-map/method/view lookup: a failed lazy can replay without the second diagnostic. This is a substantiated error-path scenario, beyond merely asking whether dropping range is safe. See original pinned cache and final pinned cache. No successful-program miscompile or executed failure is asserted.
• S2, DEV-11: pinned import publication initializes pending publication only for the claim owner, while dependency binding later runs only for contexts with pending entries (2908-2917). G traced a losing concurrent importer with an uninitialized resolver and non-framework dependency; R traced cancellation/failure between acquisition and publication, leaving a claim that later losers cannot publish. The evaluator confirmed both control-flow paths, including per-build rather than batch cleanup. Preconditions and lack of execution are stated; these are not claims of a measured leak/hang. Which additional scenario a run selects is variable.
• S3, DEV-13: pinned key construction hashes the reachable set, leaves CcuName=None for IL, and omits root identity. Two eligible cyclic IL roots have the same set/key. Last-writer publication (599-600) followed by another project's cache hit (2288) can return the wrong assembly CCU; RegisterCcu uses that returned assembly name (1503-1510). Eligibility does not exclude the cycle. This concrete source-derived collision supports risk, without claiming an executed repro.

These extensions do not turn the source corpus into a set of verified historical defects or human objections. The expected missing-evidence checks remain required, and were still found. C1/C3 did not invent blockers in the no-data, structural-only, noise, docs-only or allocation-only controls.

Durable actual-result matrix

Every G/R review contains the eight ordered groups and separates proof, unknowns and action. The entries below summarize the scored subchecks, not blanket verdicts for entire PRs. G/R denotes agreement on the described conclusion; differences are explicit. The final column identifies candidate/run and the first 12 hex characters of the actual transcript SHA-256 for audit correlation.

Compaction and local checks

C1 had 11,290 bytes / 1,397 whitespace-delimited words; C3 has 9,634 bytes / 1,169 words (14.7% fewer bytes, 16.3% fewer words). The duplicate gate list/output-group list became one ordered guidance table. Detailed historical provenance stays in the linked evidence/corpus documents. No metric family, metadata requirement, read-only boundary, behavior/lifetime gate, or outcome definition was removed. C2's producer-contract regression was measured and corrected, not assumed away.

Blind holdout freeze

Recorded 2026-09-22T13:22:53Z, before opening reserved feedback. Candidate C3 SHA-256 is 4c5d4bd0000fac52c0e1e5de9825e95891dd364c8e301200cd7952406c667954, at repository revision ac7bd61c998f0bade4aeaa00e676b87bedb6397e. The evidence table remains at 1eeae24016db78d6236636017992f5ff655c67be, file SHA-256 b8e0a424f05c3fdc91f3d9aef900be56142537daaa469811122618ca0bb01ec3. The historical evidence cutoff remains 2026-09-22T09:28:34Z; retrieval time does not extend it.

The reserve and category assignments were committed in ae294143e5890033fcb717956050a91a6f2ed6e8, before drafting the candidate. The RED correction in 1eeae24016d moved the entire exposed self-profile family to development. The 20 development packets and their expectations/runs above are preserved. The 94 reserved rows comprise 82 performance candidates, five earlier/context rows and seven real applicability controls; they are not 94 executed holdouts. Neither the inline-copy nor the IL-import-retention nor the self-profile family is eligible for independent holdout credit.

The current runtime catalog lists the named performance-improvement-review agent. Actual invocation and source acquisition results will be recorded separately; catalog presence and this freeze are not a test pass.

Isolation correction before holdout execution: the required complement-boundary document, expert-reviewer.md, already contains #20388's precise covariant-empty-array failure and repro. Git history verifies that example at the pre-drafting checkout; it was introduced by fce51f6d26358cb5c2431f84c9884aceba7bb154 (#20511), on 2026-09-11. Its being an existing-agent example rather than human evidence does not make the example unseen. Conservatively promote all of FAMILY-core-empty-array to development; do not credit #20388 as an independent holdout. No C3 rule changed in response to this discovery. Other selected untouched families must establish behavior/API-neutral coverage.

Final development regression

Actually executed: 20 fresh named performance-improvement-review task invocations, final-01 through final-20. Packet freeze: 2026-09-22T13:24:57Z; first completed output: 13:26:17Z (DEV-17); last: 13:55:55Z. Runtime: Copilot CLI 1.0.87, GPT-6 Astra (gpt-6-astra), inherited configuration, no model override. This is not the earlier fallback run. C3, all 20 RED packet hashes and all historical diffs are unchanged. Every invocation received only C3, the same sanitized rule/metric extract, its own input and pinned diff/source access; no oracle, other review, reserved feedback or later fix.

Raw transcripts and machine-readable identities are outside the repository in session 36233952-cdbf-4056-984e-1fe93e32d7a3, files\final-regression\DEV-XX\output.md. The table preserves scored actual findings; transcript SHA-256 prefixes identify exact outputs. Source pins, full input hashes, expected subchecks and withheld URLs remain in the frozen matrix above. No tests or benchmarks in the PRs were executed; these are reviewer executions.

Evaluator score: 20/20 cases meet their frozen semantic acceptance checks, zero missed required concerns, zero unsupported blocker findings. This is a small development regression suite, not independent holdout precision/recall. The outputs contain 224 scoped table rows: 85 pass, 116 needs evidence, 17 not applicable, six risk/blocker. Six risk rows represent five case-level findings in DEV-06/07/11/13/16; DEV-16's final recommendation repeats its G5 finding. These reduce to four mechanisms (S1 in two revisions, S2, S3, synthetic retention), not six independent bugs.

The evaluator checked the new explicit caller/precondition/consequence traces against the S1-S3 mechanisms documented above. The new S1 example is a nongeneric declaring type's generic method with an optional parameter but no metadata constant; it distinguishes the existing first-call import error from the new suppression on a cached failure. S2 states cancellation after successful construction but before publication, and the observable loss of sharing on later retries until clearing, not a fabricated hang or leaked CCU graph. S3 states eligible mutually referencing IL roots and loss of one assembly's type environment. No oracle was revised and C3 was not repaired.

All 20 saved tables actually passed the ordered full-name G1-G8, five-column, nonempty proof/unknown/action and exact-outcome checks. Diff hashes matched the frozen inputs. The earlier 79-invocation archive was also re-audited successfully, including its preserved rejected C2 formatting failure; no old output was relabeled as this run.

Maintenance procedure

Freeze a new date window and exact cutoff before discovery; retain source bodies, pagination totals/cursors, edit/provenance metadata and base/head/review revisions. Normalize related attempts and stacks by PR family, quarantine automated or uncertain prose, and reserve fresh families before drafting. Add the input and source-backed oracle before changing a rule. Freeze candidate/evidence/input hashes, execute each case in an oracle-free context, and record actual output, invocation identity and evaluator misses/unsupported findings. A rule change prompted by a holdout promotes that family to development and requires untouched replacements. Rerun the complete cases, check links/discovery and split/coverage accounting, and retain raw transcripts outside the repository. Do not convert unavailable categories or synthetic controls into real-human holdout credit.

C3 blind execution and retained failure

The first blind execution is not an acceptance pass. On 2026-09-22, ten fresh named performance-improvement-review invocations (blind-c3-01 through blind-c3-10) used the unchanged C3 and sanitized rule extract above. Runtime: Copilot CLI 1.0.87, GPT-6 Astra (gpt-6-astra), inherited configuration, no override. Outputs were saved from 14:23:10Z through 14:29:29Z in session c8f07943-58a7-4cdd-9a76-ce0b8d398079, files\blind-c3-NN.md. Each context received one historical packet, full diff and pinned source access; no oracle, source report, withheld feedback, later fix or other result. The frozen evaluator oracle predates these runs: SHA-256 ddfc494f450358cf527cdbf08c45066d9fca111efc84aa2aa74b8399400f59dd, in the preceding session's files\holdout-study\oracles.md.

Independence correction: HOLD-01/02 (#20388) are development regressions, not holdouts. Its final ancillary closure-characterization change also connects FAMILY-inline-closures; conservatively treat HOLD-03 (#20367) and that entire family as development. This overrides the earlier source collector's eight-connected-family holdout count. The initial execution therefore contains three development cases in one connected family, and seven blind cases in seven other families. Account-authored but unverified prose and explicitly model-attributed feedback do not become human precedent merely because the reviewer agrees.

Frozen source expectations

These are compact transcriptions of the pre-execution oracle, not expectations inferred from output. Source composition remains uncertain unless explicitly model-attributed below. All ten cases are real-PR request-origin controls, not ten verified human objections. H identifies the exact input revision in the identity table below.

Actual outputs and evaluator comparison

All inputs below use C3 and the frozen pre-holdout rules. Packets live in preceding session 36233952-cdbf-4056-984e-1fe93e32d7a3, files\holdout-study\inputs\HOLD-NN; suite.json additionally hashes every source file and coordinate manifest. HOLD-01..09 were frozen at 2026-09-22T13:38:40Z, HOLD-10 at 13:55:31Z. HOLD-08's subsequently supplied pinned TestSplit context is recorded in the pre-execution manifest; no oracle text was supplied. Historical diffs are reproducible from the merge base/head below; compact claims and exact required evidence are preserved above rather than relying on archive paths alone.

The evaluator read the outputs separately from reviewer execution. Exact transcript hash prefixes below refer to unedited saved output, not hand-authored replacements.

The first-run semantic score is 9/10 matches, one miss; among initially eligible independent cases, 6/7 matches, one miss. This is not a precision/recall estimate or final acceptance. The seven blind families cannot be silently relabeled as fresh if subsequent tuning uses their results. No verified-human-composition credit is added to the evidence rules by this run.

Additional finding verification: at HOLD-07's head, an F# options request constructs a C# dependency's delayed reference before the parent can bail out (FSharpProjectOptionsManager.fs:416-439,475-479). This inserts an emit-cache entry without a C# options-cache entry or successful emission. ProjectRemoved posts ClearOptions, but removal is guarded by options-cache membership (:588-596,653-659); the sweep enumerates only lastSuccessfulCompilations (:479-505). Thus ordinary removal/sweeps can leave the strong entry rooted. Global/solution clearing releases it; a later successful delayed callback could also make it eligible for sweeping. This supports the narrow cleanup finding, not an observed leak size, OOM, or assertion every retained entry still owns a compilation. The independent source evaluator did not read the candidate output.

The ten saved outputs passed actual ordered/full-name G1-G8 and five-column/nonempty-proof/unknown/action checks, using only the four permitted outcome strings: 106 scoped rows = 42 pass + 46 needs evidence + 16 not applicable + 2 risk/blocker. The two risk rows cover covariance and historical-version retention, with the additional supported removal subcase; zero unsupported blocker findings were established. The ten input file-set hashes and all twenty preceding final-regression transcript hashes/shape checks also passed. Output length ranges from 1,017 to 2,110 whitespace-delimited words; tables are structured but not uniformly terse. Shape checks do not erase the semantic miss.

Coverage and remaining acceptance gaps

The conditional contract still includes time; total/peak/retained memory; allocations; closures/generated artifacts; CPU; GC; threads; startup/build costs; behavior/API/compatibility; lifetime; surgical isolation and reviewability. C3 requests before/after units, commits/configurations, workloads/input sizes, repetitions, meaningful error/spread, environment/tooling and commands/links, with adjacent metadata permitted. Actual controls show these are conditional, not a universal metric matrix. No production build, test run, formatting sweep or release-note entry is required for these documentation/prompt paths.

C4 repair and replacement holdout

C3's HOLD-09 output is retained above. The sole prompt repair adds a conditional PERF-03 sentence: compare representative supported capability-dependent fast/fallback paths on the same runtime; another runtime's results do not establish fallback cost. No metric, outcome, group, behavior gate or provenance label was removed. The old sanitized rule extract is unchanged; the full C4 instructions supply this additional requirement.

The source evaluator independently confirmed the original oracle, without reading the reviewer output: T-Gro's request, 2025-08-14T12:42:51Z, explicitly names .NET 9 with DOTNET_EnableHWIntrinsic=0. At final 7e5c5d16de76ecc0b87f8da9bed6b07438d65573, array.fs:1581-1614 and seq.fs:1467-1500 choose the old loop on Framework but LINQ on modern runtimes. Disabling intrinsics on .NET 9 therefore does not select the Framework fallback. The author's later assertion refers to a .NET Framework 4.8.1 table, not the requested modern scalar path. The oracle was not relaxed; source reinspection confirms a candidate miss rather than an oracle mistake. These are earlier, composition-unverified sources, not additional human-derived votes.

No holdout laundering: #18509 and all of FAMILY-vector-sum become development. HOLD-01/02/03 remain exposed development. HOLD-04/05/06/07/08/10 are retained evaluation cases whose findings did not tune C4; their C4 runs are explicitly replays, not first-seen holdouts. A new family, FAMILY-string-spans, supplies the replacement for the affected conditional fast/fallback generated-program measurement category. It does not supply a second independent SIMD-disabled example; that narrower subcategory remains unavailable in the source-screened shortlist.

REPL-01: independent source oracle and actual result

Metadata-only reservation of #20358 predates candidate drafting. The source evaluator selected it from the predeclared #20358/#19804/#20275 shortlist based on the code/claim, not candidate behavior. Its string-construction mechanism is distinct from vector sum, empty-array sharing and inline closures. A common broad optimization umbrella issue does not make those changes duplicate mechanisms; that link was disclosed to the evaluator and omitted from reviewer navigation. No REPL-01 feedback or expected answer informed C4.

Input text SHA-256: 9fec7df157b5c59c3c20abf10acfb304980477613ebe993bacc09b7a13b322fd; complete diff SHA-256: 6efc6f9d9c230e223ad89b91a80270ccb0997de113528d1db0b17aec83a4044b; evaluator-only oracle SHA-256: e8a0f40ff5f622ef3995ca0554ff07baca5299f9476f8c35e155799e4d08ebe6. Canonical UTF-8/LF input/oracle bytes match the independently prepared payloads; initial Windows CRLF materialization was normalized before execution, without changing content. The source bundle and all fourteen reviewer-member hashes were frozen at 2026-09-22T15:01:02Z, files\replacement-study\freeze.json; the oracle is stored separately. The bundle contains complete base/head changed files, unchanged signatures, NativePtr implementation/signature, existing String tests and build/test asset routing, not discussions.

Actually executed: fresh named performance-improvement-review, task c4-repl-01, same CLI/model as the other runs; only C4, the sanitized rule extract, the single frozen reviewer bundle and access to its pinned Git objects were supplied. Saved output SHA-256 6a3d04e9709e0e13db50bd9cb607ef1444addd663450ea44263b7fa24fe0c304, files\c4-regression\REPL-01.md.

Actual finding: the reviewer identifies the exact temporary-buffer eliminations and unchanged larger-filter/short-replicate paths; requests B/op by branch and same-runtime identified binaries, warmup and repeated-run metadata. It catches the NETSTANDARD2_1_OR_GREATER-only guard, separate shipped-net asset and modern unit-test routing, distinguishing surface-area loading from behavioral execution. It requests threshold/selectivity/count tests, callback/overflow proof and fixed 1,024-byte stack-budget justification, while crediting existing null/empty/side-effect/Unicode tests and source-level lifetime/order invariants. It explicitly excludes hardware-intrinsic toggles, compiler timing and unrelated concurrency metrics. Final recommendation: evidence is insufficient; no blocker.

Evaluator result: match on every selected oracle concern, zero misses, zero unsupported blockers; one fresh case in one fresh family, not a broad precision/recall result. The actual table has fourteen scoped rows: five pass, seven needs evidence, two not applicable; ordered full group names, exact outcomes and all five evidence columns pass. This proves the broader conditional path/asset rule on a new real family, not the unavailable second SIMD-specific case.

C4 complete replay: retained development regression

C4 was actually run in thirty fresh named contexts on DEV-01..20 and HOLD-01..10, followed by one formatting-only retry and REPL-01: 32 invocations, 31 distinct cases. Outputs completed from 2026-09-22T14:45:36Z through 15:06:06Z. Inputs and source pins were unchanged. A separate evaluator read every actual output and checked added blockers against pinned sources; it did not invoke reviewers or rewrite expectations.

The original thirty cases scored 29/30 semantically, 28/30 including formatting. The sole semantic miss is HOLD-01, already-exposed development: C4 notices the covariant array's runtime type changes but asks whether compatibility policy covers it, omitting the formerly valid downcast's InvalidCastException and the required risk/blocker classification. Original transcript SHA-256 e77e0636b313d5c8fcfcdfc400ae9a7fa744f84d979b23ccd4ea9cf9d37bdc9c remains unedited. This is not an oracle ambiguity: the evaluator checked base/head clone/empty-array branches and the unchanged public copy contract. Intentional sharing does not establish permission for the separate runtime-type failure.

DEV-03's original semantics matched, but an unescaped array-comprehension pipe made one Markdown row seven cells. Its original SHA-256 is 94929ba57689a46ace5860f998386f54817cee62559c0c20e302fb3223cd7bb1. Fresh same-input/C4 task c4-dev-03-format-retry received only a generic pipe-escaping reminder, not prior findings or the oracle. Its thirteen-row output passes format and semantics, SHA-256 c701caa348ed7def763acf977f79cc57a78e92a462674fbff73ea6df0b131d99. The retry does not erase the original failure or count as a new case.

C4's HOLD-09 now explicitly requests the same modern runtime with intrinsics enabled/disabled, credits the supplied tables and does not substitute Framework data; SHA-256 e39250b5c8824679f52a89e4ebc6d37d9a6a9bd9db13a03c4a304ee6e24fee11. All other required DEV/HOLD checks matched their frozen semantic expectations, including both verified-human development rules, supported narrow wins, no-claim controls and resolved concerns. No unsupported blocker was established. With the formatting retry and successful REPL-01, C4 still scored 30/31 cases, not acceptance. Full original output hashes and concise per-case findings are retained in session files\c4-scoring.md; the final-candidate table below records the subsequent full replay separately.

C5 final-candidate freeze

C5 adds only the PERF-06/12 classification clarification: a source-established failure for previously valid callers is material risk, not merely an unspecified-policy question; distinguish explicitly intended behavior from separate collateral effects. The repair responds to development HOLD-01, not REPL-01. No case oracle, input, metric rule or source attribution changed. C4's failure and the successful but limited replacement result remain above.

Frozen at 2026-09-22T15:09:59Z, repository ac7bd61c998f0bade4aeaa00e676b87bedb6397e plus hashed working-tree changes: candidate SHA-256 65207b555cd16d94fb34b7c8186dbabac41717b516b25ed0f668d261b2c7049b; evidence SHA-256 5d88a405b2eae701c09a074679650b14dfd5ea5607562449e6a268519ac91e9e; sanitized rule extract unchanged at 60874a1849722ee4369a839d8a431565069f98aa7588f044582ccf4f3a485e14.

Hash encoding: candidate/evidence and transcript identities hash the exact saved bytes, including Windows CRLF. Git stores the prompt with LF; do not mistake that normalization for a prompt change. The LF candidate SHA-256 values are C3 3fd85a737e95e49354a1e83c7caec1d05a99aaeff25b47277f601094c215e571, C4 097018ca8a565aba01d36a893df712a55dec68f3294941318cf06e7092816a92, and C5 7bfed4f87f5ccff6b2a33c5a31cf35544b5bd2f2f78a433594daad0a965aceb6. C5 evidence's LF hash is b854b1d68c534fab1c488c43b79954887aeb234bfde91313325876bb9ac52510. Earlier fenced RED packets and the separately specified REPL/MIX input/oracle hashes use their explicitly documented UTF-8/LF convention.

Actual full execution: 31 fresh named tasks, c5-dev-01..c5-dev-20, c5-hold-01..c5-hold-10, and c5-repl-01, all with one packet and the same isolation/read-only boundary. Runtime remained Copilot CLI 1.0.87 / GPT-6 Astra (gpt-6-astra), inherited without override. Generic pipe escaping was reminded uniformly; no previous substantive answer was supplied. Outputs completed 15:12:24Z through 15:23:37Z, archived in current session files\c5-regression. This is a new complete run, not C4 results renamed.

All 31 output structures passed locally on first attempt: 327 scoped rows = 121 pass + 165 needs evidence + 32 not applicable + 9 risk/blocker; ordered full G1-G8 names, five nonempty columns and exact outcomes. Output lengths are 585-1,821 whitespace-delimited words. Input/archived-result identities were rechecked. These structure counts are separate from semantic scoring below.

Final semantic replay results

31/31 frozen cases match, zero missed required concerns, zero established unsupported blocker findings. This bounded result is not universal PR approval, review precision/recall, benchmark reproduction or complete-category coverage. The set is 24 real historical inputs from 20 distinct PRs plus seven labeled synthetic controls. Twenty-four cases are development/regression controls (including the seven synthetic cases); seven cases remain in evaluation families. Six of those seven were first run on C3, and REPL-01 was first run on C4. C5 is explicitly a complete replay, not 31 new independent holdouts.

The table preserves concise actual findings from both full repaired-candidate runs. Inputs, source URLs, required evidence and prohibited findings are the unchanged DEV/HOLD matrices and REPL-01 oracle above. SHA columns are prefixes of exact unedited transcript hashes; full hashes remain in c4-scoring.md / c5-scoring.md and their transcript files outside the repository. C4's DEV-03 prefix identifies its original formatting failure; the separately recorded retry is not hidden.

Scoring correction, not oracle relaxation: the evaluator initially recorded C5 as 30/31, treating HOLD-02's freshness-risk extension as prohibited. Reinspection of the original pre-execution oracle established that its pass is specifically for repaired covariance/documentation. It prohibits repeating the old covariance defect or claiming unchanged freshness docs, not every additional source-supported compatibility finding. C5 explicitly says the covariance mechanism is resolved and sharing is intentional/disclosed. Base copy clones two distinct empty arrays; final copy returns the same singleton, so the second addition as a key to one live ConditionalWeakTable changes from success to a duplicate-key exception. That constructed caller failure is source-supported, not an executed field report. New documentation establishes the intended sharing contract, not a blanket waiver for existing callers.

The initial erroneous score and this correction are retained in c5-scoring.md/.json; candidate, original oracle and every output are unchanged, and no retry or independent-case credit was added. The compact transcription above initially said "No stale covariance/freshness blocker," which was too broad; it now accurately states the original oracle's prohibition on repeating the covariance defect or claiming unchanged freshness documentation. This is a source-backed transcription/scoring correction, not a new oracle. Preservation is a supported remedy; the source does not establish that it is the only permissible release decision or a universal policy veto. The combined risk/blocker category includes this substantiated compatibility risk without inventing such policy. This adjudication follows the same scoped-check/independently-verified-extension rule used for S1-S3, rather than changing an oracle to fit a result.

The nine risk rows occur in eight cases. S1 repeats across two revisions; S2 uses the already-verified losing-importer path; S3 is separate from the conditional ABI finding. HOLD-07 includes historical retention and removal-cleanup subcases. No row count is presented as an independent-bug or human-vote count. Added ABI risk is scoped to an old compiled Create caller actually binding the new assembly without its old method signature, not an assertion that every deployment breaks or FCS has a universal cross-version compatibility guarantee. All source-derived failures remain distinct from observed executions.

MIX-01: full-evidence mixed-scope extension

HOLD-04 is a useful mixed-file no-claim control, not the stronger test of attributing a real performance claim separately from cleanup. This additional case closes that distinction without changing C5. #20455 was reserved as mixed-scope before drafting; its named-record cleanup and option-to-voption allocation mechanism are separable within the editor subsystem. It belongs to the existing FAMILY-vs-work-reuse, not a new independent family. The case was first executed only after C5 was frozen.

Pre-execution packet correction: the first unexecuted draft omitted the author benchmark. Before freezing or running it, the evaluator restored that entire available measurement artifact and adjusted the oracle to credit it. V1 input/oracle hashes 11306c2c00a6811dd5fa09bf7a897fe4f5c096ec577895bb741a5541ffa90ba4 / 5c6e8b9236e51c598724863548f87711b3c886d4cf62e607dc8b5ffcdae0aaa3 remain in the preserved evaluator transcript; neither was ever run. Withheld review objections are not a reason to hide author measurements. This correction was independent of candidate output.

Executed v2 was frozen at 2026-09-22T16:14:56Z, files\mixed-study\freeze.json: input SHA-256 a20d2a38438fd2c05cf0c9d37d1565db7cdd7cdf1e837bb50a009cedf9f6a77b; diff 0465ec192f53352ae798550282e04cbe65aba3230ef4909adee915e064c9d5c6; complete 25,152-byte author benchmark 2a218f35a2f7c2dc6257325fa1ce40ae4c2896407e9f23577604bf7c33f9016f; separate evaluator oracle 7d273f83ae8c05bb89b7dfcbc866dcb1736986add69753218ef0faf7e66debd2. All eight reviewer payloads are hashed; no feedback/history/oracle directory was exposed.

Actual named run: c5-mix-01, same CLI/model, isolated input and pinned-source-only boundary as the 31-case replay. Output files\c5-regression\MIX-01.md, SHA-256 3d75dec0a93a7f7c0fbb5f5c0038c91f7c3a5dd85da259547d8dfcf778ff09f9. The reviewer gives G4 pass to the modeled 8 B/op (~2.1%) saving and G7 pass to supplied attribution/cleanup. It explicitly distinguishes the Framework 4.8.1 job from the net472 editor and secondary .NET 10 comparison, credits the supplied tables/harness/configuration, and does not claim the DEBUG host proves an unoptimized benchmark.

It traces mailbox serialization and the actual removal/insertion sites, so the 5.09x synthetic Update row is not called an observed IDE regression. It retains the 674.98->733.73 ns hit means (~8.7% increase) and the author's reported 20-25% second-run movement without inferring equality or a reliable regression. Specific needs evidence requests concern the missing repeat/results/provenance, timing-parity qualification, production frequency, and the harness's always-successful stub versus no-view/subscription-failure paths. Existing lifetime/control-flow mapping passes; targeted execution remains unverified, not a fabricated correctness failure. No whole-IDE trace, compiler-wide benchmark matrix or blocker is demanded.

Parent evaluator score: all six frozen oracle requirements matched, zero misses and zero unsupported blockers; the actual ten-row table passes shape with five pass and five needs evidence rows. The complete final C5 suite is therefore 32/32 semantic matches, 337 valid scoped rows (126 pass, 170 needs evidence, 32 not applicable, 9 risk/blocker), with no output retry on C5. It contains 25 real historical inputs from 21 PRs plus seven synthetic controls; eight evaluation cases are in seven families. MIX-01 adds a first-seen C5 case, not a new family or an extra vector-repair replacement. No new verified-human-composition precedent is claimed, and benchmark-only coverage remains unavailable.

BENCH-01: synthetic contract only, not real holdout coverage

The absent real benchmark-only category is not silently replaced. This separately labeled request-origin synthetic control tests the output contract without supplying any real-PR or human-precedent credit. It changes only an existing benchmark's [<Params(4096)>] to the four parameters below: empty, short, retained existing and larger input. No implementation, target, dependency or operation changes; no speedup, allocation reduction, real-world representativeness or statistical result is claimed.

open BenchmarkDotNet.Attributes

[<MemoryDiagnoser>]
type SumBenchmarks() =
    let mutable data: int[] = [||]

    [<Params(0, 128, 4096, 65536)>]
    member val Size = 0 with get, set

    [<GlobalSetup>]
    member this.Setup() =
        data <- Array.init this.Size (fun i -> i % 1024)

    [<Benchmark>]
    member _.Sum() =
        Array.sum data

The unchanged runner is stipulated as Release net10.0 / BenchmarkDotNet 0.15.8 with its ordinary generated harness and MemoryDiagnoser. A Dry smoke completion of all four parameters is stipulated synthetic data, not a run performed here and not statistical evidence. The separately frozen oracle requires recognition of benchmark-only scope, setup exclusion/result consumption and expanded boundaries; no product speedup/table/significance gate, fabricated resource defect or claim that Dry proves timing validity is allowed.

Freeze 2026-09-22T16:39:09Z: input SHA-256 ceb1e223d8e14585af8c8da9d059078cb46e47e2f3fd6f2e3c1d1d0327006de9; evaluator-only oracle 168231e15905c8306884a0d525737d22a2cf9131fef64a9584b216eb8e0f60b2 (exact UTF-8/CRLF bytes). Fresh named task c5-bench-01 received only C5, the sanitized rules and this one fixture, not the oracle or prior outputs. Saved output SHA-256 68f5b6bbc361e7f9234c30d26bf321b6ab8b45a291268cd7b092aa671a9d6589, files\c5-regression\BENCH-01.md.

Actual result: seven pass rows and G4 not applicable. The reviewer accepts benchmark input-coverage value, recognizes that the performance-table gate does not apply, credits setup exclusion/returned-result consumption and preserves the Dry/statistical distinction. It does not demand hardware, repetitions, whole-build/IDE metrics or an actual execution certificate for the stipulated coverage-only fixture. It explicitly endorses only broader benchmark coverage, not shipped performance. Parent evaluator: all required checks match, no unsupported finding.

Final executed set: 33/33 semantic matches, zero misses, zero established unsupported blockers; all 33 output structures pass. There are 345 scoped rows: 133 pass, 170 needs evidence, 33 not applicable, nine risk/blocker. The set contains 25 real historical inputs from 21 PRs and eight synthetic controls; eight evaluation cases remain in seven families. The real benchmark-only holdout count is still zero, and no new verified-human-composition source has been added. These bounded local successes do not certify unavailable real-category coverage.

Sprint-10 reconciliation and residual manifest

Bounded reconciliation complete, not unconditional reviewer acceptance. Starting/current HEAD is ac7bd61c998f0bade4aeaa00e676b87bedb6397e. The four starting dirty files (agent, corpus, evidence, validation) were intentional C5 work and remain unstaged; this sprint edits only the three documents. Process command lines/ancestry were checked before editing: no competing writer to this worktree was identified; other active implementors belonged to other worktrees and were left alone. No acquisition restart, new reviewer batch, compiler build/test, package/format sweep, staging, commit or GitHub write occurred. The original window remains 2026-04-22T00:00:00Z through 2026-09-22T09:28:34Z.

Frozen identities and archive coordinates

Archive root R: C:\Users\tomasgrosup\.config\daily-monitor\copilot-microsoft\session-state. A = R\36233952-cdbf-4056-984e-1fe93e32d7a3\files; B = R\c8f07943-58a7-4cdd-9a76-ce0b8d398079\files; S = R\295cd124-c927-411b-84e7-ada0988de119\files. Raw data/scripts/oracles/transcripts remain there, not in the repository. S retains exact starting snapshots, starting-identities.json, both shape reports, reconciliation.json, c5-run-identities.json and final ending-identities.json. The latter records the final validation-document digest externally to avoid a self-referential hash.

Exact hashes include saved CRLF; normalized hashes replace CRLF with LF before SHA-256, without other text changes. C5's actual sanitized extract B\c5-regression\rules.md remains 60874a1849722ee4369a839d8a431565069f98aa7588f044582ccf4f3a485e14. The effective decision-rule projection before/after reconciliation is 0c226f5ae36666f41d31644c16db4967bfa0f25ab8928d0091ddefd81a5ad7b8: UTF-8 canonical JSON (ensure_ascii=False, separators (',', ':')), keys rules then metrics; twelve stripped PERF rows projected to ID/rule/evidence/limits (columns 1/6/7/8), then nine metric rows projected to their first three columns. S\reconcile_artifacts.py defines this source/provenance-excluding projection; it is not a substitute for the full candidate/extract identity or a claim that new full-context source notes were model-tested.

Exact case inputs, original expectations and run identities

The manifest below enumerates all 33 cases without replacing their original detailed expectations. For every case, its actual run name is c5- plus the lowercase case ID, its unchanged output is B\c5-regression\<CASE>.md, and its result/hash is in the 31-case C5 table or the linked extension section. B's parent events.jsonl confirms all 33 launches/completions with Copilot CLI 1.0.87 / gpt-6-astra, no model override. S\c5-run-identities.json records exact tool-call IDs, timestamps, prompts and completion records, SHA-256 797f5c84b65543ddc1a3f76400c6563bada75f4888c73ffb8bcef4cd5cdbd0c8. The twenty DEV tasks launched 15:11:35.052Z, ten HOLD plus REPL at 15:18:44.206Z, MIX at 16:15:08.300Z, BENCH at 16:39:28.178Z on 2026-09-22. Parent tool-completion times are not child output-save times.

The 31-case score record hashes are B\c5-scoring.json 3496045f204d033d8424919c13cacc6fbba55aea2cb173c79adcd792fd2a29eb and .md 25f11aafa9d92f092cc5917fbdfbf441dd2f2edaf1e398f28c73c6ca72a8f3fe. They are not 33-case files: MIX/BENCH add the separately preserved parent scores above. The C3 HOLD-09 miss, C4 HOLD-01 miss, C4 DEV-03 formatting failure/retry and C5 HOLD-02 scoring correction all remain recorded; no oracle was changed to fit an output.

Only remaining or affected checks for sprint 11

No semantic repair or missing C5 case is identified by this reconciliation. Do not reacquire 94 PRs, rerun the 804-PR search, widen the window or create a real-benchmark-only requirement. Existing fixtures satisfy that applicability boundary; the original eight evaluation cases remain the executed subset, not unused replacements.

The precise unresolved acceptance item is unqualified verified-human grounding: the named envelopes establish participants, concerns and resolutions, not unaided composition. In particular #20352 comment 5422336205 has no review envelope and #20348 comments 3854812488/3860828838 have empty parent reviews 5021077157/5028138073; neither contains a composition attestation. No missing envelope can be fixed by another bulk search. A stronger retained claim would need source-specific attribution beyond these records; do not invent an external attestation policy or treat style/User status as proof.

Proposed provenance-only delta, not applied to the candidate here: replace its description's "verified human review precedents" with "source-attributed performance-review precedents"; replace its "verified rules/metric map" label with "rules/metric map"; replace "Only PERF-08/09 are human-derived + request-required; other gates are request-required. Preserve these labels." with "All gates are request-required. Source-attributed reviewer feedback and author rationale support narrow rules with the evidence document's composition limits; neither is verified by account type or replay." In the compiler-review skill, "verified evidence resources" should become "evidence and provenance resources." Apply the same attribution correction to any new sanitized extract, never overwrite the archived one. No PERF rule meaning, metric condition, outcome or oracle change is proposed.

The affected provenance sentinels are DEV-01/02/03/04 (PERF-08/09 scope, narrower alternative, optional syntax and resolved idiom). If sprint 11 changes candidate/extract wording, freeze both new identities before running those exact unchanged inputs; compare their original scoped outcomes and check that attribution uncertainty is not turned into a missing performance prerequisite. This is development replay, not new holdout credit. The added #20353/#20287/#20296/#19995 mappings explain already-required rules; DEV-06/07/13 are existing source-related controls, not newly failing cases, and #20287/#19995 have no dedicated executed packet. DEV-05's #20261 source-label correction (unsupported model association -> uncertain composition) likewise changes neither its input/oracle nor its human-evidence credit. There is no basis here for a new semantic rule or a blanket replay mandate. A later semantic edit must name its exact rule delta and affected cases before execution; no such edit is authorized by a passing old C5 score.

Sprint-11 bounded semantic validation

Accepted after input-isolation repair: C6-S's selected complete replay matches 36/36 frozen concerns, with zero unsupported findings. C6's original shared PERF-11 extract disclosed a later executable-test repair to historical pre-fix DEV-06; its score and artifacts remain historical, not clean-isolation acceptance. C6-S uses the unchanged candidate with a sanitized extract and actual new outputs, including strict-isolation replacements described below. The set covers 28 real historical inputs from 23 PRs and eight synthetic controls. Eight previously seen evaluation cases remain in seven families; no new independent holdout or verified-human-composition credit is claimed. Source-attributed feedback is usable with the recorded composition limits; replay does not authenticate authorship.

Archive T = R\b8acc633-d504-4744-b6c6-467dd7b37ea1\files, using R from archive coordinates. P = T\bounded-validation, F = T\c6-validation. Each has frozen candidate/rules, separately stored evaluator oracles, isolated reviewer packets, verbatim outputs, invocation envelopes and per-case status/hashes. T\final-checks.json and ending-identities.json record final checks and document identities outside the repository. Private orchestration is not a deliverable.

Recovery, measured miss and final freeze

The original C5 candidate/extract, 33 transcripts and packet identities were verified, including the corrected MIX-01 author benchmark. The case-only checker again passed 33 outputs / 345 rows. They were accepted as the existing C5 full execution: there was no recovery-triggered 33-run batch. Sprint 10 changed attribution, not decisions. Its provenance-only substitutions were applied to candidate C5-P, the sanitized extract and the integration link label; four unchanged development sentinels passed. Two source conversations lacked dedicated executions, so three source-selected development inputs were frozen before outputs, below.

C5-P scored 6/7, not acceptance. SRC-02 credited automatic listener removal but missed remaining cacheId/filtering and always-on Debug Stats.Incr cost. Generic external-listener comparability did not satisfy that concern. Independent evaluation checked the original source and oracle; P\source-evaluation.md retains the miss, two other source matches and independently supported additional risks. Original SRC-02 output SHA-256 is 91ce9f2e6d65f3b7887097cf4dcdb6179a9b226742bdcdc0530aba84d4a46ac1. No oracle, packet or output was rewritten.

C6's sole decision-rule change is conditional PERF-05 instrumentation assessment: credit repaired ownership separately from remaining collection costs; request relevant enabled/disabled evidence or justify bounded accounting without banning useful diagnostics or prescribing a directive. The #19995 source mapping supports that repair. A fresh SRC-02 focused run passed (F\focused\SRC-02.md, SHA-256 02c5179e9d8d70ff0b7880cb366a07f2e126796fd9cfb03f2e164862c77fd2b3), then all 36 cases ran once under the unchanged C6 freeze. The already-exposed cache-listener family remains development; this repair consumed no evaluation family and requires no new replacement. REPL-01 retains only its original independent-family credit.

Each original reviewer ran in a separate one-case context with full candidate instructions, rule/metric extract, complete pinned diff and relevant source/signature/test context. Direct discussion/oracle/other-output reads were excluded, but the extract was not fully sanitized: PERF-11 said "Recognize the final test executes it" regardless of the selected historical revision. Hash and shape checks did not detect this content leak. Additional code reads were limited to that packet's Git pins. Full author measurements were retained, including MIX-01's 25,152-byte artifact. Reviewers only saved their own unedited private outputs; evaluators only saved private scores. The orchestrator alone edited repository documents. Neither author-reported nor synthetic measurements/tests were executed here.

Input-isolation repair and sanitized replay

C6-S uses the unchanged C6 candidate and general decision gates, with a case-neutral extract. PERF-11 now credits runtime assertions only when the supplied pinned test invokes them; the extract also removes residual thread/family references and the case-specific 321 MB figure. No packet, source pin, benchmark, oracle or prior output changed. This correction supersedes earlier "sanitized" and "later fixes excluded" descriptions wherever the older extract was supplied; their observed scores remain historical, not clean-isolation acceptance.

Archive U = R\7b621641-31f6-4465-9fb7-e0cca80e32f2\files; D = U\c6-sanitized. The first affected DEV-06 run detected the nonexecuting conditional, then all 36 cases ran under this freeze. Tool-trace inspection additionally found generic compiler-review skill loads in that focused run and DEV-06/08/10/11. Although the skill contained no withheld case fact, these five contexts were excluded rather than silently accepted. A frozen wrapper clarification identifies artifact-only calibration, without changing reviewer rules or hinting outcomes. One clean focused run and four clean replacements completed; original outputs remain untouched.

Evaluator limitation: a locator search exposed historical DEV-01 result lines to the evaluator, not to any reviewer. It then checked the actual outputs, frozen concerns and pinned mechanisms rather than importing those scores. Original evaluator records and the selected aggregate preserve this caveat; no blinded-evaluator claim is made. This does not change reviewer isolation, source provenance, case eligibility or first-seen counts.

Added source-backed oracles

All three were selected from the named sprint-10 conversations, not from candidate outputs or a reserve-wide search. Exact source comments and the oracles were frozen in P before execution and copied unchanged to F. These are development, not three new holdouts or certified-human votes. Existing DEV-01/02/03/04 cover scope/optional syntax; DEV-07 and DEV-13 actually cover the elapsed/allocation distinction and resolved typed/shared-settings key.

Final case matrix

Source/oracle coordinates and required evidence remain in the original DEV matrix, exact HOLD oracle, REPL-01, MIX-01, BENCH-01 and the SRC table above. Every row below uses C6-S's selected actual output, not the original C6 score. Hashes are prefixes of input.md / exact unedited output; D's freeze, selection and selected score contain full identities. The earlier C6 matrix/score remains in F\scoring.json/.md. Outcome words describe scoped checks, not overall PR approval.

The evaluator checked pinned callers/preconditions/consequences for additional risks. DEV-11 identifies abandoned publication ownership after a later batch import fails, not C5's losing-importer-context trace. SRC-01 requires the forced store to become the callback/context's last owner; the shared-reader workload does not establish per-wrapper release savings. SRC-03's removed public disposable listener is separate from the repaired automatic leak. Diagnostic-cache, cyclic sharing-key, conditional Create binding and editor retention/removal traces remain in D's selected score. HOLD-02 raises no CWT/freshness blocker in this replay; F's earlier conditional finding remains historical. These are source-derived conditional failures, not executed field results, measured regressions or universal compatibility policies.

HOLD-09's additional float32 risk received separate dependency verification: the packet names .NET 9.0.4, whose pinned LINQ source uses a Double accumulator for Single inputs. Under the stated ordinary-rounding preconditions, [2^24; 1; -2^24] analytically changes from 0 to 1 versus the pinned old Single fold, independently of SIMD. Exact API/source bytes and the addendum are in D\evaluations\HOLD-09-runtime-source.*; addendum SHA-256 96f21db44542fd88e9e94afc132cb4bc552c144a104d31d1c006c8847354d1c0. This is source/mathematical proof, not an executed result, new human precedent, changed oracle or enlarged corpus.

Final acceptance and coverage checks

Sprint-12 delivery audit

Audited 2026-09-22: C6-S remains the accepted replay; no reviewer semantics, input, oracle, selection or score changed during delivery. The audit reused the existing shape, anchor, effective-rule and C6-S closure assertions read-only, replacing archive writes with equality checks. Historical C3/C5 fixed-hash checks were not misapplied to C6. Checks and pre-edit identities are in R\9168de79-7d16-4602-9ba9-74a3162b550d\files\audit_delivery.py, audit-results.json and starting-identities.json; no private artifact is a repository deliverable.

Real benchmark-only PR coverage remains zero; BENCH-01 is synthetic contract coverage, not human grounding or a real holdout. Deleted/private evidence, mutable historical edits, timeline-total anomalies and unavailable Azure payloads retain their recorded limits. Replays do not create independent holdouts, and source-derived failure scenarios are not executed product failures.