Clean stale coverage objects so CI measures the real strict surface #291

Merged

stephen merged 5 commits from fix/coverage-strict-surface into main 2026-10-13 13:41:19 +00:00

Conversation 0 Commits 5 Files changed 3 +21 -24

stephen commented 2026-10-13 02:59:13 +00:00

Owner

Copy link

The strict coverage job in CI measured a wider surface than a local run and scored about twenty points lower, and the gap grew between merges with no change in tests (#249). The cause is stale instrumented objects. The coverage job restores a cached target/, the target began with cargo llvm-cov clean --workspace, and that leaves behind objects built under an older cargo layout. llvm-cov then counted the leftover set as a second, never executed copy of every file it knew about.

The CI log shows it directly: files that existed when the cache was written were counted twice at about half coverage (cli/pr.rs 1555 regions at 54%), while files added since were counted once (api/admin.rs at 98%). The covered region counts matched a local run exactly; only the denominator was inflated. That is also why the number drifted: every edit to an old file moved it further from its stale copy.

make coverage-strict now starts with a full cargo llvm-cov clean. On this branch the runner went from 59758 regions at 67.79% lines to 46074 regions at 89.40%, the same surface a local run reports, and the job took no longer (2m38s). With the two in agreement the floor moves from the provisional 60 to 88, about a point under the observed number.

I first tried measuring only the unit-test binary (--bin fj), on the theory that the binary the integration tests spawn was the second copy. That changed nothing on the runner and is reverted here; the first two commits are that detour and can be squashed away.

Local runs need cargo-llvm-cov 0.9 or newer. 0.8 misses the test binary under the pinned nightly's build layout and reports about 2%, which is noted in CLAUDE.md.

Closes #249

The strict coverage job in CI measured a wider surface than a local run and scored about twenty points lower, and the gap grew between merges with no change in tests (#249). The cause is stale instrumented objects. The coverage job restores a cached `target/`, the target began with `cargo llvm-cov clean --workspace`, and that leaves behind objects built under an older cargo layout. llvm-cov then counted the leftover set as a second, never executed copy of every file it knew about. The CI log shows it directly: files that existed when the cache was written were counted twice at about half coverage (`cli/pr.rs` 1555 regions at 54%), while files added since were counted once (`api/admin.rs` at 98%). The covered region counts matched a local run exactly; only the denominator was inflated. That is also why the number drifted: every edit to an old file moved it further from its stale copy. `make coverage-strict` now starts with a full `cargo llvm-cov clean`. On this branch the runner went from 59758 regions at 67.79% lines to 46074 regions at 89.40%, the same surface a local run reports, and the job took no longer (2m38s). With the two in agreement the floor moves from the provisional 60 to 88, about a point under the observed number. I first tried measuring only the unit-test binary (`--bin fj`), on the theory that the binary the integration tests spawn was the second copy. That changed nothing on the runner and is reverted here; the first two commits are that detour and can be squashed away. Local runs need cargo-llvm-cov 0.9 or newer. 0.8 misses the test binary under the pinned nightly's build layout and reports about 2%, which is noted in CLAUDE.md. Closes #249

stephen added 2 commits 2026-10-13 02:59:13 +00:00

Update quinn-proto past five RustSec advisories ... ef879fdf6d

RUSTSEC-2026-0338 through 0342 landed against quinn-proto 0.11.15 and fail
the audit gate. The lockfile moves to a fixed release; nothing in fj's own
manifest changes.

(cherry picked from commit 4aa845981e)

Measure strict coverage on the unit-test binary only 56a888324a

stephen added 1 commit 2026-10-13 03:33:08 +00:00

Clean every coverage artifact before the strict run f5c6396855

stephen added 2 commits 2026-10-13 04:08:33 +00:00

Merge remote-tracking branch 'origin/main' into fix/coverage-strict-surface a5fffc4619

Enforce the strict coverage floor against the real surface ... e1fcfd1d6a

The runner now measures what a local run measures (46074 regions,
89.40% lines), so the floor moves from the provisional 60 to 88 and the
notes that described the mismatch describe its cause instead.

stephen changed title from WIP: Measure strict coverage on the unit-test binary only to Clean stale coverage objects so CI measures the real strict surface 2026-10-13 04:08:33 +00:00

stephen merged commit 4e87977e99 into main 2026-10-13 13:41:19 +00:00

stephen deleted branch fix/coverage-strict-surface 2026-10-13 13:41:20 +00:00

stephen referenced this pull request from a commit 2026-10-13 13:41:20 +00:00

Clean stale coverage objects so CI measures the real strict surface (#291)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

backlog

Accepted but not scheduled

  

blocked

Blocked or stalled waiting on external progress

  

converted

Converted into tracked work

  

forseti:changes-requested

Forseti review found unaddressed findings; cleared on a clean re-review at the current head.

  

opportunity

Potential product or business opportunity

  

p1

Priority 1

  

p2

Priority 2

  

p3

Priority 3

  

parked

Paused intentionally until later

No labels

backlog

blocked

converted

forseti:changes-requested

opportunity

p1

p2

p3

parked

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rasterstate/fj!291

No description provided.