Job-log ids are ambiguous across namespaces: a wrong id returns someone else's log instead of 404 #248

New issue

Open

opened 2026-08-09 22:57:33 +00:00 by stephen · 0 comments

stephen commented

2026-08-09 22:57:33 +00:00

Owner

fj run view --log-failed works now (thank you, #239), but two adjacent things make log retrieval unreliable, and both cost real diagnosis time today.

1. `runs/{id}/jobs` job ids do not address the log endpoint

GET /api/v1/repos/rasterstate/paragon/actions/runs/5399/jobs
  -> 12596 test success
     12599 db-tests failure

GET /api/v1/repos/rasterstate/paragon/actions/jobs/12599/logs
  -> 200, a complete log dated 2026-06-15

That is not the job I asked for; it is some other job from eight weeks earlier. The same thing happened with an id taken from actions/tasks (35574 returned an August 4 log for a different PR, and 35745 returned an August 5 Forseti run on PR #1097).

So there appear to be at least three id namespaces in play, and an id from one of them silently returns a valid-looking log from another rather than 404ing. That is the dangerous part: the response looks like a successful retrieval, and the only tell is the timestamp being days or weeks off. I built a false picture from one of these before noticing the date.

If fj is going to expose job logs, it would help enormously for it to resolve the id itself and refuse a cross-namespace hit, rather than callers guessing which number to use.

2. `--log-failed` gives up on some runs

$ fj run view 5400 --repo rasterstate/paragon --log-failed
error: Loki fallback needs workflow task metadata for job index 1

$ fj run view 5399 --repo rasterstate/paragon --log-failed
No matching jobs in this run.

Run 5399 does contain a failed job (db-tests, id 12599 per the jobs endpoint), so "No matching jobs" is wrong rather than merely unhelpful. Run 5400's failure is a different path again.

Why this matters more than it looks

fleet#267 took nine occurrences and several agent sessions to root-cause, and the single biggest reason was that nobody could reliably read what a failing CI job actually did. Once fj run view --log-failed worked, it took minutes to pin the exact line. Anything that makes log retrieval ambiguous puts us back in that state, and a wrong-but-plausible log is worse than an error, because it gets believed.

Reproduced on fj built from main today (post #239, #242, #247) on usw-dev-01.

`fj run view --log-failed` works now (thank you, #239), but two adjacent things make log retrieval unreliable, and both cost real diagnosis time today. ## 1. `runs/{id}/jobs` job ids do not address the log endpoint ``` GET /api/v1/repos/rasterstate/paragon/actions/runs/5399/jobs -> 12596 test success 12599 db-tests failure GET /api/v1/repos/rasterstate/paragon/actions/jobs/12599/logs -> 200, a complete log dated 2026-06-15 ``` That is not the job I asked for; it is some other job from eight weeks earlier. The same thing happened with an id taken from `actions/tasks` (`35574` returned an August 4 log for a different PR, and `35745` returned an August 5 Forseti run on PR #1097). So there appear to be at least three id namespaces in play, and **an id from one of them silently returns a valid-looking log from another** rather than 404ing. That is the dangerous part: the response looks like a successful retrieval, and the only tell is the timestamp being days or weeks off. I built a false picture from one of these before noticing the date. If fj is going to expose job logs, it would help enormously for it to resolve the id itself and refuse a cross-namespace hit, rather than callers guessing which number to use. ## 2. `--log-failed` gives up on some runs ``` $ fj run view 5400 --repo rasterstate/paragon --log-failed error: Loki fallback needs workflow task metadata for job index 1 $ fj run view 5399 --repo rasterstate/paragon --log-failed No matching jobs in this run. ``` Run 5399 does contain a failed job (`db-tests`, id 12599 per the jobs endpoint), so "No matching jobs" is wrong rather than merely unhelpful. Run 5400's failure is a different path again. ## Why this matters more than it looks fleet#267 took nine occurrences and several agent sessions to root-cause, and the single biggest reason was that nobody could reliably read what a failing CI job actually did. Once `fj run view --log-failed` worked, it took minutes to pin the exact line. Anything that makes log retrieval ambiguous puts us back in that state, and a wrong-but-plausible log is worse than an error, because it gets believed. Reproduced on fj built from `main` today (post #239, #242, #247) on usw-dev-01.