feat(worker): async worker loop + one-shot callback delivery #22

Merged
goldstein merged 1 commit from feat/worker-loop into main 2026-04-18 09:50:12 +00:00
Owner

Worker: claim_next_pending -> pipeline -> mark_done/mark_error -> callback (if set). Non-IX pipeline exceptions collapse to IX_002_000. Startup sweep rescues stuck running rows. Sleep cancels on stop event immediately.

Callback: one POST, 2xx=delivered, else failed. No retries; terminal state never reverts.

Chunk 3 Task 3.5. 7 new integration tests; local full suite (209 unit + 31 integration) green. Forgejo Actions trigger is flaky; local verification is the gate.

Worker: claim_next_pending -> pipeline -> mark_done/mark_error -> callback (if set). Non-IX pipeline exceptions collapse to IX_002_000. Startup sweep rescues stuck running rows. Sleep cancels on stop event immediately. Callback: one POST, 2xx=delivered, else failed. No retries; terminal state never reverts. Chunk 3 Task 3.5. 7 new integration tests; local full suite (209 unit + 31 integration) green. Forgejo Actions trigger is flaky; local verification is the gate.
goldstein added 1 commit 2026-04-18 09:50:07 +00:00
feat(worker): async worker loop + one-shot callback delivery (spec §5)
All checks were successful
tests / test (push) Successful in 1m15s
tests / test (pull_request) Successful in 1m8s
406a7ea2fd
Worker:
- Startup: sweep_orphans(now, max_running_seconds) rescues rows stuck
  in 'running' from a crashed prior process.
- Loop: claim_next_pending → build pipeline via injected factory → run
  → mark_done/mark_error → deliver callback if set → record outcome.
- Non-IX exceptions from the pipeline collapse to IX_002_000 so callers
  see a stable error code.
- Sleep loop uses a cancellable wait so the stop event reacts
  immediately; the wait_for_work hook is ready for Task 3.6 to plug in
  the LISTEN-driven event without the worker knowing about NOTIFY.

Callback:
- One-shot POST, 2xx → delivered, anything else (incl. connect/timeout
  exceptions) → failed. No retries.
- Callback record never reverts the job's terminal state — GET /jobs/{id}
  stays the authoritative fallback.

7 integration tests: happy path, pipeline-raise → error, callback 2xx,
callback 5xx, orphan sweep on startup, no-callback rows stay
callback_status=None (x2 via parametrize). Unit suite still 209.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
goldstein merged commit 415e03fba1 into main 2026-04-18 09:50:12 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: goldstein/infoxtractor#22
No description provided.