feat(pg-queue): LISTEN ix_jobs_new + 10s fallback poll #23

Merged

goldstein merged 1 commit from feat/pg-queue-adapter into main

2026-04-18 09:52:45 +00:00

goldstein commented

2026-04-18 09:52:39 +00:00

Owner

Helper asyncpg_dsn_from_sqlalchemy_url strips the +asyncpg driver segment and percent-decodes the password so the same IX_POSTGRES_URL works for both libs.

2 new integration tests: NOTIFY path picks up within 2s despite a 60s poll; missed-NOTIFY path still picks up within 5s via fallback poll. Total: 209 unit + 33 integration, all green. Forgejo Actions trigger is flaky; local verification is the gate.

PgQueueListener holds a dedicated asyncpg connection (outside SQLAlchemy pool) for LISTEN ix_jobs_new; exposes wait_for_work(timeout) that resolves on either a NOTIFY or the timeout. App lifespan spawns the listener alongside the worker. Helper asyncpg_dsn_from_sqlalchemy_url strips the +asyncpg driver segment and percent-decodes the password so the same IX_POSTGRES_URL works for both libs. 2 new integration tests: NOTIFY path picks up within 2s despite a 60s poll; missed-NOTIFY path still picks up within 5s via fallback poll. Total: 209 unit + 33 integration, all green. Forgejo Actions trigger is flaky; local verification is the gate.

goldstein added 1 commit 2026-04-18 09:52:39 +00:00

feat(pg-queue): LISTEN ix_jobs_new + 10s fallback poll (spec §4)

tests / test (push) Successful in 1m8s

Details

tests / test (pull_request) Successful in 1m9s

Details

050f80dcd7

PgQueueListener:
- Dedicated asyncpg connection outside the SQLAlchemy pool (LISTEN
  needs a persistent connection; pooled connections check in/out).
- Exposes wait_for_work(timeout) — resolves on NOTIFY or timeout,
  whichever fires first. The worker treats both wakes identically.
- asyncpg_dsn_from_sqlalchemy_url strips the +asyncpg driver segment
  and percent-decodes the password so the same URL in IX_POSTGRES_URL
  works for both SQLAlchemy and raw asyncpg.

app.py lifespan now also spawns the listener alongside the worker;
both are gated on spawn_worker=True so REST-only tests stay fast.

2 new integration tests: NOTIFY path (wake within 2 s despite 60 s
poll) + missed-NOTIFY path (fallback poll recovers within 5 s). 33
integration tests total, 209 unit. Forgejo Actions trigger is flaky;
local verification is the gate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

goldstein merged commit 6183b9c886 into main

2026-04-18 09:52:45 +00:00

goldstein referenced this pull request from a commit

2026-04-18 09:52:46 +00:00

Merge pull request 'feat(pg-queue): LISTEN ix_jobs_new + 10s fallback poll' (#23) from feat/pg-queue-adapter into main

No reviewers

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: goldstein/infoxtractor#23

No description provided.