Compare commits
No commits in common. "main" and "fix/torch-cu124" have entirely different histories.
main
...
fix/torch-
29 changed files with 117 additions and 3646 deletions
|
|
@ -4,11 +4,7 @@ Async, on-prem, LLM-powered structured information extraction microservice. Give
|
||||||
|
|
||||||
Designed to be used by other on-prem services (e.g. mammon) as a reliable fallback / second opinion for format-specific deterministic parsers.
|
Designed to be used by other on-prem services (e.g. mammon) as a reliable fallback / second opinion for format-specific deterministic parsers.
|
||||||
|
|
||||||
Status: MVP deployed (2026-04-18) at `http://192.168.68.42:8994` — LAN only. Browser UI at `http://192.168.68.42:8994/ui`. Full reference spec at `docs/spec-core-pipeline.md`; MVP spec at `docs/superpowers/specs/2026-04-18-ix-mvp-design.md`; deploy runbook at `docs/deployment.md`.
|
Status: design phase. Full reference spec at `docs/spec-core-pipeline.md`. MVP spec will live at `docs/superpowers/specs/`.
|
||||||
|
|
||||||
Use cases: the built-in registry lives in `src/ix/use_cases/__init__.py` (`bank_statement_header` for MVP). Callers without a registered entry can ship an ad-hoc schema inline via `RequestIX.use_case_inline` (see README "Ad-hoc use cases"); the pipeline builds the Pydantic classes on the fly per request. The `/ui` page exposes this as a "custom" option so non-engineering users can experiment without a deploy.
|
|
||||||
|
|
||||||
UX notes: the `/ui` job page surfaces queue position + elapsed MM:SS on each poll, renders the client-provided filename (stored via `FileRef.display_name`, optional metadata — the pipeline ignores it for execution), and shows a CPU-mode notice when `/healthz` reports `ocr_gpu: false`. A paginated history lives at `/ui/jobs` (status + client_id filters, newest first).
|
|
||||||
|
|
||||||
## Guiding Principles
|
## Guiding Principles
|
||||||
|
|
||||||
|
|
|
||||||
92
README.md
92
README.md
|
|
@ -4,18 +4,10 @@ Async, on-prem, LLM-powered structured information extraction microservice.
|
||||||
|
|
||||||
Given a document (PDF, image, text) and a named *use case*, ix returns a structured JSON result whose shape matches the use-case schema — together with per-field provenance (OCR segment IDs, bounding boxes, cross-OCR agreement flags) that let the caller decide how much to trust each extracted value.
|
Given a document (PDF, image, text) and a named *use case*, ix returns a structured JSON result whose shape matches the use-case schema — together with per-field provenance (OCR segment IDs, bounding boxes, cross-OCR agreement flags) that let the caller decide how much to trust each extracted value.
|
||||||
|
|
||||||
**Status:** MVP deployed. Live on the home LAN at `http://192.168.68.42:8994` (REST API + browser UI at `/ui`).
|
**Status:** design phase. Implementation about to start.
|
||||||
|
|
||||||
## Web UI
|
|
||||||
|
|
||||||
A minimal browser UI lives at [`http://192.168.68.42:8994/ui`](http://192.168.68.42:8994/ui): drop a PDF, pick a registered use case or define one inline, submit, see the pretty-printed result. HTMX polls the job status every 2 s until the pipeline finishes. LAN-only, no auth.
|
|
||||||
|
|
||||||
Past submissions are browsable at [`/ui/jobs`](http://192.168.68.42:8994/ui/jobs) — a paginated list (newest first) with status + `client_id` filters. Each row links to `/ui/jobs/{job_id}` for the full request/response view.
|
|
||||||
|
|
||||||
- Full reference spec: [`docs/spec-core-pipeline.md`](docs/spec-core-pipeline.md) (aspirational; MVP is a strict subset)
|
- Full reference spec: [`docs/spec-core-pipeline.md`](docs/spec-core-pipeline.md) (aspirational; MVP is a strict subset)
|
||||||
- **MVP design:** [`docs/superpowers/specs/2026-04-18-ix-mvp-design.md`](docs/superpowers/specs/2026-04-18-ix-mvp-design.md)
|
- **MVP design:** [`docs/superpowers/specs/2026-04-18-ix-mvp-design.md`](docs/superpowers/specs/2026-04-18-ix-mvp-design.md)
|
||||||
- **Implementation plan:** [`docs/superpowers/plans/2026-04-18-ix-mvp-implementation.md`](docs/superpowers/plans/2026-04-18-ix-mvp-implementation.md)
|
|
||||||
- **Deployment runbook:** [`docs/deployment.md`](docs/deployment.md)
|
|
||||||
- Agent / development notes: [`AGENTS.md`](AGENTS.md)
|
- Agent / development notes: [`AGENTS.md`](AGENTS.md)
|
||||||
|
|
||||||
## Principles
|
## Principles
|
||||||
|
|
@ -23,85 +15,3 @@ Past submissions are browsable at [`/ui/jobs`](http://192.168.68.42:8994/ui/jobs
|
||||||
- **On-prem always.** LLM = Ollama, OCR = local engines (Surya first). No OpenAI / Anthropic / Azure / AWS / cloud.
|
- **On-prem always.** LLM = Ollama, OCR = local engines (Surya first). No OpenAI / Anthropic / Azure / AWS / cloud.
|
||||||
- **Grounded extraction, not DB truth.** ix returns best-effort fields + provenance; the caller decides what to trust.
|
- **Grounded extraction, not DB truth.** ix returns best-effort fields + provenance; the caller decides what to trust.
|
||||||
- **Transport-agnostic pipeline core.** REST + Postgres-queue adapters in parallel on one job store.
|
- **Transport-agnostic pipeline core.** REST + Postgres-queue adapters in parallel on one job store.
|
||||||
|
|
||||||
## Submitting a job
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -X POST http://192.168.68.42:8994/jobs \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '{
|
|
||||||
"use_case": "bank_statement_header",
|
|
||||||
"ix_client_id": "mammon",
|
|
||||||
"request_id": "some-correlation-id",
|
|
||||||
"context": {
|
|
||||||
"files": [{
|
|
||||||
"url": "http://paperless.local/api/documents/42/download/",
|
|
||||||
"headers": {"Authorization": "Token …"}
|
|
||||||
}],
|
|
||||||
"texts": ["<Paperless Tesseract OCR content>"]
|
|
||||||
}
|
|
||||||
}'
|
|
||||||
# → {"job_id":"…","ix_id":"…","status":"pending"}
|
|
||||||
```
|
|
||||||
|
|
||||||
Poll `GET /jobs/{job_id}` until `status` is `done` or `error`. Optionally pass `callback_url` to receive a webhook on completion (one-shot, no retry; polling stays authoritative).
|
|
||||||
|
|
||||||
### Ad-hoc use cases
|
|
||||||
|
|
||||||
For one-offs where a registered use case doesn't exist yet, ship the schema inline:
|
|
||||||
|
|
||||||
```jsonc
|
|
||||||
{
|
|
||||||
"use_case": "adhoc-invoice", // free-form label (logs/metrics only)
|
|
||||||
"use_case_inline": {
|
|
||||||
"use_case_name": "Invoice totals",
|
|
||||||
"system_prompt": "Extract vendor and total amount.",
|
|
||||||
"fields": [
|
|
||||||
{"name": "vendor", "type": "str", "required": true},
|
|
||||||
{"name": "total", "type": "decimal"},
|
|
||||||
{"name": "currency", "type": "str", "choices": ["USD", "EUR", "CHF"]}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
// ...ix_client_id, request_id, context...
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
When `use_case_inline` is set, the pipeline builds the response schema on the fly and skips the registry. Supported types: `str`, `int`, `float`, `decimal`, `date`, `datetime`, `bool`. `choices` is only allowed on `str` fields. Precedence: inline wins over `use_case` when both are present.
|
|
||||||
|
|
||||||
Full REST surface + provenance response shape documented in the MVP design spec.
|
|
||||||
|
|
||||||
## Running locally
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv sync --extra dev
|
|
||||||
uv run pytest tests/unit -v # hermetic unit + integration suite
|
|
||||||
IX_TEST_OLLAMA=1 uv run pytest tests/live -v # needs LAN access to Ollama + GPU
|
|
||||||
```
|
|
||||||
|
|
||||||
### UI jobs list
|
|
||||||
|
|
||||||
`GET /ui/jobs` renders a paginated, newest-first table of submitted jobs. Query params:
|
|
||||||
|
|
||||||
- `status=pending|running|done|error` — repeat for multi-select.
|
|
||||||
- `client_id=<str>` — exact match (e.g. `ui`, `mammon`).
|
|
||||||
- `limit=<n>` (default 50, max 200) + `offset=<n>` for paging.
|
|
||||||
|
|
||||||
Each row shows status badge, original filename (`FileRef.display_name` or URL basename), use case, client id, submitted time + relative, and elapsed wall-clock (terminal rows only). Each row links to `/ui/jobs/{job_id}` for the full response JSON.
|
|
||||||
|
|
||||||
### UI queue + progress UX
|
|
||||||
|
|
||||||
The `/ui` job page polls `GET /ui/jobs/{id}/fragment` every 2 s and surfaces:
|
|
||||||
|
|
||||||
- **Queue position** while pending: "Queue position: N ahead — M jobs total in flight (single worker)" so it's obvious a new submission is waiting on an earlier job rather than stuck. "About to start" when the worker has just freed up.
|
|
||||||
- **Elapsed time** while running ("Running for MM:SS") and on finish ("Finished in MM:SS").
|
|
||||||
- **Original filename** — the UI stashes the client-provided upload name in `FileRef.display_name` so the browser shows `your_statement.pdf` instead of the on-disk UUID.
|
|
||||||
- **CPU-mode notice** when `/healthz` reports `ocr_gpu: false` (the Surya OCR client observed `torch.cuda.is_available() == False`): a collapsed `<details>` pointing at the deployment runbook.
|
|
||||||
|
|
||||||
## Deploying
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git push server main # rebuilds Docker image, restarts container, /healthz deploy gate
|
|
||||||
python scripts/e2e_smoke.py # E2E acceptance against the live service
|
|
||||||
```
|
|
||||||
|
|
||||||
See [`docs/deployment.md`](docs/deployment.md) for full runbook + rollback.
|
|
||||||
|
|
|
||||||
|
|
@ -10,8 +10,6 @@
|
||||||
# The GPU reservation block matches immich-ml / the shape Docker Compose
|
# The GPU reservation block matches immich-ml / the shape Docker Compose
|
||||||
# expects for GPU allocation on this host.
|
# expects for GPU allocation on this host.
|
||||||
|
|
||||||
name: infoxtractor
|
|
||||||
|
|
||||||
services:
|
services:
|
||||||
infoxtractor:
|
infoxtractor:
|
||||||
build: .
|
build: .
|
||||||
|
|
@ -26,17 +24,8 @@ services:
|
||||||
- driver: nvidia
|
- driver: nvidia
|
||||||
count: 1
|
count: 1
|
||||||
capabilities: [gpu]
|
capabilities: [gpu]
|
||||||
volumes:
|
|
||||||
# Persist Surya (datalab) + HuggingFace model caches so rebuilds don't
|
|
||||||
# re-download ~1.5 GB of weights every time.
|
|
||||||
- ix_surya_cache:/root/.cache/datalab
|
|
||||||
- ix_hf_cache:/root/.cache/huggingface
|
|
||||||
labels:
|
labels:
|
||||||
infrastructure.web_url: "http://192.168.68.42:8994"
|
infrastructure.web_url: "http://192.168.68.42:8994"
|
||||||
backup.enable: "true"
|
backup.enable: "true"
|
||||||
backup.type: "postgres"
|
backup.type: "postgres"
|
||||||
backup.name: "infoxtractor"
|
backup.name: "infoxtractor"
|
||||||
|
|
||||||
volumes:
|
|
||||||
ix_surya_cache:
|
|
||||||
ix_hf_cache:
|
|
||||||
|
|
|
||||||
|
|
@ -71,30 +71,13 @@ git push server main
|
||||||
|
|
||||||
## First deploy
|
## First deploy
|
||||||
|
|
||||||
- **Date:** 2026-04-18
|
_(fill in after running — timestamps, commit sha, e2e_smoke output)_
|
||||||
- **Commit:** `fix/ollama-extract-json` (#36, the last of several Docker/ops follow-ups after PR #27 shipped the initial Dockerfile)
|
|
||||||
- **`/healthz`:** all three probes (`postgres`, `ollama`, `ocr`) green. First-pass took ~7 min for the fresh container because Surya's recognition (1.34 GB) + detection (73 MB) models download from HuggingFace on first run; subsequent rebuilds reuse the named volumes declared in `docker-compose.yml` and come up in <30 s.
|
|
||||||
- **E2E extraction:** `bank_statement_header` against `tests/fixtures/synthetic_giro.pdf` with Paperless-style texts:
|
|
||||||
- Pipeline completes in **35 s**.
|
|
||||||
- Extracted: `bank_name=DKB`, `account_iban=DE89370400440532013000`, `currency=EUR`, `opening_balance=1234.56`, `closing_balance=1450.22`, `statement_date=2026-03-31`, `statement_period_end=2026-03-31`, `statement_period_start=2026-03-01`, `account_type=null`.
|
|
||||||
- Provenance: 8 / 9 leaf fields have sources; 7 / 8 `provenance_verified` and `text_agreement` are True. `statement_period_start` shows up in the OCR but normalisation fails (dateutil picks a different interpretation of the cited day); to be chased in a follow-up.
|
|
||||||
|
|
||||||
### Docker-ops follow-ups that landed during the first deploy
|
- **Date:** TBD
|
||||||
|
- **Commit:** TBD
|
||||||
All small, each merged as its own PR. In commit order after the scaffold (#27):
|
- **`/healthz` first-ok time:** TBD
|
||||||
|
- **`e2e_smoke.py` status:** TBD
|
||||||
- **#31** `fix(docker): uv via standalone installer` — Python 3.12 on Ubuntu 22.04 drops `distutils`; Ubuntu's pip needed it. Switched to the `uv` standalone installer, which has no pip dependency.
|
- **Notes:** —
|
||||||
- **#32** `fix(docker): include README.md in the uv sync COPY` — `hatchling` validates the readme file exists when resolving the editable project install.
|
|
||||||
- **#33** `fix(compose): drop runtime: nvidia` — the deploy host's Docker daemon doesn't register a named `nvidia` runtime; `deploy.resources.devices` is sufficient and matches immich-ml.
|
|
||||||
- **#34** `fix(deploy): network_mode: host` — `postgis` is bound to `127.0.0.1` on the host (security hardening T12). `host.docker.internal` points at the bridge gateway, not loopback, so the container couldn't reach postgis. Goldstein uses the same pattern.
|
|
||||||
- **#35** `fix(deps): pin surya-ocr ^0.17` — earlier cu124 torch pin had forced surya to 0.14.1, which breaks our `surya.foundation` import and needs a transformers version that lacks `QuantizedCacheConfig`.
|
|
||||||
- **#36** `fix(genai): drop Ollama format flag; extract trailing JSON` — Ollama 0.11.8 segfaults on Pydantic JSON Schemas (`$ref`, `anyOf`, `pattern`), and `format="json"` terminates reasoning models (qwen3) at `{}` because their `<think>…</think>` chain-of-thought isn't valid JSON. Omit the flag, inject the schema into the system prompt, extract the outermost `{…}` balanced block from the response.
|
|
||||||
- **volumes** — named `ix_surya_cache` + `ix_hf_cache` mount `/root/.cache/datalab` + `/root/.cache/huggingface` so rebuilds don't re-download ~1.5 GB of model weights.
|
|
||||||
|
|
||||||
Production notes:
|
|
||||||
|
|
||||||
- `IX_DEFAULT_MODEL=qwen3:14b` (already pulled on the host). Spec listed `gpt-oss:20b` as a concrete example; swapped to keep the deploy on-prem without an extra `ollama pull`.
|
|
||||||
- Torch 2.11 default cu13 wheels fall back to CPU against the host's CUDA 12.4 driver — Surya runs on CPU. Expected inference times: seconds per page. Upgrading the NVIDIA driver (or pinning a cu12-compatible torch wheel newer than 2.7) will unlock GPU with no code changes.
|
|
||||||
|
|
||||||
## E2E smoke test (`scripts/e2e_smoke.py`)
|
## E2E smoke test (`scripts/e2e_smoke.py`)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -85,7 +85,6 @@ class FileRef(BaseModel):
|
||||||
url: str # http(s):// or file://
|
url: str # http(s):// or file://
|
||||||
headers: dict[str, str] = {} # e.g. {"Authorization": "Token …"}
|
headers: dict[str, str] = {} # e.g. {"Authorization": "Token …"}
|
||||||
max_bytes: Optional[int] = None # per-file override; defaults to IX_FILE_MAX_BYTES
|
max_bytes: Optional[int] = None # per-file override; defaults to IX_FILE_MAX_BYTES
|
||||||
display_name: Optional[str] = None # UI-only metadata; client-provided filename for display (pipeline ignores)
|
|
||||||
|
|
||||||
class Options(BaseModel):
|
class Options(BaseModel):
|
||||||
ocr: OCROptions = OCROptions()
|
ocr: OCROptions = OCROptions()
|
||||||
|
|
@ -109,25 +108,6 @@ class ProvenanceOptions(BaseModel):
|
||||||
|
|
||||||
**Dropped from spec (no-ops under MVP):** `OCROptions.computer_vision_scaling_factor`, `include_page_tags` (always on), `GenAIOptions.use_vision`/`vision_scaling_factor`/`vision_detail`/`reasoning_effort`, `ProvenanceOptions.granularity`/`include_bounding_boxes`/`source_type`/`min_confidence`, `RequestIX.version`.
|
**Dropped from spec (no-ops under MVP):** `OCROptions.computer_vision_scaling_factor`, `include_page_tags` (always on), `GenAIOptions.use_vision`/`vision_scaling_factor`/`vision_detail`/`reasoning_effort`, `ProvenanceOptions.granularity`/`include_bounding_boxes`/`source_type`/`min_confidence`, `RequestIX.version`.
|
||||||
|
|
||||||
**Ad-hoc use cases (post-MVP add-on).** `RequestIX` carries an optional `use_case_inline: InlineUseCase | None = None`. When set, the pipeline builds the `(Request, Response)` Pydantic class pair on the fly from that inline definition and **skips the registry lookup entirely** — the `use_case` field becomes a free-form label (still required for metrics / logging). Inline definitions look like:
|
|
||||||
|
|
||||||
```python
|
|
||||||
class UseCaseFieldDef(BaseModel):
|
|
||||||
name: str # valid Python identifier
|
|
||||||
type: Literal["str", "int", "float", "decimal", "date", "datetime", "bool"]
|
|
||||||
required: bool = False
|
|
||||||
description: str | None = None
|
|
||||||
choices: list[str] | None = None # str-typed fields only; builds Literal[*choices]
|
|
||||||
|
|
||||||
class InlineUseCase(BaseModel):
|
|
||||||
use_case_name: str
|
|
||||||
system_prompt: str
|
|
||||||
default_model: str | None = None
|
|
||||||
fields: list[UseCaseFieldDef]
|
|
||||||
```
|
|
||||||
|
|
||||||
Precedence: `use_case_inline` wins when both are set. Structural errors (dup field name, invalid identifier, `choices` on a non-str type, empty fields list) raise `IX_001_001` (same code as registry miss). The builder lives in `ix.use_cases.inline.build_use_case_classes` and returns fresh classes per call — the pipeline never caches them.
|
|
||||||
|
|
||||||
### ResponseIX
|
### ResponseIX
|
||||||
|
|
||||||
Identical to spec §2.2 except `FieldProvenance` gains two fields:
|
Identical to spec §2.2 except `FieldProvenance` gains two fields:
|
||||||
|
|
@ -226,15 +206,14 @@ Callers that prefer direct SQL (the `pg_queue_adapter` contract): insert a row w
|
||||||
| `POST` | `/jobs` | Body = `RequestIX` (+ optional `callback_url`). → `201 {job_id, ix_id, status: "pending"}`. Idempotent on `(ix_client_id, request_id)` — same pair returns the existing `job_id` with `200`. |
|
| `POST` | `/jobs` | Body = `RequestIX` (+ optional `callback_url`). → `201 {job_id, ix_id, status: "pending"}`. Idempotent on `(ix_client_id, request_id)` — same pair returns the existing `job_id` with `200`. |
|
||||||
| `GET` | `/jobs/{job_id}` | → full `Job`. Source of truth regardless of submission path or callback outcome. |
|
| `GET` | `/jobs/{job_id}` | → full `Job`. Source of truth regardless of submission path or callback outcome. |
|
||||||
| `GET` | `/jobs?client_id=…&request_id=…` | Lookup-by-correlation (caller idempotency helper). The pair is UNIQUE in the table → at most one match. Returns the job or `404`. |
|
| `GET` | `/jobs?client_id=…&request_id=…` | Lookup-by-correlation (caller idempotency helper). The pair is UNIQUE in the table → at most one match. Returns the job or `404`. |
|
||||||
| `GET` | `/healthz` | `{postgres, ollama, ocr, ocr_gpu}`. See below for semantics. Used by `infrastructure` monitoring dashboard. `ocr_gpu` is additive metadata (not part of the gate). |
|
| `GET` | `/healthz` | `{postgres, ollama, ocr}`. See below for semantics. Used by `infrastructure` monitoring dashboard. |
|
||||||
| `GET` | `/metrics` | Counters over the last 24 hours: `jobs_pending`, `jobs_running`, `jobs_done_24h`, `jobs_error_24h`, per-use-case avg seconds over the same window. Plain JSON, no Prometheus format for MVP. |
|
| `GET` | `/metrics` | Counters over the last 24 hours: `jobs_pending`, `jobs_running`, `jobs_done_24h`, `jobs_error_24h`, per-use-case avg seconds over the same window. Plain JSON, no Prometheus format for MVP. |
|
||||||
|
|
||||||
**`/healthz` semantics:**
|
**`/healthz` semantics:**
|
||||||
- `postgres`: `SELECT 1` on the job store pool; `ok` iff the query returns within 2 s.
|
- `postgres`: `SELECT 1` on the job store pool; `ok` iff the query returns within 2 s.
|
||||||
- `ollama`: `GET {IX_OLLAMA_URL}/api/tags` within 5 s; `ok` iff reachable AND the default model (`IX_DEFAULT_MODEL`) is listed in the tags response; `degraded` iff reachable but the model is missing (ops action: run `ollama pull <model>` on the host); `fail` on any other error.
|
- `ollama`: `GET {IX_OLLAMA_URL}/api/tags` within 5 s; `ok` iff reachable AND the default model (`IX_DEFAULT_MODEL`) is listed in the tags response; `degraded` iff reachable but the model is missing (ops action: run `ollama pull <model>` on the host); `fail` on any other error.
|
||||||
- `ocr`: `SuryaOCRClient.selfcheck()` — returns `ok` iff CUDA is available and the Surya text-recognition model is loaded into GPU memory at process start. `fail` on any error.
|
- `ocr`: `SuryaOCRClient.selfcheck()` — returns `ok` iff CUDA is available and the Surya text-recognition model is loaded into GPU memory at process start. `fail` on any error.
|
||||||
- `ocr_gpu`: `true | false | null`. Additive metadata: reports whether the OCR client observed `torch.cuda.is_available() == True` at first warm-up. `null` means not yet probed (fresh process, fake client, etc.). The UI reads this to surface a CPU-mode slowdown notice; never part of the 200/503 gate.
|
- Overall HTTP status: `200` iff all three are `ok`; `503` otherwise. The monitoring dashboard only surfaces `200`/`non-200`.
|
||||||
- Overall HTTP status: `200` iff all three core statuses (`postgres`, `ollama`, `ocr`) are `ok`; `503` otherwise. `ocr_gpu` does not affect the gate. The monitoring dashboard only surfaces `200`/`non-200`.
|
|
||||||
|
|
||||||
**Callback delivery** (when `callback_url` is set): one POST of the full `Job` body, 10 s timeout. 2xx → `callback_status='delivered'`. Anything else → `'failed'`. No retry. Callers always have `GET /jobs/{id}` as the authoritative fallback.
|
**Callback delivery** (when `callback_url` is set): one POST of the full `Job` body, 10 s timeout. 2xx → `callback_status='delivered'`. Anything else → `'failed'`. No retry. Callers always have `GET /jobs/{id}` as the authoritative fallback.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,6 @@
|
||||||
[project]
|
[project]
|
||||||
name = "infoxtractor"
|
name = "infoxtractor"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
# Released 2026-04-18 with the first live deploy of the MVP. See
|
|
||||||
# docs/deployment.md §"First deploy" for the commit + /healthz times.
|
|
||||||
description = "Async on-prem LLM-powered structured information extraction microservice"
|
description = "Async on-prem LLM-powered structured information extraction microservice"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.12"
|
requires-python = ">=3.12"
|
||||||
|
|
@ -29,24 +27,16 @@ dependencies = [
|
||||||
"pillow>=10.2,<11.0",
|
"pillow>=10.2,<11.0",
|
||||||
"python-magic>=0.4.27",
|
"python-magic>=0.4.27",
|
||||||
"python-dateutil>=2.9",
|
"python-dateutil>=2.9",
|
||||||
|
|
||||||
# UI (HTMX + Jinja2 templates served from /ui). Both arrive as transitive
|
|
||||||
# deps via FastAPI/Starlette already, but we pin explicitly so the import
|
|
||||||
# surface is owned by us. python-multipart backs FastAPI's `Form()` /
|
|
||||||
# `UploadFile` parsing — required by `/ui/jobs` submissions.
|
|
||||||
"jinja2>=3.1",
|
|
||||||
"aiofiles>=24.1",
|
|
||||||
"python-multipart>=0.0.12",
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
ocr = [
|
ocr = [
|
||||||
# Real OCR engine. Kept optional so CI (no GPU) can install the base
|
# Real OCR engine — pulls torch + CUDA wheels. Kept optional so CI
|
||||||
# package without the model deps.
|
# (no GPU) can install the base package without the model deps.
|
||||||
# surya >= 0.17 is required: the client code uses the
|
"surya-ocr>=0.9",
|
||||||
# `surya.foundation` module, which older releases don't expose.
|
# torch pinned to the cu124 wheel channel via tool.uv.sources below
|
||||||
"surya-ocr>=0.17,<0.18",
|
# so the GPU wheels match the deploy host's CUDA 12.4 driver.
|
||||||
"torch>=2.7",
|
"torch>=2.4",
|
||||||
]
|
]
|
||||||
dev = [
|
dev = [
|
||||||
"pytest>=8.3",
|
"pytest>=8.3",
|
||||||
|
|
@ -56,10 +46,16 @@ dev = [
|
||||||
"mypy>=1.13",
|
"mypy>=1.13",
|
||||||
]
|
]
|
||||||
|
|
||||||
# Note: the default pypi torch ships cu13 wheels, which emit a
|
# Make uv pull torch from the CUDA 12.4 wheel channel to match the
|
||||||
# UserWarning and fall back to CPU against the deploy host's CUDA 12.4
|
# deploy host's NVIDIA driver (server has CUDA 12.4; pypi torch 2.11
|
||||||
# driver. Surya then runs on CPU — slower but correct for MVP. A future
|
# ships cu13 wheels which refuse to initialise on an older driver).
|
||||||
# driver upgrade unlocks GPU Surya with no code changes.
|
[[tool.uv.index]]
|
||||||
|
name = "pytorch-cu124"
|
||||||
|
url = "https://download.pytorch.org/whl/cu124"
|
||||||
|
explicit = true
|
||||||
|
|
||||||
|
[tool.uv.sources]
|
||||||
|
torch = { index = "pytorch-cu124" }
|
||||||
|
|
||||||
[build-system]
|
[build-system]
|
||||||
requires = ["hatchling"]
|
requires = ["hatchling"]
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,7 @@ from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from datetime import UTC, datetime, timedelta
|
from datetime import UTC, datetime, timedelta
|
||||||
from typing import Annotated, Literal
|
from typing import Annotated, Literal
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
@ -44,15 +44,10 @@ class Probes:
|
||||||
keeping them sync lets tests pass plain lambdas. Real probes that need
|
keeping them sync lets tests pass plain lambdas. Real probes that need
|
||||||
async work run the call through ``asyncio.run_in_executor`` inside the
|
async work run the call through ``asyncio.run_in_executor`` inside the
|
||||||
callable (Chunk 4).
|
callable (Chunk 4).
|
||||||
|
|
||||||
``ocr_gpu`` is additive metadata for the UI (not a health gate): returns
|
|
||||||
``True`` iff the OCR client reports CUDA is available, ``False`` for
|
|
||||||
explicit CPU-mode, ``None`` if unknown (fake client, not yet warmed up).
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
ollama: Callable[[], Literal["ok", "degraded", "fail"]]
|
ollama: Callable[[], Literal["ok", "degraded", "fail"]]
|
||||||
ocr: Callable[[], Literal["ok", "fail"]]
|
ocr: Callable[[], Literal["ok", "fail"]]
|
||||||
ocr_gpu: Callable[[], bool | None] = field(default=lambda: None)
|
|
||||||
|
|
||||||
|
|
||||||
def get_session_factory_dep() -> async_sessionmaker[AsyncSession]:
|
def get_session_factory_dep() -> async_sessionmaker[AsyncSession]:
|
||||||
|
|
@ -168,16 +163,8 @@ async def healthz(
|
||||||
except Exception:
|
except Exception:
|
||||||
ocr_state = "fail"
|
ocr_state = "fail"
|
||||||
|
|
||||||
try:
|
|
||||||
ocr_gpu_state: bool | None = probes.ocr_gpu()
|
|
||||||
except Exception:
|
|
||||||
ocr_gpu_state = None
|
|
||||||
|
|
||||||
body = HealthStatus(
|
body = HealthStatus(
|
||||||
postgres=postgres_state,
|
postgres=postgres_state, ollama=ollama_state, ocr=ocr_state
|
||||||
ollama=ollama_state,
|
|
||||||
ocr=ocr_state,
|
|
||||||
ocr_gpu=ocr_gpu_state,
|
|
||||||
)
|
)
|
||||||
if postgres_state != "ok" or ollama_state != "ok" or ocr_state != "ok":
|
if postgres_state != "ok" or ollama_state != "ok" or ocr_state != "ok":
|
||||||
response.status_code = 503
|
response.status_code = 503
|
||||||
|
|
|
||||||
|
|
@ -28,15 +28,9 @@ class HealthStatus(BaseModel):
|
||||||
"""Body of GET /healthz.
|
"""Body of GET /healthz.
|
||||||
|
|
||||||
Each field reports per-subsystem state. Overall HTTP status is 200 iff
|
Each field reports per-subsystem state. Overall HTTP status is 200 iff
|
||||||
every of the three core status keys is ``"ok"`` (spec §5). ``ollama`` can
|
every field is ``"ok"`` (spec §5). ``ollama`` can be ``"degraded"``
|
||||||
be ``"degraded"`` when the backend is reachable but the default model
|
when the backend is reachable but the default model isn't pulled —
|
||||||
isn't pulled — monitoring surfaces that as non-200.
|
monitoring surfaces that as non-200.
|
||||||
|
|
||||||
``ocr_gpu`` is additive metadata, not part of the health gate: it reports
|
|
||||||
whether the Surya OCR client observed ``torch.cuda.is_available() == True``
|
|
||||||
on first warm-up. ``None`` means we haven't probed yet (fresh process,
|
|
||||||
fake client, or warm_up hasn't happened). The UI reads this to surface a
|
|
||||||
CPU-mode slowdown warning to users.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
@ -44,7 +38,6 @@ class HealthStatus(BaseModel):
|
||||||
postgres: Literal["ok", "fail"]
|
postgres: Literal["ok", "fail"]
|
||||||
ollama: Literal["ok", "degraded", "fail"]
|
ollama: Literal["ok", "degraded", "fail"]
|
||||||
ocr: Literal["ok", "fail"]
|
ocr: Literal["ok", "fail"]
|
||||||
ocr_gpu: bool | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class MetricsResponse(BaseModel):
|
class MetricsResponse(BaseModel):
|
||||||
|
|
|
||||||
|
|
@ -24,7 +24,6 @@ from contextlib import asynccontextmanager, suppress
|
||||||
from typing import Literal
|
from typing import Literal
|
||||||
|
|
||||||
from fastapi import FastAPI
|
from fastapi import FastAPI
|
||||||
from fastapi.staticfiles import StaticFiles
|
|
||||||
|
|
||||||
from ix.adapters.rest.routes import Probes, get_probes
|
from ix.adapters.rest.routes import Probes, get_probes
|
||||||
from ix.adapters.rest.routes import router as rest_router
|
from ix.adapters.rest.routes import router as rest_router
|
||||||
|
|
@ -39,8 +38,6 @@ from ix.pipeline.pipeline import Pipeline
|
||||||
from ix.pipeline.reliability_step import ReliabilityStep
|
from ix.pipeline.reliability_step import ReliabilityStep
|
||||||
from ix.pipeline.response_handler_step import ResponseHandlerStep
|
from ix.pipeline.response_handler_step import ResponseHandlerStep
|
||||||
from ix.pipeline.setup_step import SetupStep
|
from ix.pipeline.setup_step import SetupStep
|
||||||
from ix.ui import build_router as build_ui_router
|
|
||||||
from ix.ui.routes import STATIC_DIR as UI_STATIC_DIR
|
|
||||||
|
|
||||||
|
|
||||||
def build_pipeline(
|
def build_pipeline(
|
||||||
|
|
@ -108,20 +105,6 @@ def _make_ocr_probe(ocr: OCRClient) -> Callable[[], Literal["ok", "fail"]]:
|
||||||
return probe
|
return probe
|
||||||
|
|
||||||
|
|
||||||
def _make_ocr_gpu_probe(ocr: OCRClient) -> Callable[[], bool | None]:
|
|
||||||
"""Adapter: read the OCR client's recorded ``gpu_available`` attribute.
|
|
||||||
|
|
||||||
The attribute is set by :meth:`SuryaOCRClient.warm_up` on first load.
|
|
||||||
Returns ``None`` when the client has no such attribute (e.g. FakeOCRClient
|
|
||||||
in test mode) or warm_up hasn't happened yet. Never raises.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def probe() -> bool | None:
|
|
||||||
return getattr(ocr, "gpu_available", None)
|
|
||||||
|
|
||||||
return probe
|
|
||||||
|
|
||||||
|
|
||||||
def _run_async_sync(make_coro, *, fallback: str) -> str: # type: ignore[no-untyped-def]
|
def _run_async_sync(make_coro, *, fallback: str) -> str: # type: ignore[no-untyped-def]
|
||||||
"""Run ``make_coro()`` on a fresh loop in a thread; return its result.
|
"""Run ``make_coro()`` on a fresh loop in a thread; return its result.
|
||||||
|
|
||||||
|
|
@ -181,7 +164,6 @@ def create_app(*, spawn_worker: bool = True) -> FastAPI:
|
||||||
lambda: Probes(
|
lambda: Probes(
|
||||||
ollama=_make_ollama_probe(genai_client, cfg),
|
ollama=_make_ollama_probe(genai_client, cfg),
|
||||||
ocr=_make_ocr_probe(ocr_client),
|
ocr=_make_ocr_probe(ocr_client),
|
||||||
ocr_gpu=_make_ocr_gpu_probe(ocr_client),
|
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -220,16 +202,6 @@ def create_app(*, spawn_worker: bool = True) -> FastAPI:
|
||||||
|
|
||||||
app = FastAPI(lifespan=lifespan, title="infoxtractor", version="0.1.0")
|
app = FastAPI(lifespan=lifespan, title="infoxtractor", version="0.1.0")
|
||||||
app.include_router(rest_router)
|
app.include_router(rest_router)
|
||||||
# Browser UI — additive, never touches the REST paths above.
|
|
||||||
app.include_router(build_ui_router())
|
|
||||||
# Static assets for the UI. CDN-only for MVP so the directory is
|
|
||||||
# essentially empty, but the mount must exist so relative asset
|
|
||||||
# URLs resolve cleanly.
|
|
||||||
app.mount(
|
|
||||||
"/ui/static",
|
|
||||||
StaticFiles(directory=str(UI_STATIC_DIR)),
|
|
||||||
name="ui-static",
|
|
||||||
)
|
|
||||||
return app
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -22,11 +22,6 @@ class FileRef(BaseModel):
|
||||||
Used when the file URL needs authentication (e.g. Paperless ``Token``) or a
|
Used when the file URL needs authentication (e.g. Paperless ``Token``) or a
|
||||||
tighter size cap than :envvar:`IX_FILE_MAX_BYTES`. Plain URLs that need no
|
tighter size cap than :envvar:`IX_FILE_MAX_BYTES`. Plain URLs that need no
|
||||||
headers can stay as bare ``str`` values in :attr:`Context.files`.
|
headers can stay as bare ``str`` values in :attr:`Context.files`.
|
||||||
|
|
||||||
``display_name`` is pure UI metadata — the pipeline never consults it for
|
|
||||||
execution. When the UI uploads a PDF under a random ``{uuid}.pdf`` name on
|
|
||||||
disk, it stashes the client-provided filename here so the browser can
|
|
||||||
surface "your_statement.pdf" instead of "8f3a...pdf" back to the user.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
@ -34,7 +29,6 @@ class FileRef(BaseModel):
|
||||||
url: str
|
url: str
|
||||||
headers: dict[str, str] = Field(default_factory=dict)
|
headers: dict[str, str] = Field(default_factory=dict)
|
||||||
max_bytes: int | None = None
|
max_bytes: int | None = None
|
||||||
display_name: str | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class Context(BaseModel):
|
class Context(BaseModel):
|
||||||
|
|
@ -89,44 +83,6 @@ class Options(BaseModel):
|
||||||
provenance: ProvenanceOptions = Field(default_factory=ProvenanceOptions)
|
provenance: ProvenanceOptions = Field(default_factory=ProvenanceOptions)
|
||||||
|
|
||||||
|
|
||||||
class UseCaseFieldDef(BaseModel):
|
|
||||||
"""One field in an ad-hoc, caller-defined extraction schema.
|
|
||||||
|
|
||||||
The UI (and any other caller that doesn't want to wait on a backend
|
|
||||||
registry entry) ships one of these per desired output field. The pipeline
|
|
||||||
builds a fresh Pydantic response class from the list on each request.
|
|
||||||
|
|
||||||
``choices`` only applies to ``type == "str"`` — it turns the field into a
|
|
||||||
``Literal[*choices]``. For any other type the builder raises
|
|
||||||
``IX_001_001``.
|
|
||||||
"""
|
|
||||||
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
name: str # must be a valid Python identifier
|
|
||||||
type: Literal["str", "int", "float", "decimal", "date", "datetime", "bool"]
|
|
||||||
required: bool = False
|
|
||||||
description: str | None = None
|
|
||||||
choices: list[str] | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class InlineUseCase(BaseModel):
|
|
||||||
"""Caller-defined use case bundled into the :class:`RequestIX`.
|
|
||||||
|
|
||||||
When present on a request, the pipeline builds the ``(Request, Response)``
|
|
||||||
Pydantic class pair on the fly from :attr:`fields` and skips the
|
|
||||||
registered use-case lookup. The registry-based ``use_case`` field is still
|
|
||||||
required on the request for metrics/logging but becomes a free-form label.
|
|
||||||
"""
|
|
||||||
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
use_case_name: str
|
|
||||||
system_prompt: str
|
|
||||||
default_model: str | None = None
|
|
||||||
fields: list[UseCaseFieldDef]
|
|
||||||
|
|
||||||
|
|
||||||
class RequestIX(BaseModel):
|
class RequestIX(BaseModel):
|
||||||
"""Top-level job request.
|
"""Top-level job request.
|
||||||
|
|
||||||
|
|
@ -134,12 +90,6 @@ class RequestIX(BaseModel):
|
||||||
it; the REST adapter / pg-queue adapter populates it on insert. The field
|
it; the REST adapter / pg-queue adapter populates it on insert. The field
|
||||||
is kept here so the contract is closed-over-construction round-trips
|
is kept here so the contract is closed-over-construction round-trips
|
||||||
(e.g. when the worker re-hydrates a job out of the store).
|
(e.g. when the worker re-hydrates a job out of the store).
|
||||||
|
|
||||||
When ``use_case_inline`` is present, the pipeline uses it verbatim to
|
|
||||||
build an ad-hoc ``(Request, Response)`` class pair and skips the registry
|
|
||||||
lookup; ``use_case`` becomes a free-form label (still required for
|
|
||||||
metrics/logging). When ``use_case_inline`` is absent, ``use_case`` is
|
|
||||||
looked up in :data:`ix.use_cases.REGISTRY` as before.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
@ -151,4 +101,3 @@ class RequestIX(BaseModel):
|
||||||
context: Context
|
context: Context
|
||||||
options: Options = Field(default_factory=Options)
|
options: Options = Field(default_factory=Options)
|
||||||
callback_url: str | None = None
|
callback_url: str | None = None
|
||||||
use_case_inline: InlineUseCase | None = None
|
|
||||||
|
|
|
||||||
|
|
@ -96,9 +96,8 @@ class OllamaClient:
|
||||||
) from exc
|
) from exc
|
||||||
|
|
||||||
content = (payload.get("message") or {}).get("content") or ""
|
content = (payload.get("message") or {}).get("content") or ""
|
||||||
json_blob = _extract_json_blob(content)
|
|
||||||
try:
|
try:
|
||||||
parsed = response_schema.model_validate_json(json_blob)
|
parsed = response_schema.model_validate_json(content)
|
||||||
except ValidationError as exc:
|
except ValidationError as exc:
|
||||||
raise IXException(
|
raise IXException(
|
||||||
IXErrorCode.IX_002_001,
|
IXErrorCode.IX_002_001,
|
||||||
|
|
@ -160,39 +159,16 @@ class OllamaClient:
|
||||||
request_kwargs: dict[str, Any],
|
request_kwargs: dict[str, Any],
|
||||||
response_schema: type[BaseModel],
|
response_schema: type[BaseModel],
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Map provider-neutral kwargs to Ollama's /api/chat body.
|
"""Map provider-neutral kwargs to Ollama's /api/chat body."""
|
||||||
|
|
||||||
Schema strategy for Ollama 0.11.8: we pass ``format="json"`` (loose
|
|
||||||
JSON mode) and bake the Pydantic schema into a system message
|
|
||||||
ahead of the caller's own system prompt. Rationale:
|
|
||||||
|
|
||||||
* The full Pydantic schema as ``format=<schema>`` crashes llama.cpp's
|
|
||||||
structured-output implementation (SIGSEGV) on every non-trivial
|
|
||||||
shape — ``anyOf`` / ``$ref`` / ``pattern`` all trigger it.
|
|
||||||
* ``format="json"`` alone guarantees valid JSON but not the shape;
|
|
||||||
models routinely return ``{}`` when not told what fields to emit.
|
|
||||||
* Injecting the schema into the prompt is the cheapest way to
|
|
||||||
get both: the model sees the expected shape explicitly, Pydantic
|
|
||||||
validates the response at parse time (IX_002_001 on mismatch).
|
|
||||||
|
|
||||||
Non-Ollama ``GenAIClient`` impls can ignore this behaviour and use
|
|
||||||
native structured-output (``response_format`` on OpenAI, etc.).
|
|
||||||
"""
|
|
||||||
|
|
||||||
messages = self._translate_messages(
|
messages = self._translate_messages(
|
||||||
list(request_kwargs.get("messages") or [])
|
list(request_kwargs.get("messages") or [])
|
||||||
)
|
)
|
||||||
messages = _inject_schema_system_message(messages, response_schema)
|
|
||||||
body: dict[str, Any] = {
|
body: dict[str, Any] = {
|
||||||
"model": request_kwargs.get("model"),
|
"model": request_kwargs.get("model"),
|
||||||
"messages": messages,
|
"messages": messages,
|
||||||
"stream": False,
|
"stream": False,
|
||||||
# NOTE: format is deliberately omitted. `format="json"` made
|
"format": response_schema.model_json_schema(),
|
||||||
# reasoning models (qwen3) abort after emitting `{}` because the
|
|
||||||
# constrained sampler terminated before the chain-of-thought
|
|
||||||
# finished; `format=<schema>` segfaulted Ollama 0.11.8. Letting
|
|
||||||
# the model stream freely and then extracting the trailing JSON
|
|
||||||
# blob works for both reasoning and non-reasoning models.
|
|
||||||
}
|
}
|
||||||
|
|
||||||
options: dict[str, Any] = {}
|
options: dict[str, Any] = {}
|
||||||
|
|
@ -224,117 +200,4 @@ class OllamaClient:
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def _extract_json_blob(text: str) -> str:
|
|
||||||
"""Return the outermost balanced JSON object in ``text``.
|
|
||||||
|
|
||||||
Reasoning models (qwen3, deepseek-r1) wrap their real answer in
|
|
||||||
``<think>…</think>`` blocks. Other models sometimes prefix prose or
|
|
||||||
fence the JSON in ```json``` code blocks. Finding the last balanced
|
|
||||||
``{…}`` is the cheapest robust parse that works for all three shapes;
|
|
||||||
a malformed response yields the full text and Pydantic catches it
|
|
||||||
downstream as ``IX_002_001``.
|
|
||||||
"""
|
|
||||||
start = text.find("{")
|
|
||||||
if start < 0:
|
|
||||||
return text
|
|
||||||
depth = 0
|
|
||||||
in_string = False
|
|
||||||
escaped = False
|
|
||||||
for i in range(start, len(text)):
|
|
||||||
ch = text[i]
|
|
||||||
if in_string:
|
|
||||||
if escaped:
|
|
||||||
escaped = False
|
|
||||||
elif ch == "\\":
|
|
||||||
escaped = True
|
|
||||||
elif ch == '"':
|
|
||||||
in_string = False
|
|
||||||
continue
|
|
||||||
if ch == '"':
|
|
||||||
in_string = True
|
|
||||||
elif ch == "{":
|
|
||||||
depth += 1
|
|
||||||
elif ch == "}":
|
|
||||||
depth -= 1
|
|
||||||
if depth == 0:
|
|
||||||
return text[start : i + 1]
|
|
||||||
return text[start:]
|
|
||||||
|
|
||||||
|
|
||||||
def _inject_schema_system_message(
|
|
||||||
messages: list[dict[str, Any]],
|
|
||||||
response_schema: type[BaseModel],
|
|
||||||
) -> list[dict[str, Any]]:
|
|
||||||
"""Prepend a system message that pins the expected JSON shape.
|
|
||||||
|
|
||||||
Ollama's ``format="json"`` mode guarantees valid JSON but not the
|
|
||||||
field set or names. We emit the Pydantic schema as JSON and
|
|
||||||
instruct the model to match it. If the caller already provides a
|
|
||||||
system message, we prepend ours; otherwise ours becomes the first
|
|
||||||
system turn.
|
|
||||||
"""
|
|
||||||
import json as _json
|
|
||||||
|
|
||||||
schema_json = _json.dumps(
|
|
||||||
_sanitise_schema_for_ollama(response_schema.model_json_schema()),
|
|
||||||
indent=2,
|
|
||||||
)
|
|
||||||
guidance = (
|
|
||||||
"Respond ONLY with a single JSON object matching this JSON Schema "
|
|
||||||
"exactly. No prose, no code fences, no explanations. All top-level "
|
|
||||||
"properties listed in `required` MUST be present. Use null for "
|
|
||||||
"fields you cannot confidently extract. The JSON Schema:\n"
|
|
||||||
f"{schema_json}"
|
|
||||||
)
|
|
||||||
return [{"role": "system", "content": guidance}, *messages]
|
|
||||||
|
|
||||||
|
|
||||||
def _sanitise_schema_for_ollama(schema: Any) -> Any:
|
|
||||||
"""Strip null branches from ``anyOf`` unions.
|
|
||||||
|
|
||||||
Ollama 0.11.8's llama.cpp structured-output implementation segfaults on
|
|
||||||
Pydantic v2's standard Optional pattern::
|
|
||||||
|
|
||||||
{"anyOf": [{"type": "string"}, {"type": "null"}]}
|
|
||||||
|
|
||||||
We collapse any ``anyOf`` that includes a ``{"type": "null"}`` entry to
|
|
||||||
its non-null branch — single branch becomes that branch inline; multiple
|
|
||||||
branches keep the union without null. This only narrows what the LLM is
|
|
||||||
*told* it may emit; Pydantic still validates the real response and can
|
|
||||||
accept ``None`` at parse time if the field is ``Optional``.
|
|
||||||
|
|
||||||
Walk is recursive and structure-preserving. Other ``anyOf`` shapes (e.g.
|
|
||||||
polymorphic unions without null) are left alone.
|
|
||||||
"""
|
|
||||||
if isinstance(schema, dict):
|
|
||||||
cleaned: dict[str, Any] = {}
|
|
||||||
for key, value in schema.items():
|
|
||||||
if key == "anyOf" and isinstance(value, list):
|
|
||||||
non_null = [
|
|
||||||
_sanitise_schema_for_ollama(branch)
|
|
||||||
for branch in value
|
|
||||||
if not (isinstance(branch, dict) and branch.get("type") == "null")
|
|
||||||
]
|
|
||||||
if len(non_null) == 1:
|
|
||||||
# Inline the single remaining branch; merge its keys into the
|
|
||||||
# parent so siblings like ``default``/``title`` are preserved.
|
|
||||||
only = non_null[0]
|
|
||||||
if isinstance(only, dict):
|
|
||||||
for ok, ov in only.items():
|
|
||||||
cleaned.setdefault(ok, ov)
|
|
||||||
else:
|
|
||||||
cleaned[key] = non_null
|
|
||||||
elif len(non_null) == 0:
|
|
||||||
# Pathological: nothing left. Fall back to a permissive type.
|
|
||||||
cleaned["type"] = "string"
|
|
||||||
else:
|
|
||||||
cleaned[key] = non_null
|
|
||||||
else:
|
|
||||||
cleaned[key] = _sanitise_schema_for_ollama(value)
|
|
||||||
return cleaned
|
|
||||||
if isinstance(schema, list):
|
|
||||||
return [_sanitise_schema_for_ollama(item) for item in schema]
|
|
||||||
return schema
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["OllamaClient"]
|
__all__ = ["OllamaClient"]
|
||||||
|
|
|
||||||
|
|
@ -48,11 +48,6 @@ class SuryaOCRClient:
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self._recognition_predictor: Any = None
|
self._recognition_predictor: Any = None
|
||||||
self._detection_predictor: Any = None
|
self._detection_predictor: Any = None
|
||||||
# ``None`` until warm_up() has run at least once. After that it's the
|
|
||||||
# observed value of ``torch.cuda.is_available()`` at load time. We
|
|
||||||
# cache it on the instance so ``/healthz`` / the UI can surface a
|
|
||||||
# CPU-mode warning without re-probing torch each request.
|
|
||||||
self.gpu_available: bool | None = None
|
|
||||||
|
|
||||||
def warm_up(self) -> None:
|
def warm_up(self) -> None:
|
||||||
"""Load the detection + recognition predictors. Idempotent.
|
"""Load the detection + recognition predictors. Idempotent.
|
||||||
|
|
@ -77,18 +72,6 @@ class SuryaOCRClient:
|
||||||
self._recognition_predictor = RecognitionPredictor(foundation)
|
self._recognition_predictor = RecognitionPredictor(foundation)
|
||||||
self._detection_predictor = DetectionPredictor()
|
self._detection_predictor = DetectionPredictor()
|
||||||
|
|
||||||
# Best-effort CUDA probe — only after predictors loaded cleanly so we
|
|
||||||
# know torch is fully importable. ``torch`` is a Surya transitive
|
|
||||||
# dependency so if we got this far it's on sys.path. We swallow any
|
|
||||||
# exception to keep warm_up() sturdy: the attribute stays None and the
|
|
||||||
# UI falls back to "unknown" gracefully.
|
|
||||||
try:
|
|
||||||
import torch # type: ignore[import-not-found]
|
|
||||||
|
|
||||||
self.gpu_available = bool(torch.cuda.is_available())
|
|
||||||
except Exception:
|
|
||||||
self.gpu_available = None
|
|
||||||
|
|
||||||
async def ocr(
|
async def ocr(
|
||||||
self,
|
self,
|
||||||
pages: list[Page],
|
pages: list[Page],
|
||||||
|
|
|
||||||
|
|
@ -34,7 +34,6 @@ from ix.ingestion import (
|
||||||
)
|
)
|
||||||
from ix.pipeline.step import Step
|
from ix.pipeline.step import Step
|
||||||
from ix.use_cases import get_use_case
|
from ix.use_cases import get_use_case
|
||||||
from ix.use_cases.inline import build_use_case_classes
|
|
||||||
|
|
||||||
|
|
||||||
class _Fetcher(Protocol):
|
class _Fetcher(Protocol):
|
||||||
|
|
@ -89,18 +88,9 @@ class SetupStep(Step):
|
||||||
async def process(
|
async def process(
|
||||||
self, request_ix: RequestIX, response_ix: ResponseIX
|
self, request_ix: RequestIX, response_ix: ResponseIX
|
||||||
) -> ResponseIX:
|
) -> ResponseIX:
|
||||||
# 1. Load the use-case pair — either from the caller's inline
|
# 1. Load the use-case pair — early so an unknown name fails before
|
||||||
# definition (wins over registry) or from the registry by name.
|
# we waste time downloading files.
|
||||||
# Done early so an unknown name / bad inline definition fails
|
use_case_request_cls, use_case_response_cls = get_use_case(request_ix.use_case)
|
||||||
# before we waste time downloading files.
|
|
||||||
if request_ix.use_case_inline is not None:
|
|
||||||
use_case_request_cls, use_case_response_cls = build_use_case_classes(
|
|
||||||
request_ix.use_case_inline
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
use_case_request_cls, use_case_response_cls = get_use_case(
|
|
||||||
request_ix.use_case
|
|
||||||
)
|
|
||||||
use_case_request = use_case_request_cls()
|
use_case_request = use_case_request_cls()
|
||||||
|
|
||||||
# 2. Resolve the per-request scratch directory. ix_id is assigned
|
# 2. Resolve the per-request scratch directory. ix_id is assigned
|
||||||
|
|
|
||||||
|
|
@ -23,7 +23,6 @@ A few invariants worth stating up front:
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import secrets
|
import secrets
|
||||||
from collections.abc import Iterable
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from typing import TYPE_CHECKING, Literal
|
from typing import TYPE_CHECKING, Literal
|
||||||
from uuid import UUID, uuid4
|
from uuid import UUID, uuid4
|
||||||
|
|
@ -158,76 +157,6 @@ async def get(session: AsyncSession, job_id: UUID) -> Job | None:
|
||||||
return _orm_to_job(row) if row is not None else None
|
return _orm_to_job(row) if row is not None else None
|
||||||
|
|
||||||
|
|
||||||
async def queue_position(
|
|
||||||
session: AsyncSession, job_id: UUID
|
|
||||||
) -> tuple[int, int]:
|
|
||||||
"""Return ``(ahead, total_active)`` for a pending/running job.
|
|
||||||
|
|
||||||
``ahead`` counts active jobs (``pending`` or ``running``) that would be
|
|
||||||
claimed by the worker before this one:
|
|
||||||
|
|
||||||
* any ``running`` job is always ahead — it has the worker already.
|
|
||||||
* other ``pending`` jobs with a strictly older ``created_at`` are ahead
|
|
||||||
(the worker picks pending rows in ``ORDER BY created_at`` per
|
|
||||||
:func:`claim_next_pending`).
|
|
||||||
|
|
||||||
``total_active`` is the total count of ``pending`` + ``running`` rows.
|
|
||||||
|
|
||||||
Terminal jobs (``done`` / ``error``) always return ``(0, 0)`` — there is
|
|
||||||
no meaningful "position" for a finished job.
|
|
||||||
"""
|
|
||||||
|
|
||||||
row = await session.scalar(select(IxJob).where(IxJob.job_id == job_id))
|
|
||||||
if row is None:
|
|
||||||
return (0, 0)
|
|
||||||
if row.status not in ("pending", "running"):
|
|
||||||
return (0, 0)
|
|
||||||
|
|
||||||
total_active = int(
|
|
||||||
await session.scalar(
|
|
||||||
select(func.count())
|
|
||||||
.select_from(IxJob)
|
|
||||||
.where(IxJob.status.in_(("pending", "running")))
|
|
||||||
)
|
|
||||||
or 0
|
|
||||||
)
|
|
||||||
|
|
||||||
if row.status == "running":
|
|
||||||
# A running row is at the head of the queue for our purposes.
|
|
||||||
return (0, total_active)
|
|
||||||
|
|
||||||
# Pending: count running rows (always ahead) + older pending rows.
|
|
||||||
# We tiebreak on ``job_id`` for deterministic ordering when multiple
|
|
||||||
# rows share a ``created_at`` (e.g. the same transaction inserts two
|
|
||||||
# jobs, which Postgres stamps with identical ``now()`` values).
|
|
||||||
running_ahead = int(
|
|
||||||
await session.scalar(
|
|
||||||
select(func.count())
|
|
||||||
.select_from(IxJob)
|
|
||||||
.where(IxJob.status == "running")
|
|
||||||
)
|
|
||||||
or 0
|
|
||||||
)
|
|
||||||
pending_ahead = int(
|
|
||||||
await session.scalar(
|
|
||||||
select(func.count())
|
|
||||||
.select_from(IxJob)
|
|
||||||
.where(
|
|
||||||
IxJob.status == "pending",
|
|
||||||
(
|
|
||||||
(IxJob.created_at < row.created_at)
|
|
||||||
| (
|
|
||||||
(IxJob.created_at == row.created_at)
|
|
||||||
& (IxJob.job_id < row.job_id)
|
|
||||||
)
|
|
||||||
),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
or 0
|
|
||||||
)
|
|
||||||
return (running_ahead + pending_ahead, total_active)
|
|
||||||
|
|
||||||
|
|
||||||
async def get_by_correlation(
|
async def get_by_correlation(
|
||||||
session: AsyncSession, client_id: str, request_id: str
|
session: AsyncSession, client_id: str, request_id: str
|
||||||
) -> Job | None:
|
) -> Job | None:
|
||||||
|
|
@ -334,75 +263,6 @@ async def sweep_orphans(
|
||||||
return list(candidates)
|
return list(candidates)
|
||||||
|
|
||||||
|
|
||||||
_LIST_RECENT_LIMIT_CAP = 200
|
|
||||||
|
|
||||||
|
|
||||||
async def list_recent(
|
|
||||||
session: AsyncSession,
|
|
||||||
*,
|
|
||||||
limit: int = 50,
|
|
||||||
offset: int = 0,
|
|
||||||
status: str | Iterable[str] | None = None,
|
|
||||||
client_id: str | None = None,
|
|
||||||
) -> tuple[list[Job], int]:
|
|
||||||
"""Return a page of recent jobs, newest first, plus total matching count.
|
|
||||||
|
|
||||||
Powers the ``/ui/jobs`` listing page. Ordering is ``created_at DESC``.
|
|
||||||
``total`` reflects matching rows *before* limit/offset so the template
|
|
||||||
can render "showing N of M".
|
|
||||||
|
|
||||||
Parameters
|
|
||||||
----------
|
|
||||||
limit:
|
|
||||||
Maximum rows to return. Capped at
|
|
||||||
:data:`_LIST_RECENT_LIMIT_CAP` (200) to bound the JSON payload
|
|
||||||
size — callers that pass a larger value get clamped silently.
|
|
||||||
offset:
|
|
||||||
Non-negative row offset. Negative values raise ``ValueError``
|
|
||||||
because the template treats offset as a page cursor; a negative
|
|
||||||
cursor is a bug at the call site, not something to paper over.
|
|
||||||
status:
|
|
||||||
If set, restrict to the given status(es). Accepts a single
|
|
||||||
:data:`Job.status` value or any iterable (list/tuple/set). Values
|
|
||||||
outside the lifecycle enum simply match nothing — we don't try
|
|
||||||
to validate here; the DB CHECK constraint already bounds the set.
|
|
||||||
client_id:
|
|
||||||
If set, exact match on :attr:`IxJob.client_id`. No substring /
|
|
||||||
prefix match — simple and predictable.
|
|
||||||
"""
|
|
||||||
|
|
||||||
if offset < 0:
|
|
||||||
raise ValueError(f"offset must be >= 0, got {offset}")
|
|
||||||
effective_limit = max(0, min(limit, _LIST_RECENT_LIMIT_CAP))
|
|
||||||
|
|
||||||
filters = []
|
|
||||||
if status is not None:
|
|
||||||
if isinstance(status, str):
|
|
||||||
filters.append(IxJob.status == status)
|
|
||||||
else:
|
|
||||||
status_list = list(status)
|
|
||||||
if not status_list:
|
|
||||||
# Empty iterable → no rows match. Return a sentinel
|
|
||||||
# IN-list that can never hit so we don't blow up.
|
|
||||||
filters.append(IxJob.status.in_(status_list))
|
|
||||||
else:
|
|
||||||
filters.append(IxJob.status.in_(status_list))
|
|
||||||
if client_id is not None:
|
|
||||||
filters.append(IxJob.client_id == client_id)
|
|
||||||
|
|
||||||
total_q = select(func.count()).select_from(IxJob)
|
|
||||||
list_q = select(IxJob).order_by(IxJob.created_at.desc())
|
|
||||||
for f in filters:
|
|
||||||
total_q = total_q.where(f)
|
|
||||||
list_q = list_q.where(f)
|
|
||||||
|
|
||||||
total = int(await session.scalar(total_q) or 0)
|
|
||||||
rows = (
|
|
||||||
await session.scalars(list_q.limit(effective_limit).offset(offset))
|
|
||||||
).all()
|
|
||||||
return [_orm_to_job(r) for r in rows], total
|
|
||||||
|
|
||||||
|
|
||||||
def _as_interval(seconds: int): # type: ignore[no-untyped-def]
|
def _as_interval(seconds: int): # type: ignore[no-untyped-def]
|
||||||
"""Return a SQL interval expression for ``seconds``.
|
"""Return a SQL interval expression for ``seconds``.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,13 +0,0 @@
|
||||||
"""Minimal browser UI served alongside the REST API at ``/ui``.
|
|
||||||
|
|
||||||
The module is intentionally thin: templates + HTMX + Pico CSS (all from
|
|
||||||
CDNs, no build step). Uploads land in ``{cfg.tmp_dir}/ui/<uuid>.pdf`` and
|
|
||||||
are submitted through the same :func:`ix.store.jobs_repo.insert_pending`
|
|
||||||
entry point the REST adapter uses — the UI does not duplicate that logic.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from ix.ui.routes import build_router
|
|
||||||
|
|
||||||
__all__ = ["build_router"]
|
|
||||||
|
|
@ -1,568 +0,0 @@
|
||||||
"""``/ui`` router — thin HTML wrapper over the existing jobs pipeline.
|
|
||||||
|
|
||||||
Design notes:
|
|
||||||
|
|
||||||
* Uploads stream to ``{cfg.tmp_dir}/ui/{uuid4()}.pdf`` via aiofiles; the
|
|
||||||
file persists for the lifetime of the ``ix_id`` (no cleanup cron — spec
|
|
||||||
deferred).
|
|
||||||
* The submission handler builds a :class:`RequestIX` (inline use case
|
|
||||||
supported) and inserts it via the same
|
|
||||||
:func:`ix.store.jobs_repo.insert_pending` the REST adapter uses.
|
|
||||||
* Responses are HTML. For HTMX-triggered submissions the handler returns
|
|
||||||
``HX-Redirect`` so the whole page swaps; for plain form posts it returns
|
|
||||||
a 303 redirect.
|
|
||||||
* The fragment endpoint powers the polling loop: while the job is
|
|
||||||
pending/running, the fragment auto-refreshes every 2s via
|
|
||||||
``hx-trigger="every 2s"``; when terminal, the trigger is dropped and the
|
|
||||||
pretty-printed response is rendered with highlight.js.
|
|
||||||
* A process-wide 60-second cache of the OCR GPU flag (read from the
|
|
||||||
injected :class:`Probes`) gates a "Surya is running on CPU" notice on
|
|
||||||
the fragment. The fragment is polled every 2 s; re-probing the OCR
|
|
||||||
client on every poll is waste — one probe per minute is plenty.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import time
|
|
||||||
import uuid
|
|
||||||
from datetime import UTC, datetime
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Annotated
|
|
||||||
from urllib.parse import unquote, urlencode, urlsplit
|
|
||||||
from uuid import UUID
|
|
||||||
|
|
||||||
import aiofiles
|
|
||||||
from fastapi import (
|
|
||||||
APIRouter,
|
|
||||||
Depends,
|
|
||||||
File,
|
|
||||||
Form,
|
|
||||||
HTTPException,
|
|
||||||
Query,
|
|
||||||
Request,
|
|
||||||
UploadFile,
|
|
||||||
)
|
|
||||||
from fastapi.responses import HTMLResponse, RedirectResponse, Response
|
|
||||||
from fastapi.templating import Jinja2Templates
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
|
||||||
|
|
||||||
from ix.adapters.rest.routes import Probes, get_probes, get_session_factory_dep
|
|
||||||
from ix.config import AppConfig, get_config
|
|
||||||
from ix.contracts.request import (
|
|
||||||
Context,
|
|
||||||
FileRef,
|
|
||||||
GenAIOptions,
|
|
||||||
InlineUseCase,
|
|
||||||
OCROptions,
|
|
||||||
Options,
|
|
||||||
ProvenanceOptions,
|
|
||||||
RequestIX,
|
|
||||||
UseCaseFieldDef,
|
|
||||||
)
|
|
||||||
from ix.store import jobs_repo
|
|
||||||
from ix.use_cases import REGISTRY
|
|
||||||
|
|
||||||
TEMPLATES_DIR = Path(__file__).parent / "templates"
|
|
||||||
STATIC_DIR = Path(__file__).parent / "static"
|
|
||||||
|
|
||||||
# Module-level cache for the OCR GPU flag. The tuple is ``(value, expires_at)``
|
|
||||||
# where ``expires_at`` is a monotonic-clock deadline. A per-request call to
|
|
||||||
# :func:`_cached_ocr_gpu` re-probes only once the deadline has passed.
|
|
||||||
_OCR_GPU_CACHE: tuple[bool | None, float] = (None, 0.0)
|
|
||||||
_OCR_GPU_TTL_SECONDS = 60.0
|
|
||||||
|
|
||||||
|
|
||||||
def _templates() -> Jinja2Templates:
|
|
||||||
"""One Jinja env per process; cheap enough to build per DI call."""
|
|
||||||
|
|
||||||
return Jinja2Templates(directory=str(TEMPLATES_DIR))
|
|
||||||
|
|
||||||
|
|
||||||
def _ui_tmp_dir(cfg: AppConfig) -> Path:
|
|
||||||
"""Where uploads land. Created on first use; never cleaned up."""
|
|
||||||
|
|
||||||
d = Path(cfg.tmp_dir) / "ui"
|
|
||||||
d.mkdir(parents=True, exist_ok=True)
|
|
||||||
return d
|
|
||||||
|
|
||||||
|
|
||||||
def _cached_ocr_gpu(probes: Probes) -> bool | None:
|
|
||||||
"""Read the cached OCR GPU flag, re-probing if the TTL has elapsed.
|
|
||||||
|
|
||||||
Used by the index + fragment routes so the HTMX poll loop doesn't hit
|
|
||||||
the OCR client's torch-probe every 2 seconds. Falls back to ``None``
|
|
||||||
(unknown) on any probe error.
|
|
||||||
"""
|
|
||||||
|
|
||||||
global _OCR_GPU_CACHE
|
|
||||||
value, expires_at = _OCR_GPU_CACHE
|
|
||||||
now = time.monotonic()
|
|
||||||
if now >= expires_at:
|
|
||||||
try:
|
|
||||||
value = probes.ocr_gpu()
|
|
||||||
except Exception:
|
|
||||||
value = None
|
|
||||||
_OCR_GPU_CACHE = (value, now + _OCR_GPU_TTL_SECONDS)
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
_VALID_STATUSES = ("pending", "running", "done", "error")
|
|
||||||
_JOBS_LIST_DEFAULT_LIMIT = 50
|
|
||||||
_JOBS_LIST_MAX_LIMIT = 200
|
|
||||||
|
|
||||||
|
|
||||||
def _use_case_label(request: RequestIX | None) -> str:
|
|
||||||
"""Prefer inline use-case label, fall back to the registered name."""
|
|
||||||
|
|
||||||
if request is None:
|
|
||||||
return "—"
|
|
||||||
if request.use_case_inline is not None:
|
|
||||||
return request.use_case_inline.use_case_name or request.use_case
|
|
||||||
return request.use_case or "—"
|
|
||||||
|
|
||||||
|
|
||||||
def _row_elapsed_seconds(job) -> int | None: # type: ignore[no-untyped-def]
|
|
||||||
"""Wall-clock seconds for a terminal row (finished - started).
|
|
||||||
|
|
||||||
Used in the list view's "Elapsed" column. Returns ``None`` for rows
|
|
||||||
that haven't run yet (pending / running-with-missing-started_at) so
|
|
||||||
the template can render ``—`` instead.
|
|
||||||
"""
|
|
||||||
|
|
||||||
if job.status in ("done", "error") and job.started_at and job.finished_at:
|
|
||||||
return max(0, int((job.finished_at - job.started_at).total_seconds()))
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _humanize_delta(seconds: int) -> str:
|
|
||||||
"""Coarse-grained "N min ago" for the list view.
|
|
||||||
|
|
||||||
The list renders many rows; we don't need second-accuracy here. For
|
|
||||||
sub-minute values we still say "just now" to avoid a jumpy display.
|
|
||||||
"""
|
|
||||||
|
|
||||||
if seconds < 45:
|
|
||||||
return "just now"
|
|
||||||
mins = seconds // 60
|
|
||||||
if mins < 60:
|
|
||||||
return f"{mins} min ago"
|
|
||||||
hours = mins // 60
|
|
||||||
if hours < 24:
|
|
||||||
return f"{hours} h ago"
|
|
||||||
days = hours // 24
|
|
||||||
return f"{days} d ago"
|
|
||||||
|
|
||||||
|
|
||||||
def _fmt_elapsed_seconds(seconds: int | None) -> str:
|
|
||||||
if seconds is None:
|
|
||||||
return "—"
|
|
||||||
return f"{seconds // 60:02d}:{seconds % 60:02d}"
|
|
||||||
|
|
||||||
|
|
||||||
def _file_display_entries(
|
|
||||||
request: RequestIX | None,
|
|
||||||
) -> list[str]:
|
|
||||||
"""Human-readable filename(s) for a request's context.files.
|
|
||||||
|
|
||||||
Prefers :attr:`FileRef.display_name`. Falls back to the URL's basename
|
|
||||||
(``unquote``ed so ``%20`` → space). Plain string entries use the same
|
|
||||||
basename rule. Empty list for a request with no files.
|
|
||||||
"""
|
|
||||||
|
|
||||||
if request is None:
|
|
||||||
return []
|
|
||||||
out: list[str] = []
|
|
||||||
for entry in request.context.files:
|
|
||||||
if isinstance(entry, FileRef):
|
|
||||||
if entry.display_name:
|
|
||||||
out.append(entry.display_name)
|
|
||||||
continue
|
|
||||||
url = entry.url
|
|
||||||
else:
|
|
||||||
url = entry
|
|
||||||
basename = unquote(urlsplit(url).path.rsplit("/", 1)[-1]) or url
|
|
||||||
out.append(basename)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def build_router() -> APIRouter:
|
|
||||||
"""Return a fresh router. Kept as a factory so :mod:`ix.app` can wire DI."""
|
|
||||||
|
|
||||||
router = APIRouter(prefix="/ui", tags=["ui"])
|
|
||||||
|
|
||||||
@router.get("", response_class=HTMLResponse)
|
|
||||||
@router.get("/", response_class=HTMLResponse)
|
|
||||||
async def index(
|
|
||||||
request: Request,
|
|
||||||
probes: Annotated[Probes, Depends(get_probes)],
|
|
||||||
) -> Response:
|
|
||||||
tpl = _templates()
|
|
||||||
return tpl.TemplateResponse(
|
|
||||||
request,
|
|
||||||
"index.html",
|
|
||||||
{
|
|
||||||
"registered_use_cases": sorted(REGISTRY.keys()),
|
|
||||||
"job": None,
|
|
||||||
"form_error": None,
|
|
||||||
"form_values": {},
|
|
||||||
"file_names": [],
|
|
||||||
"cpu_mode": _cached_ocr_gpu(probes) is False,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
@router.get("/jobs", response_class=HTMLResponse)
|
|
||||||
async def jobs_list(
|
|
||||||
request: Request,
|
|
||||||
session_factory: Annotated[
|
|
||||||
async_sessionmaker[AsyncSession], Depends(get_session_factory_dep)
|
|
||||||
],
|
|
||||||
status: Annotated[list[str] | None, Query()] = None,
|
|
||||||
client_id: Annotated[str | None, Query()] = None,
|
|
||||||
limit: Annotated[int, Query(ge=1, le=_JOBS_LIST_MAX_LIMIT)] = _JOBS_LIST_DEFAULT_LIMIT,
|
|
||||||
offset: Annotated[int, Query(ge=0)] = 0,
|
|
||||||
) -> Response:
|
|
||||||
# Drop unknown statuses silently — we don't want a stray query
|
|
||||||
# param to 400. The filter bar only offers valid values anyway.
|
|
||||||
status_filter: list[str] = []
|
|
||||||
if status:
|
|
||||||
status_filter = [s for s in status if s in _VALID_STATUSES]
|
|
||||||
client_filter = (client_id or "").strip() or None
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
jobs, total = await jobs_repo.list_recent(
|
|
||||||
session,
|
|
||||||
limit=limit,
|
|
||||||
offset=offset,
|
|
||||||
status=status_filter if status_filter else None,
|
|
||||||
client_id=client_filter,
|
|
||||||
)
|
|
||||||
|
|
||||||
now = datetime.now(UTC)
|
|
||||||
rows = []
|
|
||||||
for job in jobs:
|
|
||||||
files = _file_display_entries(job.request)
|
|
||||||
display = files[0] if files else "—"
|
|
||||||
created = job.created_at
|
|
||||||
created_delta = _humanize_delta(
|
|
||||||
int((now - created).total_seconds())
|
|
||||||
) if created is not None else "—"
|
|
||||||
created_local = (
|
|
||||||
created.strftime("%Y-%m-%d %H:%M:%S")
|
|
||||||
if created is not None
|
|
||||||
else "—"
|
|
||||||
)
|
|
||||||
rows.append(
|
|
||||||
{
|
|
||||||
"job_id": str(job.job_id),
|
|
||||||
"status": job.status,
|
|
||||||
"display_name": display,
|
|
||||||
"use_case": _use_case_label(job.request),
|
|
||||||
"client_id": job.client_id,
|
|
||||||
"created_at": created_local,
|
|
||||||
"created_delta": created_delta,
|
|
||||||
"elapsed": _fmt_elapsed_seconds(_row_elapsed_seconds(job)),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
prev_offset = max(0, offset - limit) if offset > 0 else None
|
|
||||||
next_offset = offset + limit if (offset + limit) < total else None
|
|
||||||
|
|
||||||
def _link(new_offset: int) -> str:
|
|
||||||
params: list[tuple[str, str]] = []
|
|
||||||
for s in status_filter:
|
|
||||||
params.append(("status", s))
|
|
||||||
if client_filter:
|
|
||||||
params.append(("client_id", client_filter))
|
|
||||||
params.append(("limit", str(limit)))
|
|
||||||
params.append(("offset", str(new_offset)))
|
|
||||||
return f"/ui/jobs?{urlencode(params)}"
|
|
||||||
|
|
||||||
tpl = _templates()
|
|
||||||
return tpl.TemplateResponse(
|
|
||||||
request,
|
|
||||||
"jobs_list.html",
|
|
||||||
{
|
|
||||||
"rows": rows,
|
|
||||||
"total": total,
|
|
||||||
"shown": len(rows),
|
|
||||||
"limit": limit,
|
|
||||||
"offset": offset,
|
|
||||||
"status_filter": status_filter,
|
|
||||||
"client_filter": client_filter or "",
|
|
||||||
"valid_statuses": _VALID_STATUSES,
|
|
||||||
"prev_link": _link(prev_offset) if prev_offset is not None else None,
|
|
||||||
"next_link": _link(next_offset) if next_offset is not None else None,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
@router.get("/jobs/{job_id}", response_class=HTMLResponse)
|
|
||||||
async def job_page(
|
|
||||||
request: Request,
|
|
||||||
job_id: UUID,
|
|
||||||
session_factory: Annotated[
|
|
||||||
async_sessionmaker[AsyncSession], Depends(get_session_factory_dep)
|
|
||||||
],
|
|
||||||
probes: Annotated[Probes, Depends(get_probes)],
|
|
||||||
) -> Response:
|
|
||||||
async with session_factory() as session:
|
|
||||||
job = await jobs_repo.get(session, job_id)
|
|
||||||
if job is None:
|
|
||||||
raise HTTPException(status_code=404, detail="job not found")
|
|
||||||
tpl = _templates()
|
|
||||||
return tpl.TemplateResponse(
|
|
||||||
request,
|
|
||||||
"index.html",
|
|
||||||
{
|
|
||||||
"registered_use_cases": sorted(REGISTRY.keys()),
|
|
||||||
"job": job,
|
|
||||||
"form_error": None,
|
|
||||||
"form_values": {},
|
|
||||||
"file_names": _file_display_entries(job.request),
|
|
||||||
"cpu_mode": _cached_ocr_gpu(probes) is False,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
@router.get("/jobs/{job_id}/fragment", response_class=HTMLResponse)
|
|
||||||
async def job_fragment(
|
|
||||||
request: Request,
|
|
||||||
job_id: UUID,
|
|
||||||
session_factory: Annotated[
|
|
||||||
async_sessionmaker[AsyncSession], Depends(get_session_factory_dep)
|
|
||||||
],
|
|
||||||
probes: Annotated[Probes, Depends(get_probes)],
|
|
||||||
) -> Response:
|
|
||||||
async with session_factory() as session:
|
|
||||||
job = await jobs_repo.get(session, job_id)
|
|
||||||
if job is None:
|
|
||||||
raise HTTPException(status_code=404, detail="job not found")
|
|
||||||
ahead, total_active = await jobs_repo.queue_position(
|
|
||||||
session, job_id
|
|
||||||
)
|
|
||||||
|
|
||||||
response_json: str | None = None
|
|
||||||
if job.response is not None:
|
|
||||||
response_json = json.dumps(
|
|
||||||
job.response.model_dump(mode="json"),
|
|
||||||
indent=2,
|
|
||||||
sort_keys=True,
|
|
||||||
default=str,
|
|
||||||
)
|
|
||||||
|
|
||||||
elapsed_text = _format_elapsed(job)
|
|
||||||
file_names = _file_display_entries(job.request)
|
|
||||||
|
|
||||||
tpl = _templates()
|
|
||||||
return tpl.TemplateResponse(
|
|
||||||
request,
|
|
||||||
"job_fragment.html",
|
|
||||||
{
|
|
||||||
"job": job,
|
|
||||||
"response_json": response_json,
|
|
||||||
"ahead": ahead,
|
|
||||||
"total_active": total_active,
|
|
||||||
"elapsed_text": elapsed_text,
|
|
||||||
"file_names": file_names,
|
|
||||||
"cpu_mode": _cached_ocr_gpu(probes) is False,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
@router.post("/jobs")
|
|
||||||
async def submit_job(
|
|
||||||
request: Request,
|
|
||||||
session_factory: Annotated[
|
|
||||||
async_sessionmaker[AsyncSession], Depends(get_session_factory_dep)
|
|
||||||
],
|
|
||||||
pdf: Annotated[UploadFile, File()],
|
|
||||||
use_case_name: Annotated[str, Form()],
|
|
||||||
use_case_mode: Annotated[str, Form()] = "registered",
|
|
||||||
texts: Annotated[str, Form()] = "",
|
|
||||||
ix_client_id: Annotated[str, Form()] = "ui",
|
|
||||||
request_id: Annotated[str, Form()] = "",
|
|
||||||
system_prompt: Annotated[str, Form()] = "",
|
|
||||||
default_model: Annotated[str, Form()] = "",
|
|
||||||
fields_json: Annotated[str, Form()] = "",
|
|
||||||
use_ocr: Annotated[str, Form()] = "",
|
|
||||||
ocr_only: Annotated[str, Form()] = "",
|
|
||||||
include_ocr_text: Annotated[str, Form()] = "",
|
|
||||||
include_geometries: Annotated[str, Form()] = "",
|
|
||||||
gen_ai_model_name: Annotated[str, Form()] = "",
|
|
||||||
include_provenance: Annotated[str, Form()] = "",
|
|
||||||
max_sources_per_field: Annotated[str, Form()] = "10",
|
|
||||||
) -> Response:
|
|
||||||
cfg = get_config()
|
|
||||||
form_values = {
|
|
||||||
"use_case_mode": use_case_mode,
|
|
||||||
"use_case_name": use_case_name,
|
|
||||||
"ix_client_id": ix_client_id,
|
|
||||||
"request_id": request_id,
|
|
||||||
"texts": texts,
|
|
||||||
"system_prompt": system_prompt,
|
|
||||||
"default_model": default_model,
|
|
||||||
"fields_json": fields_json,
|
|
||||||
"use_ocr": use_ocr,
|
|
||||||
"ocr_only": ocr_only,
|
|
||||||
"include_ocr_text": include_ocr_text,
|
|
||||||
"include_geometries": include_geometries,
|
|
||||||
"gen_ai_model_name": gen_ai_model_name,
|
|
||||||
"include_provenance": include_provenance,
|
|
||||||
"max_sources_per_field": max_sources_per_field,
|
|
||||||
}
|
|
||||||
|
|
||||||
def _rerender(error: str, status: int = 200) -> Response:
|
|
||||||
tpl = _templates()
|
|
||||||
return tpl.TemplateResponse(
|
|
||||||
request,
|
|
||||||
"index.html",
|
|
||||||
{
|
|
||||||
"registered_use_cases": sorted(REGISTRY.keys()),
|
|
||||||
"job": None,
|
|
||||||
"form_error": error,
|
|
||||||
"form_values": form_values,
|
|
||||||
},
|
|
||||||
status_code=status,
|
|
||||||
)
|
|
||||||
|
|
||||||
# --- Inline use case (optional) ---
|
|
||||||
inline: InlineUseCase | None = None
|
|
||||||
if use_case_mode == "custom":
|
|
||||||
try:
|
|
||||||
raw_fields = json.loads(fields_json)
|
|
||||||
except json.JSONDecodeError as exc:
|
|
||||||
return _rerender(f"Invalid fields JSON: {exc}", status=422)
|
|
||||||
if not isinstance(raw_fields, list):
|
|
||||||
return _rerender(
|
|
||||||
"Invalid fields JSON: must be a list of field objects",
|
|
||||||
status=422,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
parsed = [UseCaseFieldDef.model_validate(f) for f in raw_fields]
|
|
||||||
inline = InlineUseCase(
|
|
||||||
use_case_name=use_case_name,
|
|
||||||
system_prompt=system_prompt,
|
|
||||||
default_model=default_model or None,
|
|
||||||
fields=parsed,
|
|
||||||
)
|
|
||||||
except Exception as exc: # pydantic ValidationError or similar
|
|
||||||
return _rerender(
|
|
||||||
f"Invalid inline use-case definition: {exc}",
|
|
||||||
status=422,
|
|
||||||
)
|
|
||||||
|
|
||||||
# --- PDF upload ---
|
|
||||||
upload_dir = _ui_tmp_dir(cfg)
|
|
||||||
target = upload_dir / f"{uuid.uuid4().hex}.pdf"
|
|
||||||
# Stream copy with a size cap matching IX_FILE_MAX_BYTES.
|
|
||||||
total = 0
|
|
||||||
limit = cfg.file_max_bytes
|
|
||||||
async with aiofiles.open(target, "wb") as out:
|
|
||||||
while True:
|
|
||||||
chunk = await pdf.read(64 * 1024)
|
|
||||||
if not chunk:
|
|
||||||
break
|
|
||||||
total += len(chunk)
|
|
||||||
if total > limit:
|
|
||||||
# Drop the partial file; no stored state.
|
|
||||||
from contextlib import suppress
|
|
||||||
|
|
||||||
with suppress(FileNotFoundError):
|
|
||||||
target.unlink()
|
|
||||||
return _rerender(
|
|
||||||
f"PDF exceeds IX_FILE_MAX_BYTES ({limit} bytes)",
|
|
||||||
status=413,
|
|
||||||
)
|
|
||||||
await out.write(chunk)
|
|
||||||
|
|
||||||
# --- Build RequestIX ---
|
|
||||||
ctx_texts: list[str] = []
|
|
||||||
if texts.strip():
|
|
||||||
ctx_texts = [texts.strip()]
|
|
||||||
|
|
||||||
req_id = request_id.strip() or uuid.uuid4().hex
|
|
||||||
# Preserve the client-provided filename so the UI can surface the
|
|
||||||
# original name to the user (the on-disk name is a UUID). Strip any
|
|
||||||
# path prefix a browser included.
|
|
||||||
original_name = (pdf.filename or "").rsplit("/", 1)[-1].rsplit(
|
|
||||||
"\\", 1
|
|
||||||
)[-1] or None
|
|
||||||
try:
|
|
||||||
request_ix = RequestIX(
|
|
||||||
use_case=use_case_name or "adhoc",
|
|
||||||
use_case_inline=inline,
|
|
||||||
ix_client_id=(ix_client_id.strip() or "ui"),
|
|
||||||
request_id=req_id,
|
|
||||||
context=Context(
|
|
||||||
files=[
|
|
||||||
FileRef(
|
|
||||||
url=f"file://{target.resolve()}",
|
|
||||||
display_name=original_name,
|
|
||||||
)
|
|
||||||
],
|
|
||||||
texts=ctx_texts,
|
|
||||||
),
|
|
||||||
options=Options(
|
|
||||||
ocr=OCROptions(
|
|
||||||
use_ocr=_flag(use_ocr, default=True),
|
|
||||||
ocr_only=_flag(ocr_only, default=False),
|
|
||||||
include_ocr_text=_flag(include_ocr_text, default=False),
|
|
||||||
include_geometries=_flag(include_geometries, default=False),
|
|
||||||
),
|
|
||||||
gen_ai=GenAIOptions(
|
|
||||||
gen_ai_model_name=(gen_ai_model_name.strip() or None),
|
|
||||||
),
|
|
||||||
provenance=ProvenanceOptions(
|
|
||||||
include_provenance=_flag(include_provenance, default=True),
|
|
||||||
max_sources_per_field=int(max_sources_per_field or 10),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
return _rerender(f"Invalid request: {exc}", status=422)
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
job = await jobs_repo.insert_pending(
|
|
||||||
session, request_ix, callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
redirect_to = f"/ui/jobs/{job.job_id}"
|
|
||||||
if request.headers.get("HX-Request", "").lower() == "true":
|
|
||||||
return Response(status_code=200, headers={"HX-Redirect": redirect_to})
|
|
||||||
return RedirectResponse(url=redirect_to, status_code=303)
|
|
||||||
|
|
||||||
return router
|
|
||||||
|
|
||||||
|
|
||||||
def _flag(value: str, *, default: bool) -> bool:
|
|
||||||
"""HTML forms omit unchecked checkboxes. Treat absence as ``default``."""
|
|
||||||
|
|
||||||
if value == "":
|
|
||||||
return default
|
|
||||||
return value.lower() in ("on", "true", "1", "yes")
|
|
||||||
|
|
||||||
|
|
||||||
def _format_elapsed(job) -> str | None: # type: ignore[no-untyped-def]
|
|
||||||
"""Render a ``MM:SS`` elapsed string for the fragment template.
|
|
||||||
|
|
||||||
* running → time since ``started_at``
|
|
||||||
* done/error → ``finished_at - created_at`` (total wall-clock including
|
|
||||||
queue time)
|
|
||||||
* pending / missing timestamps → ``None`` (template omits the line)
|
|
||||||
"""
|
|
||||||
|
|
||||||
from datetime import UTC, datetime
|
|
||||||
|
|
||||||
def _fmt(seconds: float) -> str:
|
|
||||||
s = max(0, int(seconds))
|
|
||||||
return f"{s // 60:02d}:{s % 60:02d}"
|
|
||||||
|
|
||||||
if job.status == "running" and job.started_at is not None:
|
|
||||||
now = datetime.now(UTC)
|
|
||||||
return _fmt((now - job.started_at).total_seconds())
|
|
||||||
if (
|
|
||||||
job.status in ("done", "error")
|
|
||||||
and job.finished_at is not None
|
|
||||||
and job.created_at is not None
|
|
||||||
):
|
|
||||||
return _fmt((job.finished_at - job.created_at).total_seconds())
|
|
||||||
return None
|
|
||||||
|
|
@ -1,242 +0,0 @@
|
||||||
<!doctype html>
|
|
||||||
<html lang="en" data-theme="light">
|
|
||||||
<head>
|
|
||||||
<meta charset="utf-8" />
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
||||||
<title>
|
|
||||||
InfoXtractor{% if job %} — job {{ job.job_id }}{% endif %}
|
|
||||||
</title>
|
|
||||||
<link
|
|
||||||
rel="stylesheet"
|
|
||||||
href="https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.min.css"
|
|
||||||
/>
|
|
||||||
<link
|
|
||||||
rel="stylesheet"
|
|
||||||
href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.10.0/styles/atom-one-light.min.css"
|
|
||||||
/>
|
|
||||||
<script src="https://unpkg.com/htmx.org@1.9.12"></script>
|
|
||||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.10.0/highlight.min.js"></script>
|
|
||||||
<style>
|
|
||||||
main { padding-top: 1.5rem; padding-bottom: 4rem; }
|
|
||||||
pre code.hljs { padding: 1rem; border-radius: 0.4rem; }
|
|
||||||
.form-error { color: var(--pico-del-color, #c44); font-weight: 600; }
|
|
||||||
details[open] > summary { margin-bottom: 0.5rem; }
|
|
||||||
.field-hint { font-size: 0.85rem; color: var(--pico-muted-color); }
|
|
||||||
nav.ix-header {
|
|
||||||
display: flex; gap: 1rem; align-items: baseline;
|
|
||||||
padding: 0.6rem 0; border-bottom: 1px solid var(--pico-muted-border-color, #ddd);
|
|
||||||
margin-bottom: 1rem; flex-wrap: wrap;
|
|
||||||
}
|
|
||||||
nav.ix-header .brand { font-weight: 700; margin-right: auto; }
|
|
||||||
nav.ix-header code { font-size: 0.9em; }
|
|
||||||
.status-panel, .result-panel { margin-top: 0.75rem; }
|
|
||||||
.status-panel header, .result-panel header { font-size: 0.95rem; }
|
|
||||||
.job-files code { font-size: 0.9em; }
|
|
||||||
.cpu-notice { margin-top: 0.6rem; font-size: 0.9rem; color: var(--pico-muted-color); }
|
|
||||||
.live-dot {
|
|
||||||
display: inline-block; margin-left: 0.3rem;
|
|
||||||
animation: ix-blink 1.2s ease-in-out infinite;
|
|
||||||
color: var(--pico-primary, #4f8cc9);
|
|
||||||
}
|
|
||||||
@keyframes ix-blink {
|
|
||||||
0%, 100% { opacity: 0.2; }
|
|
||||||
50% { opacity: 1; }
|
|
||||||
}
|
|
||||||
.copy-btn {
|
|
||||||
margin-left: 0.3rem; padding: 0.1rem 0.5rem;
|
|
||||||
font-size: 0.8rem; line-height: 1.2;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<main class="container">
|
|
||||||
<nav class="ix-header" aria-label="InfoXtractor navigation">
|
|
||||||
<span class="brand">InfoXtractor</span>
|
|
||||||
<a href="/ui">Upload a new extraction</a>
|
|
||||||
<a href="/ui/jobs">Recent jobs</a>
|
|
||||||
{% if job %}
|
|
||||||
<span>
|
|
||||||
Job:
|
|
||||||
<code id="current-job-id">{{ job.job_id }}</code>
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
class="secondary outline copy-btn"
|
|
||||||
onclick="navigator.clipboard && navigator.clipboard.writeText('{{ job.job_id }}')"
|
|
||||||
aria-label="Copy job id to clipboard"
|
|
||||||
>Copy</button>
|
|
||||||
</span>
|
|
||||||
{% endif %}
|
|
||||||
</nav>
|
|
||||||
|
|
||||||
<hgroup>
|
|
||||||
<h1>infoxtractor</h1>
|
|
||||||
<p>Drop a PDF, pick or define a use case, run the pipeline.</p>
|
|
||||||
</hgroup>
|
|
||||||
|
|
||||||
{% if form_error %}
|
|
||||||
<article class="form-error">
|
|
||||||
<p><strong>Form error:</strong> {{ form_error }}</p>
|
|
||||||
</article>
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
{% if not job %}
|
|
||||||
<article>
|
|
||||||
<form
|
|
||||||
action="/ui/jobs"
|
|
||||||
method="post"
|
|
||||||
enctype="multipart/form-data"
|
|
||||||
hx-post="/ui/jobs"
|
|
||||||
hx-encoding="multipart/form-data"
|
|
||||||
>
|
|
||||||
<label>
|
|
||||||
PDF file
|
|
||||||
<input type="file" name="pdf" accept="application/pdf" required />
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Extra texts (optional, e.g. Paperless OCR output)
|
|
||||||
<textarea
|
|
||||||
name="texts"
|
|
||||||
rows="3"
|
|
||||||
placeholder="Plain text passed as context.texts[0]"
|
|
||||||
>{{ form_values.get("texts", "") }}</textarea>
|
|
||||||
<small class="field-hint">Whatever you type is submitted as a single entry in <code>context.texts</code>.</small>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<fieldset>
|
|
||||||
<legend>Use case</legend>
|
|
||||||
<label>
|
|
||||||
<input
|
|
||||||
type="radio"
|
|
||||||
name="use_case_mode"
|
|
||||||
value="registered"
|
|
||||||
{% if form_values.get("use_case_mode", "registered") == "registered" %}checked{% endif %}
|
|
||||||
onchange="document.getElementById('custom-fields').hidden = true"
|
|
||||||
/>
|
|
||||||
Registered
|
|
||||||
</label>
|
|
||||||
<label>
|
|
||||||
<input
|
|
||||||
type="radio"
|
|
||||||
name="use_case_mode"
|
|
||||||
value="custom"
|
|
||||||
{% if form_values.get("use_case_mode") == "custom" %}checked{% endif %}
|
|
||||||
onchange="document.getElementById('custom-fields').hidden = false"
|
|
||||||
/>
|
|
||||||
Custom (inline)
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Use case name
|
|
||||||
<input
|
|
||||||
type="text"
|
|
||||||
name="use_case_name"
|
|
||||||
list="registered-use-cases"
|
|
||||||
value="{{ form_values.get('use_case_name', 'bank_statement_header') }}"
|
|
||||||
required
|
|
||||||
/>
|
|
||||||
<datalist id="registered-use-cases">
|
|
||||||
{% for name in registered_use_cases %}
|
|
||||||
<option value="{{ name }}"></option>
|
|
||||||
{% endfor %}
|
|
||||||
</datalist>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<div id="custom-fields" {% if form_values.get("use_case_mode") != "custom" %}hidden{% endif %}>
|
|
||||||
<label>
|
|
||||||
System prompt
|
|
||||||
<textarea name="system_prompt" rows="3">{{ form_values.get("system_prompt", "") }}</textarea>
|
|
||||||
</label>
|
|
||||||
<label>
|
|
||||||
Default model (optional)
|
|
||||||
<input
|
|
||||||
type="text"
|
|
||||||
name="default_model"
|
|
||||||
value="{{ form_values.get('default_model', '') }}"
|
|
||||||
placeholder="qwen3:14b"
|
|
||||||
/>
|
|
||||||
</label>
|
|
||||||
<label>
|
|
||||||
Fields (JSON list of {name, type, required?, choices?, description?})
|
|
||||||
<textarea name="fields_json" rows="6" placeholder='[{"name": "vendor", "type": "str", "required": true}]'>{{ form_values.get("fields_json", "") }}</textarea>
|
|
||||||
<small class="field-hint">Types: str, int, float, decimal, date, datetime, bool. <code>choices</code> works on <code>str</code> only.</small>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
</fieldset>
|
|
||||||
|
|
||||||
<details>
|
|
||||||
<summary>Advanced options</summary>
|
|
||||||
<label>
|
|
||||||
Client id
|
|
||||||
<input type="text" name="ix_client_id" value="{{ form_values.get('ix_client_id', 'ui') }}" />
|
|
||||||
</label>
|
|
||||||
<label>
|
|
||||||
Request id (blank → random)
|
|
||||||
<input type="text" name="request_id" value="{{ form_values.get('request_id', '') }}" />
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<fieldset>
|
|
||||||
<legend>OCR</legend>
|
|
||||||
<label><input type="checkbox" name="use_ocr" {% if form_values.get("use_ocr", "on") %}checked{% endif %} /> use_ocr</label>
|
|
||||||
<label><input type="checkbox" name="ocr_only" {% if form_values.get("ocr_only") %}checked{% endif %} /> ocr_only</label>
|
|
||||||
<label><input type="checkbox" name="include_ocr_text" {% if form_values.get("include_ocr_text") %}checked{% endif %} /> include_ocr_text</label>
|
|
||||||
<label><input type="checkbox" name="include_geometries" {% if form_values.get("include_geometries") %}checked{% endif %} /> include_geometries</label>
|
|
||||||
</fieldset>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
GenAI model override (optional)
|
|
||||||
<input type="text" name="gen_ai_model_name" value="{{ form_values.get('gen_ai_model_name', '') }}" />
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<fieldset>
|
|
||||||
<legend>Provenance</legend>
|
|
||||||
<label><input type="checkbox" name="include_provenance" {% if form_values.get("include_provenance", "on") %}checked{% endif %} /> include_provenance</label>
|
|
||||||
<label>
|
|
||||||
max_sources_per_field
|
|
||||||
<input type="number" name="max_sources_per_field" min="1" max="100" value="{{ form_values.get('max_sources_per_field', '10') }}" />
|
|
||||||
</label>
|
|
||||||
</fieldset>
|
|
||||||
</details>
|
|
||||||
|
|
||||||
<button type="submit">Submit</button>
|
|
||||||
</form>
|
|
||||||
</article>
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
{% if job %}
|
|
||||||
<article id="job-panel">
|
|
||||||
<header>
|
|
||||||
<strong>Job</strong> <code>{{ job.job_id }}</code>
|
|
||||||
<br /><small>ix_id: <code>{{ job.ix_id }}</code></small>
|
|
||||||
{% if file_names %}
|
|
||||||
<br /><small>
|
|
||||||
File{% if file_names|length > 1 %}s{% endif %}:
|
|
||||||
{% for name in file_names %}
|
|
||||||
<code>{{ name }}</code>{% if not loop.last %}, {% endif %}
|
|
||||||
{% endfor %}
|
|
||||||
</small>
|
|
||||||
{% endif %}
|
|
||||||
</header>
|
|
||||||
<div
|
|
||||||
id="job-status"
|
|
||||||
hx-get="/ui/jobs/{{ job.job_id }}/fragment"
|
|
||||||
hx-trigger="load"
|
|
||||||
hx-swap="innerHTML"
|
|
||||||
>
|
|
||||||
Loading…
|
|
||||||
</div>
|
|
||||||
</article>
|
|
||||||
{% endif %}
|
|
||||||
</main>
|
|
||||||
|
|
||||||
<script>
|
|
||||||
document.body.addEventListener("htmx:afterSettle", () => {
|
|
||||||
if (window.hljs) {
|
|
||||||
document.querySelectorAll("pre code").forEach((el) => {
|
|
||||||
try { hljs.highlightElement(el); } catch (_) { /* noop */ }
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
</script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -1,77 +0,0 @@
|
||||||
{#- HTMX fragment rendered into #job-status on the results panel.
|
|
||||||
Pending/running → keep polling every 2s; terminal → render JSON. -#}
|
|
||||||
{% set terminal = job.status in ("done", "error") %}
|
|
||||||
<div
|
|
||||||
id="job-fragment"
|
|
||||||
{% if not terminal %}
|
|
||||||
hx-get="/ui/jobs/{{ job.job_id }}/fragment"
|
|
||||||
hx-trigger="every 2s"
|
|
||||||
hx-swap="outerHTML"
|
|
||||||
{% endif %}
|
|
||||||
>
|
|
||||||
<article class="status-panel">
|
|
||||||
<header>
|
|
||||||
<strong>Job status</strong>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
Status:
|
|
||||||
<strong>{{ job.status }}</strong>
|
|
||||||
{% if not terminal %}
|
|
||||||
<span class="live-dot" aria-hidden="true">●</span>
|
|
||||||
{% endif %}
|
|
||||||
</p>
|
|
||||||
|
|
||||||
{% if file_names %}
|
|
||||||
<p class="job-files">
|
|
||||||
File{% if file_names|length > 1 %}s{% endif %}:
|
|
||||||
{% for name in file_names %}
|
|
||||||
<code>{{ name }}</code>{% if not loop.last %}, {% endif %}
|
|
||||||
{% endfor %}
|
|
||||||
</p>
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
{% if job.status == "pending" %}
|
|
||||||
<p>
|
|
||||||
{% if ahead == 0 %}
|
|
||||||
About to start — the worker just freed up.
|
|
||||||
{% else %}
|
|
||||||
Queue position: {{ ahead }} ahead — {{ total_active }} job{% if total_active != 1 %}s{% endif %} total in flight (single worker).
|
|
||||||
{% endif %}
|
|
||||||
</p>
|
|
||||||
<progress></progress>
|
|
||||||
{% elif job.status == "running" %}
|
|
||||||
{% if elapsed_text %}
|
|
||||||
<p>Running for {{ elapsed_text }}.</p>
|
|
||||||
{% endif %}
|
|
||||||
<progress></progress>
|
|
||||||
{% elif terminal %}
|
|
||||||
{% if elapsed_text %}
|
|
||||||
<p>Finished in {{ elapsed_text }}.</p>
|
|
||||||
{% endif %}
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
{% if cpu_mode and not terminal %}
|
|
||||||
<details class="cpu-notice">
|
|
||||||
<summary>Surya is running on CPU (~1–2 min/page)</summary>
|
|
||||||
<p>
|
|
||||||
A host NVIDIA driver upgrade would unlock GPU extraction; tracked in
|
|
||||||
<code>docs/deployment.md</code>.
|
|
||||||
</p>
|
|
||||||
</details>
|
|
||||||
{% endif %}
|
|
||||||
</article>
|
|
||||||
|
|
||||||
<article class="result-panel">
|
|
||||||
<header>
|
|
||||||
<strong>Result</strong>
|
|
||||||
</header>
|
|
||||||
{% if terminal and response_json %}
|
|
||||||
<pre><code class="language-json">{{ response_json }}</code></pre>
|
|
||||||
{% elif terminal %}
|
|
||||||
<p><em>No response body.</em></p>
|
|
||||||
{% else %}
|
|
||||||
<p><em>Waiting for the pipeline to finish…</em></p>
|
|
||||||
{% endif %}
|
|
||||||
</article>
|
|
||||||
</div>
|
|
||||||
|
|
@ -1,164 +0,0 @@
|
||||||
<!doctype html>
|
|
||||||
<html lang="en" data-theme="light">
|
|
||||||
<head>
|
|
||||||
<meta charset="utf-8" />
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
||||||
<title>InfoXtractor — Recent jobs</title>
|
|
||||||
<link
|
|
||||||
rel="stylesheet"
|
|
||||||
href="https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.min.css"
|
|
||||||
/>
|
|
||||||
<style>
|
|
||||||
main { padding-top: 1.5rem; padding-bottom: 4rem; }
|
|
||||||
nav.ix-header {
|
|
||||||
display: flex; gap: 1rem; align-items: baseline;
|
|
||||||
padding: 0.6rem 0; border-bottom: 1px solid var(--pico-muted-border-color, #ddd);
|
|
||||||
margin-bottom: 1rem; flex-wrap: wrap;
|
|
||||||
}
|
|
||||||
nav.ix-header .brand { font-weight: 700; margin-right: auto; }
|
|
||||||
.breadcrumb {
|
|
||||||
font-size: 0.9rem; color: var(--pico-muted-color);
|
|
||||||
margin-bottom: 0.75rem;
|
|
||||||
}
|
|
||||||
.breadcrumb a { text-decoration: none; }
|
|
||||||
.filter-bar {
|
|
||||||
display: flex; flex-wrap: wrap; gap: 1rem; align-items: flex-end;
|
|
||||||
margin-bottom: 1rem;
|
|
||||||
}
|
|
||||||
.filter-bar fieldset { margin: 0; padding: 0; border: none; }
|
|
||||||
.filter-bar label.inline { display: inline-flex; gap: 0.3rem; align-items: center; margin-right: 0.8rem; font-weight: normal; }
|
|
||||||
.counter { color: var(--pico-muted-color); margin-bottom: 0.5rem; }
|
|
||||||
table.jobs-table { width: 100%; font-size: 0.92rem; }
|
|
||||||
table.jobs-table th { white-space: nowrap; }
|
|
||||||
table.jobs-table td { vertical-align: middle; }
|
|
||||||
td.col-created small { color: var(--pico-muted-color); display: block; }
|
|
||||||
.status-badge {
|
|
||||||
display: inline-block; padding: 0.1rem 0.55rem;
|
|
||||||
border-radius: 0.8rem; font-size: 0.78rem; font-weight: 600;
|
|
||||||
text-transform: uppercase; letter-spacing: 0.04em;
|
|
||||||
}
|
|
||||||
.status-done { background: #d1f4dc; color: #1a6d35; }
|
|
||||||
.status-error { background: #fadadd; color: #8a1d2b; }
|
|
||||||
.status-pending, .status-running { background: #fff1c2; color: #805600; }
|
|
||||||
.pagination {
|
|
||||||
display: flex; gap: 0.75rem; margin-top: 1rem;
|
|
||||||
align-items: center; flex-wrap: wrap;
|
|
||||||
}
|
|
||||||
.empty-note { color: var(--pico-muted-color); font-style: italic; }
|
|
||||||
td.col-filename code { font-size: 0.9em; word-break: break-all; }
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<main class="container">
|
|
||||||
<nav class="ix-header" aria-label="InfoXtractor navigation">
|
|
||||||
<span class="brand">InfoXtractor</span>
|
|
||||||
<a href="/ui">Upload a new extraction</a>
|
|
||||||
<a href="/ui/jobs">Recent jobs</a>
|
|
||||||
</nav>
|
|
||||||
|
|
||||||
<p class="breadcrumb">
|
|
||||||
<a href="/ui">Home</a> › Jobs
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<hgroup>
|
|
||||||
<h1>Recent jobs</h1>
|
|
||||||
<p>All submitted extractions, newest first.</p>
|
|
||||||
</hgroup>
|
|
||||||
|
|
||||||
<form class="filter-bar" method="get" action="/ui/jobs">
|
|
||||||
<fieldset>
|
|
||||||
<legend><small>Status</small></legend>
|
|
||||||
{% for s in valid_statuses %}
|
|
||||||
<label class="inline">
|
|
||||||
<input
|
|
||||||
type="checkbox"
|
|
||||||
name="status"
|
|
||||||
value="{{ s }}"
|
|
||||||
{% if s in status_filter %}checked{% endif %}
|
|
||||||
/>
|
|
||||||
{{ s }}
|
|
||||||
</label>
|
|
||||||
{% endfor %}
|
|
||||||
</fieldset>
|
|
||||||
<label>
|
|
||||||
Client id
|
|
||||||
<input
|
|
||||||
type="text"
|
|
||||||
name="client_id"
|
|
||||||
value="{{ client_filter }}"
|
|
||||||
placeholder="e.g. ui, mammon"
|
|
||||||
/>
|
|
||||||
</label>
|
|
||||||
<label>
|
|
||||||
Page size
|
|
||||||
<input
|
|
||||||
type="number"
|
|
||||||
name="limit"
|
|
||||||
min="1"
|
|
||||||
max="200"
|
|
||||||
value="{{ limit }}"
|
|
||||||
/>
|
|
||||||
</label>
|
|
||||||
<button type="submit">Apply</button>
|
|
||||||
</form>
|
|
||||||
|
|
||||||
<p class="counter">
|
|
||||||
Showing {{ shown }} of {{ total }} job{% if total != 1 %}s{% endif %}.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
{% if rows %}
|
|
||||||
<figure>
|
|
||||||
<table class="jobs-table" role="grid">
|
|
||||||
<thead>
|
|
||||||
<tr>
|
|
||||||
<th>Status</th>
|
|
||||||
<th>Filename</th>
|
|
||||||
<th>Use case</th>
|
|
||||||
<th>Client</th>
|
|
||||||
<th>Submitted</th>
|
|
||||||
<th>Elapsed</th>
|
|
||||||
<th></th>
|
|
||||||
</tr>
|
|
||||||
</thead>
|
|
||||||
<tbody>
|
|
||||||
{% for row in rows %}
|
|
||||||
<tr>
|
|
||||||
<td>
|
|
||||||
<span class="status-badge status-{{ row.status }}">{{ row.status }}</span>
|
|
||||||
</td>
|
|
||||||
<td class="col-filename"><code>{{ row.display_name }}</code></td>
|
|
||||||
<td>{{ row.use_case }}</td>
|
|
||||||
<td>{{ row.client_id }}</td>
|
|
||||||
<td class="col-created">
|
|
||||||
{{ row.created_at }}
|
|
||||||
<small>{{ row.created_delta }}</small>
|
|
||||||
</td>
|
|
||||||
<td>{{ row.elapsed }}</td>
|
|
||||||
<td>
|
|
||||||
<a href="/ui/jobs/{{ row.job_id }}">open ›</a>
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
{% endfor %}
|
|
||||||
</tbody>
|
|
||||||
</table>
|
|
||||||
</figure>
|
|
||||||
{% else %}
|
|
||||||
<p class="empty-note">No jobs match the current filters.</p>
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
<div class="pagination">
|
|
||||||
{% if prev_link %}
|
|
||||||
<a href="{{ prev_link }}" role="button" class="secondary outline">« Prev</a>
|
|
||||||
{% else %}
|
|
||||||
<span aria-disabled="true" class="secondary outline" role="button" style="opacity: 0.4;">« Prev</span>
|
|
||||||
{% endif %}
|
|
||||||
<span class="counter">Offset {{ offset }}</span>
|
|
||||||
{% if next_link %}
|
|
||||||
<a href="{{ next_link }}" role="button" class="secondary outline">Next »</a>
|
|
||||||
{% else %}
|
|
||||||
<span aria-disabled="true" class="secondary outline" role="button" style="opacity: 0.4;">Next »</span>
|
|
||||||
{% endif %}
|
|
||||||
</div>
|
|
||||||
</main>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -1,132 +0,0 @@
|
||||||
"""Dynamic Pydantic class builder for caller-supplied use cases.
|
|
||||||
|
|
||||||
Input: an :class:`ix.contracts.request.InlineUseCase` carried on the
|
|
||||||
:class:`~ix.contracts.request.RequestIX`.
|
|
||||||
|
|
||||||
Output: a fresh ``(RequestClass, ResponseClass)`` pair with the same shape
|
|
||||||
as a registered use case. The :class:`~ix.pipeline.setup_step.SetupStep`
|
|
||||||
calls this when ``request_ix.use_case_inline`` is set, bypassing the
|
|
||||||
registry lookup entirely.
|
|
||||||
|
|
||||||
The builder returns brand-new classes on every call — safe to call per
|
|
||||||
request, so two concurrent jobs can't step on each other's schemas even if
|
|
||||||
they happen to share a ``use_case_name``. Validation errors map to
|
|
||||||
``IX_001_001`` (same code the registry-miss path uses); the error is
|
|
||||||
recoverable from the caller's perspective (fix the JSON and retry), not an
|
|
||||||
infra problem.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import keyword
|
|
||||||
import re
|
|
||||||
from datetime import date, datetime
|
|
||||||
from decimal import Decimal
|
|
||||||
from typing import Any, Literal, cast
|
|
||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, Field, create_model
|
|
||||||
|
|
||||||
from ix.contracts.request import InlineUseCase, UseCaseFieldDef
|
|
||||||
from ix.errors import IXErrorCode, IXException
|
|
||||||
|
|
||||||
# Map the ``UseCaseFieldDef.type`` literal to concrete Python types.
|
|
||||||
_TYPE_MAP: dict[str, type] = {
|
|
||||||
"str": str,
|
|
||||||
"int": int,
|
|
||||||
"float": float,
|
|
||||||
"decimal": Decimal,
|
|
||||||
"date": date,
|
|
||||||
"datetime": datetime,
|
|
||||||
"bool": bool,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _fail(detail: str) -> IXException:
|
|
||||||
return IXException(IXErrorCode.IX_001_001, detail=detail)
|
|
||||||
|
|
||||||
|
|
||||||
def _valid_field_name(name: str) -> bool:
|
|
||||||
"""Require a valid Python identifier that isn't a reserved keyword."""
|
|
||||||
|
|
||||||
return name.isidentifier() and not keyword.iskeyword(name)
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_field_type(field: UseCaseFieldDef) -> Any:
|
|
||||||
"""Return the annotation for a single field, with ``choices`` honoured."""
|
|
||||||
|
|
||||||
base = _TYPE_MAP[field.type]
|
|
||||||
if field.choices:
|
|
||||||
if field.type != "str":
|
|
||||||
raise _fail(
|
|
||||||
f"field {field.name!r}: 'choices' is only allowed for "
|
|
||||||
f"type='str' (got {field.type!r})"
|
|
||||||
)
|
|
||||||
return Literal[tuple(field.choices)] # type: ignore[valid-type]
|
|
||||||
return base
|
|
||||||
|
|
||||||
|
|
||||||
def _sanitise_class_name(raw: str) -> str:
|
|
||||||
"""``re.sub(r"\\W", "_", name)`` + ``Inline_`` prefix.
|
|
||||||
|
|
||||||
Keeps the generated class name debuggable (shows up in repr / tracebacks)
|
|
||||||
while ensuring it's always a valid Python identifier.
|
|
||||||
"""
|
|
||||||
|
|
||||||
return "Inline_" + re.sub(r"\W", "_", raw)
|
|
||||||
|
|
||||||
|
|
||||||
def build_use_case_classes(
|
|
||||||
inline: InlineUseCase,
|
|
||||||
) -> tuple[type[BaseModel], type[BaseModel]]:
|
|
||||||
"""Build a fresh ``(RequestClass, ResponseClass)`` from ``inline``.
|
|
||||||
|
|
||||||
* Every call returns new classes. The caller may cache if desired; the
|
|
||||||
pipeline intentionally does not.
|
|
||||||
* Raises :class:`~ix.errors.IXException` with code
|
|
||||||
:attr:`~ix.errors.IXErrorCode.IX_001_001` on any structural problem
|
|
||||||
(empty fields, bad name, dup name, bad ``choices``).
|
|
||||||
"""
|
|
||||||
|
|
||||||
if not inline.fields:
|
|
||||||
raise _fail("inline use case must define at least one field")
|
|
||||||
|
|
||||||
seen: set[str] = set()
|
|
||||||
for fd in inline.fields:
|
|
||||||
if not _valid_field_name(fd.name):
|
|
||||||
raise _fail(f"field name {fd.name!r} is not a valid Python identifier")
|
|
||||||
if fd.name in seen:
|
|
||||||
raise _fail(f"duplicate field name {fd.name!r}")
|
|
||||||
seen.add(fd.name)
|
|
||||||
|
|
||||||
response_fields: dict[str, Any] = {}
|
|
||||||
for fd in inline.fields:
|
|
||||||
annotation = _resolve_field_type(fd)
|
|
||||||
field_info = Field(
|
|
||||||
...,
|
|
||||||
description=fd.description,
|
|
||||||
) if fd.required else Field(
|
|
||||||
default=None,
|
|
||||||
description=fd.description,
|
|
||||||
)
|
|
||||||
if not fd.required:
|
|
||||||
annotation = annotation | None
|
|
||||||
response_fields[fd.name] = (annotation, field_info)
|
|
||||||
|
|
||||||
response_cls = create_model( # type: ignore[call-overload]
|
|
||||||
_sanitise_class_name(inline.use_case_name),
|
|
||||||
__config__=ConfigDict(extra="forbid"),
|
|
||||||
**response_fields,
|
|
||||||
)
|
|
||||||
|
|
||||||
request_cls = create_model( # type: ignore[call-overload]
|
|
||||||
"Inline_Request_" + re.sub(r"\W", "_", inline.use_case_name),
|
|
||||||
__config__=ConfigDict(extra="forbid"),
|
|
||||||
use_case_name=(str, inline.use_case_name),
|
|
||||||
system_prompt=(str, inline.system_prompt),
|
|
||||||
default_model=(str | None, inline.default_model),
|
|
||||||
)
|
|
||||||
|
|
||||||
return cast(type[BaseModel], request_cls), cast(type[BaseModel], response_cls)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["build_use_case_classes"]
|
|
||||||
|
|
@ -341,117 +341,6 @@ async def test_sweep_orphans_leaves_fresh_running_alone(
|
||||||
assert after.status == "running"
|
assert after.status == "running"
|
||||||
|
|
||||||
|
|
||||||
async def test_queue_position_pending_only(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
"""Three pending rows in insertion order → positions 0, 1, 2; total 3.
|
|
||||||
|
|
||||||
Each row is committed in its own transaction so the DB stamps a
|
|
||||||
distinct ``created_at`` per row (``now()`` is transaction-stable).
|
|
||||||
"""
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
a = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", "qp-a"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
async with session_factory() as session:
|
|
||||||
b = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", "qp-b"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
async with session_factory() as session:
|
|
||||||
c = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", "qp-c"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
pa = await jobs_repo.queue_position(session, a.job_id)
|
|
||||||
pb = await jobs_repo.queue_position(session, b.job_id)
|
|
||||||
pc = await jobs_repo.queue_position(session, c.job_id)
|
|
||||||
|
|
||||||
# All three active; total == 3.
|
|
||||||
assert pa == (0, 3)
|
|
||||||
assert pb == (1, 3)
|
|
||||||
assert pc == (2, 3)
|
|
||||||
|
|
||||||
|
|
||||||
async def test_queue_position_running_plus_pending(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
"""One running + two pending → running:(0,3), next:(1,3), last:(2,3)."""
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
first = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", "qp-r-1"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
async with session_factory() as session:
|
|
||||||
second = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", "qp-r-2"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
async with session_factory() as session:
|
|
||||||
third = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", "qp-r-3"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
# Claim the first → it becomes running.
|
|
||||||
async with session_factory() as session:
|
|
||||||
claimed = await jobs_repo.claim_next_pending(session)
|
|
||||||
await session.commit()
|
|
||||||
assert claimed is not None
|
|
||||||
assert claimed.job_id == first.job_id
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
p_running = await jobs_repo.queue_position(session, first.job_id)
|
|
||||||
p_second = await jobs_repo.queue_position(session, second.job_id)
|
|
||||||
p_third = await jobs_repo.queue_position(session, third.job_id)
|
|
||||||
|
|
||||||
# Running row reports 0 ahead (itself is the head).
|
|
||||||
assert p_running == (0, 3)
|
|
||||||
# Second pending: running is ahead (1) + zero older pendings.
|
|
||||||
assert p_second == (1, 3)
|
|
||||||
# Third pending: running ahead + one older pending.
|
|
||||||
assert p_third == (2, 3)
|
|
||||||
|
|
||||||
|
|
||||||
async def test_queue_position_terminal_returns_zero_zero(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
"""Finished jobs have no queue position — always (0, 0)."""
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
inserted = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", "qp-term"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
response = ResponseIX(
|
|
||||||
use_case="bank_statement_header",
|
|
||||||
ix_client_id="c",
|
|
||||||
request_id="qp-term",
|
|
||||||
)
|
|
||||||
async with session_factory() as session:
|
|
||||||
await jobs_repo.mark_done(session, inserted.job_id, response)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
pos = await jobs_repo.queue_position(session, inserted.job_id)
|
|
||||||
|
|
||||||
assert pos == (0, 0)
|
|
||||||
|
|
||||||
|
|
||||||
async def test_queue_position_unknown_id_returns_zero_zero(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
async with session_factory() as session:
|
|
||||||
pos = await jobs_repo.queue_position(session, uuid4())
|
|
||||||
assert pos == (0, 0)
|
|
||||||
|
|
||||||
|
|
||||||
async def test_concurrent_claim_never_double_dispatches(
|
async def test_concurrent_claim_never_double_dispatches(
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
session_factory: async_sessionmaker[AsyncSession],
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|
@ -476,204 +365,3 @@ async def test_concurrent_claim_never_double_dispatches(
|
||||||
non_null = [r for r in results if r is not None]
|
non_null = [r for r in results if r is not None]
|
||||||
# Every inserted id appears at most once.
|
# Every inserted id appears at most once.
|
||||||
assert sorted(non_null) == sorted(ids)
|
assert sorted(non_null) == sorted(ids)
|
||||||
|
|
||||||
|
|
||||||
# ---------- list_recent ---------------------------------------------------
|
|
||||||
#
|
|
||||||
# The UI's ``/ui/jobs`` page needs a paginated, filterable view of recent
|
|
||||||
# jobs. We keep the contract intentionally small: list_recent returns
|
|
||||||
# ``(jobs, total)`` — ``total`` is the count after filters but before
|
|
||||||
# limit/offset — so the template can render "Showing N of M".
|
|
||||||
|
|
||||||
|
|
||||||
async def test_list_recent_empty_db(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
async with session_factory() as session:
|
|
||||||
jobs, total = await jobs_repo.list_recent(session, limit=50, offset=0)
|
|
||||||
assert jobs == []
|
|
||||||
assert total == 0
|
|
||||||
|
|
||||||
|
|
||||||
async def test_list_recent_orders_newest_first(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
ids: list[UUID] = []
|
|
||||||
for i in range(3):
|
|
||||||
async with session_factory() as session:
|
|
||||||
job = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", f"lr-{i}"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
ids.append(job.job_id)
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
jobs, total = await jobs_repo.list_recent(session, limit=50, offset=0)
|
|
||||||
|
|
||||||
assert total == 3
|
|
||||||
# Newest first → reverse of insertion order.
|
|
||||||
assert [j.job_id for j in jobs] == list(reversed(ids))
|
|
||||||
|
|
||||||
|
|
||||||
async def test_list_recent_status_single_filter(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
# Two pending, one done.
|
|
||||||
async with session_factory() as session:
|
|
||||||
for i in range(3):
|
|
||||||
await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", f"sf-{i}"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
claimed = await jobs_repo.claim_next_pending(session)
|
|
||||||
assert claimed is not None
|
|
||||||
await jobs_repo.mark_done(
|
|
||||||
session,
|
|
||||||
claimed.job_id,
|
|
||||||
ResponseIX(
|
|
||||||
use_case="bank_statement_header",
|
|
||||||
ix_client_id="c",
|
|
||||||
request_id=claimed.request_id,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
done_jobs, done_total = await jobs_repo.list_recent(
|
|
||||||
session, limit=50, offset=0, status="done"
|
|
||||||
)
|
|
||||||
assert done_total == 1
|
|
||||||
assert len(done_jobs) == 1
|
|
||||||
assert done_jobs[0].status == "done"
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
pending_jobs, pending_total = await jobs_repo.list_recent(
|
|
||||||
session, limit=50, offset=0, status="pending"
|
|
||||||
)
|
|
||||||
assert pending_total == 2
|
|
||||||
assert all(j.status == "pending" for j in pending_jobs)
|
|
||||||
|
|
||||||
|
|
||||||
async def test_list_recent_status_iterable_filter(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
# Two pending, one done, one errored.
|
|
||||||
async with session_factory() as session:
|
|
||||||
for i in range(4):
|
|
||||||
await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", f"if-{i}"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
a = await jobs_repo.claim_next_pending(session)
|
|
||||||
assert a is not None
|
|
||||||
await jobs_repo.mark_done(
|
|
||||||
session,
|
|
||||||
a.job_id,
|
|
||||||
ResponseIX(
|
|
||||||
use_case="bank_statement_header",
|
|
||||||
ix_client_id="c",
|
|
||||||
request_id=a.request_id,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
async with session_factory() as session:
|
|
||||||
b = await jobs_repo.claim_next_pending(session)
|
|
||||||
assert b is not None
|
|
||||||
await jobs_repo.mark_error(session, b.job_id, ResponseIX(error="boom"))
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
jobs, total = await jobs_repo.list_recent(
|
|
||||||
session, limit=50, offset=0, status=["done", "error"]
|
|
||||||
)
|
|
||||||
assert total == 2
|
|
||||||
assert {j.status for j in jobs} == {"done", "error"}
|
|
||||||
|
|
||||||
|
|
||||||
async def test_list_recent_client_id_filter(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
async with session_factory() as session:
|
|
||||||
await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("alpha", "a-1"), callback_url=None
|
|
||||||
)
|
|
||||||
await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("beta", "b-1"), callback_url=None
|
|
||||||
)
|
|
||||||
await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("alpha", "a-2"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
jobs, total = await jobs_repo.list_recent(
|
|
||||||
session, limit=50, offset=0, client_id="alpha"
|
|
||||||
)
|
|
||||||
assert total == 2
|
|
||||||
assert all(j.client_id == "alpha" for j in jobs)
|
|
||||||
|
|
||||||
|
|
||||||
async def test_list_recent_pagination(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
ids: list[UUID] = []
|
|
||||||
for i in range(7):
|
|
||||||
async with session_factory() as session:
|
|
||||||
job = await jobs_repo.insert_pending(
|
|
||||||
session, _make_request("c", f"pg-{i}"), callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
ids.append(job.job_id)
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
page1, total1 = await jobs_repo.list_recent(
|
|
||||||
session, limit=3, offset=0
|
|
||||||
)
|
|
||||||
assert total1 == 7
|
|
||||||
assert len(page1) == 3
|
|
||||||
# Newest three are the last three inserted.
|
|
||||||
assert [j.job_id for j in page1] == list(reversed(ids[-3:]))
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
page2, total2 = await jobs_repo.list_recent(
|
|
||||||
session, limit=3, offset=3
|
|
||||||
)
|
|
||||||
assert total2 == 7
|
|
||||||
assert len(page2) == 3
|
|
||||||
expected = list(reversed(ids))[3:6]
|
|
||||||
assert [j.job_id for j in page2] == expected
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
page3, total3 = await jobs_repo.list_recent(
|
|
||||||
session, limit=3, offset=6
|
|
||||||
)
|
|
||||||
assert total3 == 7
|
|
||||||
assert len(page3) == 1
|
|
||||||
assert page3[0].job_id == ids[0]
|
|
||||||
|
|
||||||
|
|
||||||
async def test_list_recent_caps_limit(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
"""limit is capped at 200 — asking for 9999 gets clamped."""
|
|
||||||
|
|
||||||
async with session_factory() as session:
|
|
||||||
jobs, total = await jobs_repo.list_recent(
|
|
||||||
session, limit=9999, offset=0
|
|
||||||
)
|
|
||||||
assert total == 0
|
|
||||||
assert jobs == []
|
|
||||||
|
|
||||||
|
|
||||||
async def test_list_recent_rejects_negative_offset(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession],
|
|
||||||
) -> None:
|
|
||||||
async with session_factory() as session:
|
|
||||||
import pytest as _pytest
|
|
||||||
|
|
||||||
with _pytest.raises(ValueError):
|
|
||||||
await jobs_repo.list_recent(session, limit=50, offset=-1)
|
|
||||||
|
|
|
||||||
|
|
@ -1,792 +0,0 @@
|
||||||
"""Integration tests for the `/ui` router (spec §PR 2).
|
|
||||||
|
|
||||||
Covers the full round-trip through `POST /ui/jobs` — the handler parses
|
|
||||||
multipart form data into a `RequestIX` and hands it to
|
|
||||||
`ix.store.jobs_repo.insert_pending`, the same entry point the REST adapter
|
|
||||||
uses. Tests assert the job row exists with the right client/request ids and
|
|
||||||
that custom-use-case forms produce a `use_case_inline` block in the stored
|
|
||||||
request JSON.
|
|
||||||
|
|
||||||
The DB-touching tests depend on the shared integration conftest which
|
|
||||||
spins up migrations against the configured Postgres; the pure-template
|
|
||||||
tests (`GET /ui` and the fragment renderer) still need a factory but
|
|
||||||
won't actually query — they're cheap.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
from collections.abc import Iterator
|
|
||||||
from pathlib import Path
|
|
||||||
from uuid import UUID, uuid4
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
from sqlalchemy import select
|
|
||||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
|
||||||
|
|
||||||
from ix.adapters.rest.routes import Probes, get_probes, get_session_factory_dep
|
|
||||||
from ix.app import create_app
|
|
||||||
from ix.store.models import IxJob
|
|
||||||
|
|
||||||
FIXTURE_DIR = Path(__file__).resolve().parents[1] / "fixtures"
|
|
||||||
FIXTURE_PDF = FIXTURE_DIR / "synthetic_giro.pdf"
|
|
||||||
|
|
||||||
|
|
||||||
def _factory_for_url(postgres_url: str): # type: ignore[no-untyped-def]
|
|
||||||
def _factory(): # type: ignore[no-untyped-def]
|
|
||||||
eng = create_async_engine(postgres_url, pool_pre_ping=True)
|
|
||||||
return async_sessionmaker(eng, expire_on_commit=False)
|
|
||||||
|
|
||||||
return _factory
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def app(postgres_url: str) -> Iterator[TestClient]:
|
|
||||||
app_obj = create_app(spawn_worker=False)
|
|
||||||
app_obj.dependency_overrides[get_session_factory_dep] = _factory_for_url(
|
|
||||||
postgres_url
|
|
||||||
)
|
|
||||||
app_obj.dependency_overrides[get_probes] = lambda: Probes(
|
|
||||||
ollama=lambda: "ok", ocr=lambda: "ok"
|
|
||||||
)
|
|
||||||
with TestClient(app_obj) as client:
|
|
||||||
yield client
|
|
||||||
|
|
||||||
|
|
||||||
class TestIndexPage:
|
|
||||||
def test_index_returns_html(self, app: TestClient) -> None:
|
|
||||||
resp = app.get("/ui")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert "text/html" in resp.headers["content-type"]
|
|
||||||
body = resp.text
|
|
||||||
# Dropdown prefilled with the registered use case.
|
|
||||||
assert "bank_statement_header" in body
|
|
||||||
# Marker for the submission form.
|
|
||||||
assert '<form' in body
|
|
||||||
|
|
||||||
def test_static_mount_is_reachable(self, app: TestClient) -> None:
|
|
||||||
# StaticFiles returns 404 for the keepfile; the mount itself must
|
|
||||||
# exist so asset URLs resolve. We probe the directory root instead.
|
|
||||||
resp = app.get("/ui/static/.gitkeep")
|
|
||||||
# .gitkeep exists in the repo — expect 200 (or at minimum not a 404
|
|
||||||
# due to missing mount). A 405/403 would also indicate the mount is
|
|
||||||
# wired; we assert the response is *not* a 404 from a missing route.
|
|
||||||
assert resp.status_code != 404
|
|
||||||
|
|
||||||
|
|
||||||
class TestSubmitJobRegistered:
|
|
||||||
def test_post_registered_use_case_creates_row(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
request_id = f"ui-reg-{uuid4().hex[:8]}"
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
resp = app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "registered",
|
|
||||||
"use_case_name": "bank_statement_header",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
"texts": "",
|
|
||||||
"use_ocr": "on",
|
|
||||||
"include_provenance": "on",
|
|
||||||
"max_sources_per_field": "10",
|
|
||||||
},
|
|
||||||
files={"pdf": ("sample.pdf", fh, "application/pdf")},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
assert resp.status_code in (200, 303), resp.text
|
|
||||||
|
|
||||||
# Assert the row exists in the DB.
|
|
||||||
job_row = _find_job(postgres_url, "ui-test", request_id)
|
|
||||||
assert job_row is not None
|
|
||||||
assert job_row.status == "pending"
|
|
||||||
assert job_row.request["use_case"] == "bank_statement_header"
|
|
||||||
# Context.files must reference a local file:// path.
|
|
||||||
files = job_row.request["context"]["files"]
|
|
||||||
assert len(files) == 1
|
|
||||||
entry = files[0]
|
|
||||||
url = entry if isinstance(entry, str) else entry["url"]
|
|
||||||
assert url.startswith("file://")
|
|
||||||
|
|
||||||
def test_htmx_submit_uses_hx_redirect_header(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
) -> None:
|
|
||||||
request_id = f"ui-htmx-{uuid4().hex[:8]}"
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
resp = app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "registered",
|
|
||||||
"use_case_name": "bank_statement_header",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
},
|
|
||||||
files={"pdf": ("sample.pdf", fh, "application/pdf")},
|
|
||||||
headers={"HX-Request": "true"},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert "HX-Redirect" in resp.headers
|
|
||||||
|
|
||||||
|
|
||||||
class TestSubmitJobCustom:
|
|
||||||
def test_post_custom_use_case_stores_inline(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
request_id = f"ui-cust-{uuid4().hex[:8]}"
|
|
||||||
fields_json = json.dumps(
|
|
||||||
[
|
|
||||||
{"name": "vendor", "type": "str", "required": True},
|
|
||||||
{"name": "total", "type": "decimal"},
|
|
||||||
]
|
|
||||||
)
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
resp = app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "custom",
|
|
||||||
"use_case_name": "invoice_adhoc",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
"system_prompt": "Extract vendor and total.",
|
|
||||||
"default_model": "qwen3:14b",
|
|
||||||
"fields_json": fields_json,
|
|
||||||
},
|
|
||||||
files={"pdf": ("sample.pdf", fh, "application/pdf")},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
assert resp.status_code in (200, 303), resp.text
|
|
||||||
job_row = _find_job(postgres_url, "ui-test", request_id)
|
|
||||||
assert job_row is not None
|
|
||||||
stored = job_row.request["use_case_inline"]
|
|
||||||
assert stored is not None
|
|
||||||
assert stored["use_case_name"] == "invoice_adhoc"
|
|
||||||
assert stored["system_prompt"] == "Extract vendor and total."
|
|
||||||
names = [f["name"] for f in stored["fields"]]
|
|
||||||
assert names == ["vendor", "total"]
|
|
||||||
|
|
||||||
def test_post_malformed_fields_json_rejected(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
request_id = f"ui-bad-{uuid4().hex[:8]}"
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
resp = app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "custom",
|
|
||||||
"use_case_name": "adhoc_bad",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
"system_prompt": "p",
|
|
||||||
"fields_json": "this is not json",
|
|
||||||
},
|
|
||||||
files={"pdf": ("sample.pdf", fh, "application/pdf")},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
# Either re-rendered form (422 / 200 with error) — what matters is
|
|
||||||
# that no row was inserted.
|
|
||||||
assert resp.status_code in (200, 400, 422)
|
|
||||||
job_row = _find_job(postgres_url, "ui-test", request_id)
|
|
||||||
assert job_row is None
|
|
||||||
# A helpful error should appear somewhere in the body.
|
|
||||||
assert (
|
|
||||||
"error" in resp.text.lower()
|
|
||||||
or "invalid" in resp.text.lower()
|
|
||||||
or "json" in resp.text.lower()
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class TestDisplayName:
|
|
||||||
def test_post_persists_display_name_in_file_ref(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
"""The client-provided upload filename lands in FileRef.display_name."""
|
|
||||||
|
|
||||||
request_id = f"ui-name-{uuid4().hex[:8]}"
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
resp = app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "registered",
|
|
||||||
"use_case_name": "bank_statement_header",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
},
|
|
||||||
files={
|
|
||||||
"pdf": ("my statement.pdf", fh, "application/pdf")
|
|
||||||
},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
assert resp.status_code in (200, 303), resp.text
|
|
||||||
|
|
||||||
job_row = _find_job(postgres_url, "ui-test", request_id)
|
|
||||||
assert job_row is not None
|
|
||||||
entry = job_row.request["context"]["files"][0]
|
|
||||||
assert isinstance(entry, dict)
|
|
||||||
assert entry["display_name"] == "my statement.pdf"
|
|
||||||
|
|
||||||
|
|
||||||
class TestFragment:
|
|
||||||
def test_fragment_pending_has_trigger(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
request_id = f"ui-frag-p-{uuid4().hex[:8]}"
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "registered",
|
|
||||||
"use_case_name": "bank_statement_header",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
},
|
|
||||||
files={"pdf": ("sample.pdf", fh, "application/pdf")},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
job_row = _find_job(postgres_url, "ui-test", request_id)
|
|
||||||
assert job_row is not None
|
|
||||||
|
|
||||||
resp = app.get(f"/ui/jobs/{job_row.job_id}/fragment")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.text
|
|
||||||
# Pending → auto-refresh every 2s.
|
|
||||||
assert "hx-trigger" in body
|
|
||||||
assert "2s" in body
|
|
||||||
assert "pending" in body.lower() or "running" in body.lower()
|
|
||||||
# New queue-awareness copy.
|
|
||||||
assert "Queue position" in body or "About to start" in body
|
|
||||||
|
|
||||||
def test_fragment_pending_shows_filename(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
request_id = f"ui-frag-pf-{uuid4().hex[:8]}"
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "registered",
|
|
||||||
"use_case_name": "bank_statement_header",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
},
|
|
||||||
files={
|
|
||||||
"pdf": (
|
|
||||||
"client-side-name.pdf",
|
|
||||||
fh,
|
|
||||||
"application/pdf",
|
|
||||||
)
|
|
||||||
},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
job_row = _find_job(postgres_url, "ui-test", request_id)
|
|
||||||
assert job_row is not None
|
|
||||||
resp = app.get(f"/ui/jobs/{job_row.job_id}/fragment")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert "client-side-name.pdf" in resp.text
|
|
||||||
|
|
||||||
def test_fragment_running_shows_elapsed(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
"""After flipping a row to running with a backdated started_at, the
|
|
||||||
fragment renders a ``Running for MM:SS`` line."""
|
|
||||||
|
|
||||||
request_id = f"ui-frag-r-{uuid4().hex[:8]}"
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "registered",
|
|
||||||
"use_case_name": "bank_statement_header",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
},
|
|
||||||
files={"pdf": ("sample.pdf", fh, "application/pdf")},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
job_row = _find_job(postgres_url, "ui-test", request_id)
|
|
||||||
assert job_row is not None
|
|
||||||
|
|
||||||
_force_running(postgres_url, job_row.job_id)
|
|
||||||
|
|
||||||
resp = app.get(f"/ui/jobs/{job_row.job_id}/fragment")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.text
|
|
||||||
assert "Running for" in body
|
|
||||||
# MM:SS; our backdate is ~10s so expect 00:1? or higher.
|
|
||||||
import re
|
|
||||||
|
|
||||||
assert re.search(r"\d{2}:\d{2}", body), body
|
|
||||||
|
|
||||||
def test_fragment_backward_compat_no_display_name(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
"""Older rows (stored before display_name existed) must still render."""
|
|
||||||
|
|
||||||
from ix.contracts.request import Context, FileRef, RequestIX
|
|
||||||
|
|
||||||
legacy_req = RequestIX(
|
|
||||||
use_case="bank_statement_header",
|
|
||||||
ix_client_id="ui-test",
|
|
||||||
request_id=f"ui-legacy-{uuid4().hex[:8]}",
|
|
||||||
context=Context(
|
|
||||||
files=[
|
|
||||||
FileRef(url="file:///tmp/ix/ui/legacy.pdf")
|
|
||||||
]
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
|
|
||||||
from ix.store import jobs_repo as _repo
|
|
||||||
|
|
||||||
async def _insert() -> UUID:
|
|
||||||
eng = create_async_engine(postgres_url)
|
|
||||||
sf = async_sessionmaker(eng, expire_on_commit=False)
|
|
||||||
try:
|
|
||||||
async with sf() as session:
|
|
||||||
job = await _repo.insert_pending(
|
|
||||||
session, legacy_req, callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
return job.job_id
|
|
||||||
finally:
|
|
||||||
await eng.dispose()
|
|
||||||
|
|
||||||
job_id = asyncio.run(_insert())
|
|
||||||
resp = app.get(f"/ui/jobs/{job_id}/fragment")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.text
|
|
||||||
# Must not crash; must include the fallback basename from the URL.
|
|
||||||
assert "legacy.pdf" in body
|
|
||||||
|
|
||||||
def test_fragment_done_shows_pretty_json(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
request_id = f"ui-frag-d-{uuid4().hex[:8]}"
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "registered",
|
|
||||||
"use_case_name": "bank_statement_header",
|
|
||||||
"ix_client_id": "ui-test",
|
|
||||||
"request_id": request_id,
|
|
||||||
},
|
|
||||||
files={
|
|
||||||
"pdf": (
|
|
||||||
"my-done-doc.pdf",
|
|
||||||
fh,
|
|
||||||
"application/pdf",
|
|
||||||
)
|
|
||||||
},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
job_row = _find_job(postgres_url, "ui-test", request_id)
|
|
||||||
assert job_row is not None
|
|
||||||
|
|
||||||
# Hand-tick the row to done with a fake response.
|
|
||||||
_force_done(
|
|
||||||
postgres_url,
|
|
||||||
job_row.job_id,
|
|
||||||
response_body={
|
|
||||||
"use_case": "bank_statement_header",
|
|
||||||
"ix_result": {"result": {"bank_name": "UBS AG", "currency": "CHF"}},
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
resp = app.get(f"/ui/jobs/{job_row.job_id}/fragment")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.text
|
|
||||||
# Terminal → no auto-refresh.
|
|
||||||
assert "every 2s" not in body and "every 2s" not in body
|
|
||||||
# JSON present.
|
|
||||||
assert "UBS AG" in body
|
|
||||||
assert "CHF" in body
|
|
||||||
# Filename surfaced on the done fragment.
|
|
||||||
assert "my-done-doc.pdf" in body
|
|
||||||
|
|
||||||
|
|
||||||
class TestJobsListPage:
|
|
||||||
"""Tests for the ``GET /ui/jobs`` listing page (feat/ui-jobs-list)."""
|
|
||||||
|
|
||||||
def _submit(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
client_id: str,
|
|
||||||
request_id: str,
|
|
||||||
filename: str = "sample.pdf",
|
|
||||||
) -> None:
|
|
||||||
with FIXTURE_PDF.open("rb") as fh:
|
|
||||||
app.post(
|
|
||||||
"/ui/jobs",
|
|
||||||
data={
|
|
||||||
"use_case_mode": "registered",
|
|
||||||
"use_case_name": "bank_statement_header",
|
|
||||||
"ix_client_id": client_id,
|
|
||||||
"request_id": request_id,
|
|
||||||
},
|
|
||||||
files={"pdf": (filename, fh, "application/pdf")},
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
|
|
||||||
def test_jobs_list_returns_html(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
for i in range(3):
|
|
||||||
self._submit(
|
|
||||||
app,
|
|
||||||
"ui-list",
|
|
||||||
f"lp-{uuid4().hex[:6]}-{i}",
|
|
||||||
filename=f"doc-{i}.pdf",
|
|
||||||
)
|
|
||||||
|
|
||||||
resp = app.get("/ui/jobs")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert "text/html" in resp.headers["content-type"]
|
|
||||||
body = resp.text
|
|
||||||
# Breadcrumb / header shows "Jobs".
|
|
||||||
assert "Jobs" in body
|
|
||||||
# display_name surfaces for each row.
|
|
||||||
for i in range(3):
|
|
||||||
assert f"doc-{i}.pdf" in body
|
|
||||||
# Showing N of M counter present.
|
|
||||||
assert "Showing" in body
|
|
||||||
assert "of" in body
|
|
||||||
|
|
||||||
def test_jobs_list_links_to_job_detail(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
rid = f"lp-link-{uuid4().hex[:6]}"
|
|
||||||
self._submit(app, "ui-list", rid)
|
|
||||||
row = _find_job(postgres_url, "ui-list", rid)
|
|
||||||
assert row is not None
|
|
||||||
resp = app.get("/ui/jobs")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert f"/ui/jobs/{row.job_id}" in resp.text
|
|
||||||
|
|
||||||
def test_jobs_list_status_filter_single(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
# Create two jobs, flip one to done.
|
|
||||||
rid_pending = f"lp-p-{uuid4().hex[:6]}"
|
|
||||||
rid_done = f"lp-d-{uuid4().hex[:6]}"
|
|
||||||
self._submit(app, "ui-filt", rid_pending, filename="pending-doc.pdf")
|
|
||||||
self._submit(app, "ui-filt", rid_done, filename="done-doc.pdf")
|
|
||||||
done_row = _find_job(postgres_url, "ui-filt", rid_done)
|
|
||||||
assert done_row is not None
|
|
||||||
_force_done(
|
|
||||||
postgres_url,
|
|
||||||
done_row.job_id,
|
|
||||||
response_body={"use_case": "bank_statement_header"},
|
|
||||||
)
|
|
||||||
|
|
||||||
# ?status=done → only done row shown.
|
|
||||||
resp = app.get("/ui/jobs?status=done")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert "done-doc.pdf" in resp.text
|
|
||||||
assert "pending-doc.pdf" not in resp.text
|
|
||||||
|
|
||||||
def test_jobs_list_status_filter_multi(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
rid_p = f"lp-mp-{uuid4().hex[:6]}"
|
|
||||||
rid_d = f"lp-md-{uuid4().hex[:6]}"
|
|
||||||
rid_e = f"lp-me-{uuid4().hex[:6]}"
|
|
||||||
self._submit(app, "ui-multi", rid_p, filename="pending-m.pdf")
|
|
||||||
self._submit(app, "ui-multi", rid_d, filename="done-m.pdf")
|
|
||||||
self._submit(app, "ui-multi", rid_e, filename="error-m.pdf")
|
|
||||||
|
|
||||||
done_row = _find_job(postgres_url, "ui-multi", rid_d)
|
|
||||||
err_row = _find_job(postgres_url, "ui-multi", rid_e)
|
|
||||||
assert done_row is not None and err_row is not None
|
|
||||||
_force_done(
|
|
||||||
postgres_url,
|
|
||||||
done_row.job_id,
|
|
||||||
response_body={"use_case": "bank_statement_header"},
|
|
||||||
)
|
|
||||||
_force_error(postgres_url, err_row.job_id)
|
|
||||||
|
|
||||||
resp = app.get("/ui/jobs?status=done&status=error")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.text
|
|
||||||
assert "done-m.pdf" in body
|
|
||||||
assert "error-m.pdf" in body
|
|
||||||
assert "pending-m.pdf" not in body
|
|
||||||
|
|
||||||
def test_jobs_list_client_id_filter(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
rid_a = f"lp-a-{uuid4().hex[:6]}"
|
|
||||||
rid_b = f"lp-b-{uuid4().hex[:6]}"
|
|
||||||
self._submit(app, "client-alpha", rid_a, filename="alpha.pdf")
|
|
||||||
self._submit(app, "client-beta", rid_b, filename="beta.pdf")
|
|
||||||
|
|
||||||
resp = app.get("/ui/jobs?client_id=client-alpha")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
body = resp.text
|
|
||||||
assert "alpha.pdf" in body
|
|
||||||
assert "beta.pdf" not in body
|
|
||||||
|
|
||||||
def test_jobs_list_pagination(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
rids = []
|
|
||||||
for i in range(7):
|
|
||||||
rid = f"lp-pg-{uuid4().hex[:6]}-{i}"
|
|
||||||
rids.append(rid)
|
|
||||||
self._submit(app, "ui-pg", rid, filename=f"pg-{i}.pdf")
|
|
||||||
|
|
||||||
resp_p1 = app.get("/ui/jobs?limit=5&offset=0&client_id=ui-pg")
|
|
||||||
assert resp_p1.status_code == 200
|
|
||||||
body_p1 = resp_p1.text
|
|
||||||
# Newest-first: last 5 uploaded are pg-6..pg-2.
|
|
||||||
for i in (2, 3, 4, 5, 6):
|
|
||||||
assert f"pg-{i}.pdf" in body_p1
|
|
||||||
assert "pg-1.pdf" not in body_p1
|
|
||||||
assert "pg-0.pdf" not in body_p1
|
|
||||||
|
|
||||||
resp_p2 = app.get("/ui/jobs?limit=5&offset=5&client_id=ui-pg")
|
|
||||||
assert resp_p2.status_code == 200
|
|
||||||
body_p2 = resp_p2.text
|
|
||||||
assert "pg-1.pdf" in body_p2
|
|
||||||
assert "pg-0.pdf" in body_p2
|
|
||||||
# Showing 2 of 7 on page 2.
|
|
||||||
assert "of 7" in body_p2
|
|
||||||
|
|
||||||
def test_jobs_list_missing_display_name_falls_back_to_basename(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
"""Legacy rows without display_name must still render via basename."""
|
|
||||||
|
|
||||||
from ix.contracts.request import Context, FileRef, RequestIX
|
|
||||||
|
|
||||||
legacy_req = RequestIX(
|
|
||||||
use_case="bank_statement_header",
|
|
||||||
ix_client_id="ui-legacy",
|
|
||||||
request_id=f"lp-legacy-{uuid4().hex[:6]}",
|
|
||||||
context=Context(
|
|
||||||
files=[FileRef(url="file:///tmp/ix/ui/listing-legacy.pdf")]
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
|
|
||||||
from ix.store import jobs_repo as _repo
|
|
||||||
|
|
||||||
async def _insert() -> UUID:
|
|
||||||
eng = create_async_engine(postgres_url)
|
|
||||||
sf = async_sessionmaker(eng, expire_on_commit=False)
|
|
||||||
try:
|
|
||||||
async with sf() as session:
|
|
||||||
job = await _repo.insert_pending(
|
|
||||||
session, legacy_req, callback_url=None
|
|
||||||
)
|
|
||||||
await session.commit()
|
|
||||||
return job.job_id
|
|
||||||
finally:
|
|
||||||
await eng.dispose()
|
|
||||||
|
|
||||||
asyncio.run(_insert())
|
|
||||||
|
|
||||||
resp = app.get("/ui/jobs?client_id=ui-legacy")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert "listing-legacy.pdf" in resp.text
|
|
||||||
|
|
||||||
def test_jobs_list_header_link_from_index(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
) -> None:
|
|
||||||
resp = app.get("/ui")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert 'href="/ui/jobs"' in resp.text
|
|
||||||
|
|
||||||
def test_jobs_list_header_link_from_detail(
|
|
||||||
self,
|
|
||||||
app: TestClient,
|
|
||||||
postgres_url: str,
|
|
||||||
) -> None:
|
|
||||||
rid = f"lp-hd-{uuid4().hex[:6]}"
|
|
||||||
self._submit(app, "ui-hd", rid)
|
|
||||||
row = _find_job(postgres_url, "ui-hd", rid)
|
|
||||||
assert row is not None
|
|
||||||
resp = app.get(f"/ui/jobs/{row.job_id}")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert 'href="/ui/jobs"' in resp.text
|
|
||||||
|
|
||||||
|
|
||||||
def _force_error(
|
|
||||||
postgres_url: str,
|
|
||||||
job_id, # type: ignore[no-untyped-def]
|
|
||||||
) -> None:
|
|
||||||
"""Flip a pending/running job to ``error`` with a canned error body."""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from datetime import UTC, datetime
|
|
||||||
|
|
||||||
from sqlalchemy import text
|
|
||||||
|
|
||||||
async def _go(): # type: ignore[no-untyped-def]
|
|
||||||
eng = create_async_engine(postgres_url)
|
|
||||||
try:
|
|
||||||
async with eng.begin() as conn:
|
|
||||||
await conn.execute(
|
|
||||||
text(
|
|
||||||
"UPDATE ix_jobs SET status='error', "
|
|
||||||
"response=CAST(:resp AS JSONB), finished_at=:now "
|
|
||||||
"WHERE job_id=:jid"
|
|
||||||
),
|
|
||||||
{
|
|
||||||
"resp": json.dumps({"error": "IX_002_000: forced"}),
|
|
||||||
"now": datetime.now(UTC),
|
|
||||||
"jid": str(job_id),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
await eng.dispose()
|
|
||||||
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
|
|
||||||
def _find_job(postgres_url: str, client_id: str, request_id: str): # type: ignore[no-untyped-def]
|
|
||||||
"""Look up an ``ix_jobs`` row via the async engine, wrapping the coroutine
|
|
||||||
for test convenience."""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json as _json
|
|
||||||
|
|
||||||
async def _go(): # type: ignore[no-untyped-def]
|
|
||||||
eng = create_async_engine(postgres_url)
|
|
||||||
sf = async_sessionmaker(eng, expire_on_commit=False)
|
|
||||||
try:
|
|
||||||
async with sf() as session:
|
|
||||||
r = await session.scalar(
|
|
||||||
select(IxJob).where(
|
|
||||||
IxJob.client_id == client_id,
|
|
||||||
IxJob.request_id == request_id,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
if r is None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
class _JobRow:
|
|
||||||
pass
|
|
||||||
|
|
||||||
out = _JobRow()
|
|
||||||
out.job_id = r.job_id
|
|
||||||
out.client_id = r.client_id
|
|
||||||
out.request_id = r.request_id
|
|
||||||
out.status = r.status
|
|
||||||
if isinstance(r.request, str):
|
|
||||||
out.request = _json.loads(r.request)
|
|
||||||
else:
|
|
||||||
out.request = r.request
|
|
||||||
return out
|
|
||||||
finally:
|
|
||||||
await eng.dispose()
|
|
||||||
|
|
||||||
return asyncio.run(_go())
|
|
||||||
|
|
||||||
|
|
||||||
def _force_done(
|
|
||||||
postgres_url: str,
|
|
||||||
job_id, # type: ignore[no-untyped-def]
|
|
||||||
response_body: dict,
|
|
||||||
) -> None:
|
|
||||||
"""Flip a pending job to ``done`` with the given response payload."""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from datetime import UTC, datetime
|
|
||||||
|
|
||||||
from sqlalchemy import text
|
|
||||||
|
|
||||||
async def _go(): # type: ignore[no-untyped-def]
|
|
||||||
eng = create_async_engine(postgres_url)
|
|
||||||
try:
|
|
||||||
async with eng.begin() as conn:
|
|
||||||
await conn.execute(
|
|
||||||
text(
|
|
||||||
"UPDATE ix_jobs SET status='done', "
|
|
||||||
"response=CAST(:resp AS JSONB), finished_at=:now "
|
|
||||||
"WHERE job_id=:jid"
|
|
||||||
),
|
|
||||||
{
|
|
||||||
"resp": json.dumps(response_body),
|
|
||||||
"now": datetime.now(UTC),
|
|
||||||
"jid": str(job_id),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
await eng.dispose()
|
|
||||||
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
||||||
|
|
||||||
def _force_running(
|
|
||||||
postgres_url: str,
|
|
||||||
job_id, # type: ignore[no-untyped-def]
|
|
||||||
seconds_ago: int = 10,
|
|
||||||
) -> None:
|
|
||||||
"""Flip a pending job to ``running`` with a backdated ``started_at``.
|
|
||||||
|
|
||||||
The fragment renders "Running for MM:SS" which needs a ``started_at`` in
|
|
||||||
the past; 10s is enough to produce a deterministic non-zero MM:SS.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from datetime import UTC, datetime, timedelta
|
|
||||||
|
|
||||||
from sqlalchemy import text
|
|
||||||
|
|
||||||
async def _go(): # type: ignore[no-untyped-def]
|
|
||||||
eng = create_async_engine(postgres_url)
|
|
||||||
try:
|
|
||||||
async with eng.begin() as conn:
|
|
||||||
await conn.execute(
|
|
||||||
text(
|
|
||||||
"UPDATE ix_jobs SET status='running', started_at=:t "
|
|
||||||
"WHERE job_id=:jid"
|
|
||||||
),
|
|
||||||
{
|
|
||||||
"t": datetime.now(UTC) - timedelta(seconds=seconds_ago),
|
|
||||||
"jid": str(job_id),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
await eng.dispose()
|
|
||||||
|
|
||||||
asyncio.run(_go())
|
|
||||||
|
|
@ -31,7 +31,6 @@ from ix.contracts import (
|
||||||
ResponseIX,
|
ResponseIX,
|
||||||
SegmentCitation,
|
SegmentCitation,
|
||||||
)
|
)
|
||||||
from ix.contracts.request import InlineUseCase, UseCaseFieldDef
|
|
||||||
|
|
||||||
|
|
||||||
class TestFileRef:
|
class TestFileRef:
|
||||||
|
|
@ -50,24 +49,6 @@ class TestFileRef:
|
||||||
assert fr.headers == {"Authorization": "Token abc"}
|
assert fr.headers == {"Authorization": "Token abc"}
|
||||||
assert fr.max_bytes == 1_000_000
|
assert fr.max_bytes == 1_000_000
|
||||||
|
|
||||||
def test_display_name_defaults_to_none(self) -> None:
|
|
||||||
fr = FileRef(url="file:///tmp/ix/ui/abc.pdf")
|
|
||||||
assert fr.display_name is None
|
|
||||||
|
|
||||||
def test_display_name_roundtrip(self) -> None:
|
|
||||||
fr = FileRef(
|
|
||||||
url="file:///tmp/ix/ui/abc.pdf",
|
|
||||||
display_name="my statement.pdf",
|
|
||||||
)
|
|
||||||
assert fr.display_name == "my statement.pdf"
|
|
||||||
dumped = fr.model_dump_json()
|
|
||||||
rt = FileRef.model_validate_json(dumped)
|
|
||||||
assert rt.display_name == "my statement.pdf"
|
|
||||||
# Backward-compat: a serialised FileRef without display_name still
|
|
||||||
# validates cleanly (older stored jobs predate the field).
|
|
||||||
legacy = FileRef.model_validate({"url": "file:///x.pdf"})
|
|
||||||
assert legacy.display_name is None
|
|
||||||
|
|
||||||
|
|
||||||
class TestOptionDefaults:
|
class TestOptionDefaults:
|
||||||
def test_ocr_defaults_match_spec(self) -> None:
|
def test_ocr_defaults_match_spec(self) -> None:
|
||||||
|
|
@ -201,32 +182,6 @@ class TestRequestIX:
|
||||||
with pytest.raises(ValidationError):
|
with pytest.raises(ValidationError):
|
||||||
RequestIX.model_validate({"use_case": "x"})
|
RequestIX.model_validate({"use_case": "x"})
|
||||||
|
|
||||||
def test_use_case_inline_defaults_to_none(self) -> None:
|
|
||||||
r = RequestIX(**self._minimal_payload())
|
|
||||||
assert r.use_case_inline is None
|
|
||||||
|
|
||||||
def test_use_case_inline_roundtrip(self) -> None:
|
|
||||||
payload = self._minimal_payload()
|
|
||||||
payload["use_case_inline"] = {
|
|
||||||
"use_case_name": "adhoc",
|
|
||||||
"system_prompt": "extract stuff",
|
|
||||||
"fields": [
|
|
||||||
{"name": "a", "type": "str", "required": True},
|
|
||||||
{"name": "b", "type": "int"},
|
|
||||||
],
|
|
||||||
}
|
|
||||||
r = RequestIX.model_validate(payload)
|
|
||||||
assert r.use_case_inline is not None
|
|
||||||
assert isinstance(r.use_case_inline, InlineUseCase)
|
|
||||||
assert r.use_case_inline.use_case_name == "adhoc"
|
|
||||||
assert len(r.use_case_inline.fields) == 2
|
|
||||||
assert isinstance(r.use_case_inline.fields[0], UseCaseFieldDef)
|
|
||||||
# Round-trip through JSON
|
|
||||||
dumped = r.model_dump_json()
|
|
||||||
r2 = RequestIX.model_validate_json(dumped)
|
|
||||||
assert r2.use_case_inline is not None
|
|
||||||
assert r2.use_case_inline.fields[1].type == "int"
|
|
||||||
|
|
||||||
|
|
||||||
class TestOCRResult:
|
class TestOCRResult:
|
||||||
def test_minimal_defaults(self) -> None:
|
def test_minimal_defaults(self) -> None:
|
||||||
|
|
|
||||||
|
|
@ -79,19 +79,10 @@ class TestInvokeHappyPath:
|
||||||
body_json = json.loads(body)
|
body_json = json.loads(body)
|
||||||
assert body_json["model"] == "gpt-oss:20b"
|
assert body_json["model"] == "gpt-oss:20b"
|
||||||
assert body_json["stream"] is False
|
assert body_json["stream"] is False
|
||||||
# No `format` is sent: Ollama 0.11.8 segfaults on full schemas and
|
assert body_json["format"] == _Schema.model_json_schema()
|
||||||
# aborts to `{}` with `format=json` on reasoning models. Schema is
|
|
||||||
# injected into the system prompt instead; we extract the trailing
|
|
||||||
# JSON blob from the response and validate via Pydantic.
|
|
||||||
assert "format" not in body_json
|
|
||||||
assert body_json["options"]["temperature"] == 0.2
|
assert body_json["options"]["temperature"] == 0.2
|
||||||
assert "reasoning_effort" not in body_json
|
assert "reasoning_effort" not in body_json
|
||||||
# A schema-guidance system message is prepended to the caller's
|
assert body_json["messages"] == [
|
||||||
# messages so Ollama (format=json loose mode) emits the right shape.
|
|
||||||
msgs = body_json["messages"]
|
|
||||||
assert msgs[0]["role"] == "system"
|
|
||||||
assert "JSON Schema" in msgs[0]["content"]
|
|
||||||
assert msgs[1:] == [
|
|
||||||
{"role": "system", "content": "You extract."},
|
{"role": "system", "content": "You extract."},
|
||||||
{"role": "user", "content": "Doc body"},
|
{"role": "user", "content": "Doc body"},
|
||||||
]
|
]
|
||||||
|
|
@ -125,10 +116,7 @@ class TestInvokeHappyPath:
|
||||||
import json
|
import json
|
||||||
|
|
||||||
request_body = json.loads(httpx_mock.get_requests()[0].read())
|
request_body = json.loads(httpx_mock.get_requests()[0].read())
|
||||||
# First message is the auto-injected schema guidance; after that
|
assert request_body["messages"] == [
|
||||||
# the caller's user message has its text parts joined.
|
|
||||||
assert request_body["messages"][0]["role"] == "system"
|
|
||||||
assert request_body["messages"][1:] == [
|
|
||||||
{"role": "user", "content": "part-a\npart-b"}
|
{"role": "user", "content": "part-a\npart-b"}
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,6 @@ from ix.contracts import (
|
||||||
RequestIX,
|
RequestIX,
|
||||||
ResponseIX,
|
ResponseIX,
|
||||||
)
|
)
|
||||||
from ix.contracts.request import InlineUseCase, UseCaseFieldDef
|
|
||||||
from ix.contracts.response import _InternalContext
|
from ix.contracts.response import _InternalContext
|
||||||
from ix.errors import IXErrorCode, IXException
|
from ix.errors import IXErrorCode, IXException
|
||||||
from ix.ingestion import FetchConfig
|
from ix.ingestion import FetchConfig
|
||||||
|
|
@ -245,102 +244,6 @@ class TestTextOnly:
|
||||||
assert ctx.texts == ["hello", "there"]
|
assert ctx.texts == ["hello", "there"]
|
||||||
|
|
||||||
|
|
||||||
class TestInlineUseCase:
|
|
||||||
def _make_inline_request(
|
|
||||||
self,
|
|
||||||
inline: InlineUseCase,
|
|
||||||
use_case: str = "adhoc-label",
|
|
||||||
texts: list[str] | None = None,
|
|
||||||
) -> RequestIX:
|
|
||||||
return RequestIX(
|
|
||||||
use_case=use_case,
|
|
||||||
use_case_inline=inline,
|
|
||||||
ix_client_id="test",
|
|
||||||
request_id="r-inline",
|
|
||||||
context=Context(files=[], texts=texts or ["hello"]),
|
|
||||||
options=Options(
|
|
||||||
ocr=OCROptions(use_ocr=True),
|
|
||||||
provenance=ProvenanceOptions(include_provenance=True),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
async def test_inline_use_case_overrides_registry(self, tmp_path: Path) -> None:
|
|
||||||
fetcher = FakeFetcher({})
|
|
||||||
ingestor = FakeIngestor([])
|
|
||||||
step = SetupStep(
|
|
||||||
fetcher=fetcher,
|
|
||||||
ingestor=ingestor,
|
|
||||||
tmp_dir=tmp_path / "work",
|
|
||||||
fetch_config=_make_cfg(),
|
|
||||||
mime_detector=_AlwaysMimePdf(),
|
|
||||||
)
|
|
||||||
inline = InlineUseCase(
|
|
||||||
use_case_name="adhoc",
|
|
||||||
system_prompt="Extract things.",
|
|
||||||
fields=[
|
|
||||||
UseCaseFieldDef(name="vendor", type="str", required=True),
|
|
||||||
UseCaseFieldDef(name="amount", type="decimal"),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
req = self._make_inline_request(inline)
|
|
||||||
resp = _make_response()
|
|
||||||
resp = await step.process(req, resp)
|
|
||||||
|
|
||||||
ctx = resp.context
|
|
||||||
assert ctx is not None
|
|
||||||
# The response class must have been built from our field list.
|
|
||||||
resp_cls = ctx.use_case_response # type: ignore[union-attr]
|
|
||||||
assert set(resp_cls.model_fields.keys()) == {"vendor", "amount"}
|
|
||||||
# Public display name reflects the inline label.
|
|
||||||
assert resp.use_case_name == "adhoc"
|
|
||||||
|
|
||||||
async def test_inline_precedence_when_both_set(self, tmp_path: Path) -> None:
|
|
||||||
# ``use_case`` is a valid registered name; ``use_case_inline`` is also
|
|
||||||
# present. Inline MUST win (documented precedence).
|
|
||||||
fetcher = FakeFetcher({})
|
|
||||||
ingestor = FakeIngestor([])
|
|
||||||
step = SetupStep(
|
|
||||||
fetcher=fetcher,
|
|
||||||
ingestor=ingestor,
|
|
||||||
tmp_dir=tmp_path / "work",
|
|
||||||
fetch_config=_make_cfg(),
|
|
||||||
mime_detector=_AlwaysMimePdf(),
|
|
||||||
)
|
|
||||||
inline = InlineUseCase(
|
|
||||||
use_case_name="override",
|
|
||||||
system_prompt="override prompt",
|
|
||||||
fields=[UseCaseFieldDef(name="just_me", type="str", required=True)],
|
|
||||||
)
|
|
||||||
req = self._make_inline_request(
|
|
||||||
inline, use_case="bank_statement_header"
|
|
||||||
)
|
|
||||||
resp = await step.process(req, _make_response())
|
|
||||||
resp_cls = resp.context.use_case_response # type: ignore[union-attr]
|
|
||||||
assert set(resp_cls.model_fields.keys()) == {"just_me"}
|
|
||||||
|
|
||||||
async def test_inline_with_bad_field_raises_ix_001_001(
|
|
||||||
self, tmp_path: Path
|
|
||||||
) -> None:
|
|
||||||
fetcher = FakeFetcher({})
|
|
||||||
ingestor = FakeIngestor([])
|
|
||||||
step = SetupStep(
|
|
||||||
fetcher=fetcher,
|
|
||||||
ingestor=ingestor,
|
|
||||||
tmp_dir=tmp_path / "work",
|
|
||||||
fetch_config=_make_cfg(),
|
|
||||||
mime_detector=_AlwaysMimePdf(),
|
|
||||||
)
|
|
||||||
inline = InlineUseCase(
|
|
||||||
use_case_name="bad",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="123bad", type="str")],
|
|
||||||
)
|
|
||||||
req = self._make_inline_request(inline)
|
|
||||||
with pytest.raises(IXException) as ei:
|
|
||||||
await step.process(req, _make_response())
|
|
||||||
assert ei.value.code is IXErrorCode.IX_001_001
|
|
||||||
|
|
||||||
|
|
||||||
class TestInternalContextShape:
|
class TestInternalContextShape:
|
||||||
async def test_context_is_internal_context_instance(self, tmp_path: Path) -> None:
|
async def test_context_is_internal_context_instance(self, tmp_path: Path) -> None:
|
||||||
fetcher = FakeFetcher({})
|
fetcher = FakeFetcher({})
|
||||||
|
|
|
||||||
|
|
@ -161,78 +161,6 @@ def _write_tiny_png(path: Path) -> None:
|
||||||
Image.new("RGB", (2, 2), color="white").save(path, format="PNG")
|
Image.new("RGB", (2, 2), color="white").save(path, format="PNG")
|
||||||
|
|
||||||
|
|
||||||
class TestGpuAvailableFlag:
|
|
||||||
def test_default_is_none(self) -> None:
|
|
||||||
client = SuryaOCRClient()
|
|
||||||
assert client.gpu_available is None
|
|
||||||
|
|
||||||
def test_warm_up_probes_cuda_true(self) -> None:
|
|
||||||
"""When torch reports CUDA, warm_up records True on the instance."""
|
|
||||||
|
|
||||||
client = SuryaOCRClient()
|
|
||||||
fake_foundation = MagicMock()
|
|
||||||
fake_recognition = MagicMock()
|
|
||||||
fake_detection = MagicMock()
|
|
||||||
fake_torch = SimpleNamespace(
|
|
||||||
cuda=SimpleNamespace(is_available=lambda: True)
|
|
||||||
)
|
|
||||||
|
|
||||||
module_patches = {
|
|
||||||
"surya.detection": SimpleNamespace(
|
|
||||||
DetectionPredictor=lambda: fake_detection
|
|
||||||
),
|
|
||||||
"surya.foundation": SimpleNamespace(
|
|
||||||
FoundationPredictor=lambda: fake_foundation
|
|
||||||
),
|
|
||||||
"surya.recognition": SimpleNamespace(
|
|
||||||
RecognitionPredictor=lambda _f: fake_recognition
|
|
||||||
),
|
|
||||||
"torch": fake_torch,
|
|
||||||
}
|
|
||||||
with patch.dict("sys.modules", module_patches):
|
|
||||||
client.warm_up()
|
|
||||||
|
|
||||||
assert client.gpu_available is True
|
|
||||||
assert client._recognition_predictor is fake_recognition
|
|
||||||
assert client._detection_predictor is fake_detection
|
|
||||||
|
|
||||||
def test_warm_up_probes_cuda_false(self) -> None:
|
|
||||||
"""CPU-mode host → warm_up records False."""
|
|
||||||
|
|
||||||
client = SuryaOCRClient()
|
|
||||||
fake_torch = SimpleNamespace(
|
|
||||||
cuda=SimpleNamespace(is_available=lambda: False)
|
|
||||||
)
|
|
||||||
module_patches = {
|
|
||||||
"surya.detection": SimpleNamespace(
|
|
||||||
DetectionPredictor=lambda: MagicMock()
|
|
||||||
),
|
|
||||||
"surya.foundation": SimpleNamespace(
|
|
||||||
FoundationPredictor=lambda: MagicMock()
|
|
||||||
),
|
|
||||||
"surya.recognition": SimpleNamespace(
|
|
||||||
RecognitionPredictor=lambda _f: MagicMock()
|
|
||||||
),
|
|
||||||
"torch": fake_torch,
|
|
||||||
}
|
|
||||||
with patch.dict("sys.modules", module_patches):
|
|
||||||
client.warm_up()
|
|
||||||
|
|
||||||
assert client.gpu_available is False
|
|
||||||
|
|
||||||
def test_warm_up_is_idempotent_for_probe(self) -> None:
|
|
||||||
"""Second warm_up short-circuits; probed flag is preserved."""
|
|
||||||
|
|
||||||
client = SuryaOCRClient()
|
|
||||||
client._recognition_predictor = MagicMock()
|
|
||||||
client._detection_predictor = MagicMock()
|
|
||||||
client.gpu_available = True
|
|
||||||
|
|
||||||
# No module patches — warm_up must NOT touch sys.modules or torch.
|
|
||||||
client.warm_up()
|
|
||||||
assert client.gpu_available is True
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("unused", [None]) # keep pytest happy if file ever runs alone
|
@pytest.mark.parametrize("unused", [None]) # keep pytest happy if file ever runs alone
|
||||||
def test_module_imports(unused: None) -> None:
|
def test_module_imports(unused: None) -> None:
|
||||||
assert SuryaOCRClient is not None
|
assert SuryaOCRClient is not None
|
||||||
|
|
|
||||||
|
|
@ -1,313 +0,0 @@
|
||||||
"""Tests for :mod:`ix.use_cases.inline` — dynamic Pydantic class builder.
|
|
||||||
|
|
||||||
The builder takes an :class:`InlineUseCase` (carried on :class:`RequestIX` as
|
|
||||||
``use_case_inline``) and produces a fresh ``(RequestClass, ResponseClass)``
|
|
||||||
pair that the pipeline can consume in place of a registered use case.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
from datetime import date, datetime
|
|
||||||
from decimal import Decimal
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from pydantic import BaseModel, ValidationError
|
|
||||||
|
|
||||||
from ix.contracts.request import InlineUseCase, UseCaseFieldDef
|
|
||||||
from ix.errors import IXErrorCode, IXException
|
|
||||||
from ix.use_cases.inline import build_use_case_classes
|
|
||||||
|
|
||||||
|
|
||||||
class TestUseCaseFieldDef:
|
|
||||||
def test_minimal(self) -> None:
|
|
||||||
fd = UseCaseFieldDef(name="foo", type="str")
|
|
||||||
assert fd.name == "foo"
|
|
||||||
assert fd.type == "str"
|
|
||||||
assert fd.required is False
|
|
||||||
assert fd.description is None
|
|
||||||
assert fd.choices is None
|
|
||||||
|
|
||||||
def test_extra_forbidden(self) -> None:
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
UseCaseFieldDef.model_validate(
|
|
||||||
{"name": "foo", "type": "str", "bogus": 1}
|
|
||||||
)
|
|
||||||
|
|
||||||
def test_invalid_type_rejected(self) -> None:
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
UseCaseFieldDef.model_validate({"name": "foo", "type": "list"})
|
|
||||||
|
|
||||||
|
|
||||||
class TestInlineUseCaseRoundtrip:
|
|
||||||
def test_json_roundtrip(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="Vendor Total",
|
|
||||||
system_prompt="Extract invoice total and vendor.",
|
|
||||||
default_model="qwen3:14b",
|
|
||||||
fields=[
|
|
||||||
UseCaseFieldDef(name="vendor", type="str", required=True),
|
|
||||||
UseCaseFieldDef(
|
|
||||||
name="total",
|
|
||||||
type="decimal",
|
|
||||||
required=True,
|
|
||||||
description="total amount due",
|
|
||||||
),
|
|
||||||
UseCaseFieldDef(
|
|
||||||
name="currency",
|
|
||||||
type="str",
|
|
||||||
choices=["USD", "EUR", "CHF"],
|
|
||||||
),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
dumped = iuc.model_dump_json()
|
|
||||||
round = InlineUseCase.model_validate_json(dumped)
|
|
||||||
assert round == iuc
|
|
||||||
# JSON is well-formed
|
|
||||||
json.loads(dumped)
|
|
||||||
|
|
||||||
def test_extra_forbidden(self) -> None:
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
InlineUseCase.model_validate(
|
|
||||||
{
|
|
||||||
"use_case_name": "X",
|
|
||||||
"system_prompt": "p",
|
|
||||||
"fields": [],
|
|
||||||
"bogus": 1,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class TestBuildBasicTypes:
|
|
||||||
@pytest.mark.parametrize(
|
|
||||||
"type_name, sample_value, bad_value",
|
|
||||||
[
|
|
||||||
("str", "hello", 123),
|
|
||||||
("int", 42, "nope"),
|
|
||||||
("float", 3.14, "nope"),
|
|
||||||
("bool", True, "nope"),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
def test_simple_type(
|
|
||||||
self, type_name: str, sample_value: object, bad_value: object
|
|
||||||
) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="field", type=type_name, required=True)],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
instance = resp_cls(field=sample_value)
|
|
||||||
assert instance.field == sample_value
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
resp_cls(field=bad_value)
|
|
||||||
|
|
||||||
def test_decimal_type(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="amount", type="decimal", required=True)],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
instance = resp_cls(amount="12.34")
|
|
||||||
assert isinstance(instance.amount, Decimal)
|
|
||||||
assert instance.amount == Decimal("12.34")
|
|
||||||
|
|
||||||
def test_date_type(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="d", type="date", required=True)],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
instance = resp_cls(d="2026-04-18")
|
|
||||||
assert instance.d == date(2026, 4, 18)
|
|
||||||
|
|
||||||
def test_datetime_type(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="ts", type="datetime", required=True)],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
instance = resp_cls(ts="2026-04-18T10:00:00")
|
|
||||||
assert isinstance(instance.ts, datetime)
|
|
||||||
|
|
||||||
|
|
||||||
class TestOptionalVsRequired:
|
|
||||||
def test_required_field_cannot_be_missing(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="must", type="str", required=True)],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
resp_cls()
|
|
||||||
|
|
||||||
def test_optional_field_defaults_to_none(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="maybe", type="str", required=False)],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
instance = resp_cls()
|
|
||||||
assert instance.maybe is None
|
|
||||||
|
|
||||||
def test_optional_field_schema_allows_null(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="maybe", type="str", required=False)],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
schema = resp_cls.model_json_schema()
|
|
||||||
# "maybe" accepts string or null
|
|
||||||
prop = schema["properties"]["maybe"]
|
|
||||||
# Pydantic may express Optional as anyOf [str, null] or a type list.
|
|
||||||
# Either is fine — just assert null is allowed somewhere.
|
|
||||||
dumped = json.dumps(prop)
|
|
||||||
assert "null" in dumped
|
|
||||||
|
|
||||||
|
|
||||||
class TestChoices:
|
|
||||||
def test_choices_for_str_produces_literal(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[
|
|
||||||
UseCaseFieldDef(
|
|
||||||
name="kind",
|
|
||||||
type="str",
|
|
||||||
required=True,
|
|
||||||
choices=["a", "b", "c"],
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
inst = resp_cls(kind="a")
|
|
||||||
assert inst.kind == "a"
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
resp_cls(kind="nope")
|
|
||||||
schema = resp_cls.model_json_schema()
|
|
||||||
# enum or const wind up in a referenced definition; walk the schema
|
|
||||||
dumped = json.dumps(schema)
|
|
||||||
assert '"a"' in dumped and '"b"' in dumped and '"c"' in dumped
|
|
||||||
|
|
||||||
def test_choices_for_non_str_raises_ix_001_001(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[
|
|
||||||
UseCaseFieldDef(
|
|
||||||
name="kind",
|
|
||||||
type="int",
|
|
||||||
required=True,
|
|
||||||
choices=["1", "2"],
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
with pytest.raises(IXException) as exc:
|
|
||||||
build_use_case_classes(iuc)
|
|
||||||
assert exc.value.code is IXErrorCode.IX_001_001
|
|
||||||
|
|
||||||
def test_empty_choices_list_ignored(self) -> None:
|
|
||||||
# An explicitly empty list is as-if choices were unset; builder must
|
|
||||||
# not break. If the caller sent choices=[] we treat the field as
|
|
||||||
# plain str.
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[
|
|
||||||
UseCaseFieldDef(
|
|
||||||
name="kind", type="str", required=True, choices=[]
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
inst = resp_cls(kind="anything")
|
|
||||||
assert inst.kind == "anything"
|
|
||||||
|
|
||||||
|
|
||||||
class TestValidation:
|
|
||||||
def test_duplicate_field_names_raise(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[
|
|
||||||
UseCaseFieldDef(name="foo", type="str"),
|
|
||||||
UseCaseFieldDef(name="foo", type="int"),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
with pytest.raises(IXException) as exc:
|
|
||||||
build_use_case_classes(iuc)
|
|
||||||
assert exc.value.code is IXErrorCode.IX_001_001
|
|
||||||
|
|
||||||
def test_invalid_field_name_raises(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="123abc", type="str")],
|
|
||||||
)
|
|
||||||
with pytest.raises(IXException) as exc:
|
|
||||||
build_use_case_classes(iuc)
|
|
||||||
assert exc.value.code is IXErrorCode.IX_001_001
|
|
||||||
|
|
||||||
def test_empty_fields_list_raises(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X", system_prompt="p", fields=[]
|
|
||||||
)
|
|
||||||
with pytest.raises(IXException) as exc:
|
|
||||||
build_use_case_classes(iuc)
|
|
||||||
assert exc.value.code is IXErrorCode.IX_001_001
|
|
||||||
|
|
||||||
|
|
||||||
class TestResponseClassNaming:
|
|
||||||
def test_class_name_sanitised(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="Bank / Statement — header!",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="x", type="str")],
|
|
||||||
)
|
|
||||||
_req_cls, resp_cls = build_use_case_classes(iuc)
|
|
||||||
assert resp_cls.__name__.startswith("Inline_")
|
|
||||||
# Only alphanumerics and underscores remain.
|
|
||||||
assert all(c.isalnum() or c == "_" for c in resp_cls.__name__)
|
|
||||||
|
|
||||||
def test_fresh_instances_per_call(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="X",
|
|
||||||
system_prompt="p",
|
|
||||||
fields=[UseCaseFieldDef(name="x", type="str")],
|
|
||||||
)
|
|
||||||
req1, resp1 = build_use_case_classes(iuc)
|
|
||||||
req2, resp2 = build_use_case_classes(iuc)
|
|
||||||
assert resp1 is not resp2
|
|
||||||
assert req1 is not req2
|
|
||||||
|
|
||||||
|
|
||||||
class TestRequestClassShape:
|
|
||||||
def test_request_class_exposes_prompt_and_default(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="My Case",
|
|
||||||
system_prompt="Follow directions.",
|
|
||||||
default_model="qwen3:14b",
|
|
||||||
fields=[UseCaseFieldDef(name="x", type="str")],
|
|
||||||
)
|
|
||||||
req_cls, _resp_cls = build_use_case_classes(iuc)
|
|
||||||
inst = req_cls()
|
|
||||||
assert inst.use_case_name == "My Case"
|
|
||||||
assert inst.system_prompt == "Follow directions."
|
|
||||||
assert inst.default_model == "qwen3:14b"
|
|
||||||
assert issubclass(req_cls, BaseModel)
|
|
||||||
|
|
||||||
def test_default_model_none_when_unset(self) -> None:
|
|
||||||
iuc = InlineUseCase(
|
|
||||||
use_case_name="My Case",
|
|
||||||
system_prompt="Follow directions.",
|
|
||||||
fields=[UseCaseFieldDef(name="x", type="str")],
|
|
||||||
)
|
|
||||||
req_cls, _resp_cls = build_use_case_classes(iuc)
|
|
||||||
inst = req_cls()
|
|
||||||
assert inst.default_model is None
|
|
||||||
294
uv.lock
294
uv.lock
|
|
@ -7,15 +7,6 @@ resolution-markers = [
|
||||||
"(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')",
|
"(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "aiofiles"
|
|
||||||
version = "25.1.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/41/c3/534eac40372d8ee36ef40df62ec129bee4fdb5ad9706e58a29be53b2c970/aiofiles-25.1.0.tar.gz", hash = "sha256:a8d728f0a29de45dc521f18f07297428d56992a742f0cd2701ba86e44d23d5b2", size = 46354, upload-time = "2025-10-09T20:51:04.358Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/bc/8a/340a1555ae33d7354dbca4faa54948d76d89a27ceef032c8c3bc661d003e/aiofiles-25.1.0-py3-none-any.whl", hash = "sha256:abe311e527c862958650f9438e859c1fa7568a141b22abcd015e120e86a85695", size = 14668, upload-time = "2025-10-09T20:51:03.174Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "alembic"
|
name = "alembic"
|
||||||
version = "1.18.4"
|
version = "1.18.4"
|
||||||
|
|
@ -213,75 +204,6 @@ wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
|
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "cuda-bindings"
|
|
||||||
version = "13.2.0"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
dependencies = [
|
|
||||||
{ name = "cuda-pathfinder", marker = "sys_platform != 'darwin'" },
|
|
||||||
]
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/52/c8/b2589d68acf7e3d63e2be330b84bc25712e97ed799affbca7edd7eae25d6/cuda_bindings-13.2.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e865447abfb83d6a98ad5130ed3c70b1fc295ae3eeee39fd07b4ddb0671b6788", size = 5722404, upload-time = "2026-03-11T00:12:44.041Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/1f/92/f899f7bbb5617bb65ec52a6eac1e9a1447a86b916c4194f8a5001b8cde0c/cuda_bindings-13.2.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:46d8776a55d6d5da9dd6e9858fba2efcda2abe6743871dee47dd06eb8cb6d955", size = 6320619, upload-time = "2026-03-11T00:12:45.939Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/df/93/eef988860a3ca985f82c4f3174fc0cdd94e07331ba9a92e8e064c260337f/cuda_bindings-13.2.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6629ca2df6f795b784752409bcaedbd22a7a651b74b56a165ebc0c9dcbd504d0", size = 5614610, upload-time = "2026-03-11T00:12:50.337Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/18/23/6db3aba46864aee357ab2415135b3fe3da7e9f1fa0221fa2a86a5968099c/cuda_bindings-13.2.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7dca0da053d3b4cc4869eff49c61c03f3c5dbaa0bcd712317a358d5b8f3f385d", size = 6149914, upload-time = "2026-03-11T00:12:52.374Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c0/87/87a014f045b77c6de5c8527b0757fe644417b184e5367db977236a141602/cuda_bindings-13.2.0-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a6464b30f46692d6c7f65d4a0e0450d81dd29de3afc1bb515653973d01c2cd6e", size = 5685673, upload-time = "2026-03-11T00:12:56.371Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ee/5e/c0fe77a73aaefd3fff25ffaccaac69c5a63eafdf8b9a4c476626ef0ac703/cuda_bindings-13.2.0-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f4af9f3e1be603fa12d5ad6cfca7844c9d230befa9792b5abdf7dd79979c3626", size = 6191386, upload-time = "2026-03-11T00:12:58.965Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/5f/58/ed2c3b39c8dd5f96aa7a4abef0d47a73932c7a988e30f5fa428f00ed0da1/cuda_bindings-13.2.0-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:df850a1ff8ce1b3385257b08e47b70e959932f5f432d0a4e46a355962b4e4771", size = 5507469, upload-time = "2026-03-11T00:13:04.063Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/1f/01/0c941b112ceeb21439b05895eace78ca1aa2eaaf695c8521a068fd9b4c00/cuda_bindings-13.2.0-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e8a16384c6494e5485f39314b0b4afb04bee48d49edb16d5d8593fd35bbd231b", size = 6059693, upload-time = "2026-03-11T00:13:06.003Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "cuda-pathfinder"
|
|
||||||
version = "1.5.3"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/d3/d6/ac63065d33dd700fee7ebd7d287332401b54e31b9346e142f871e1f0b116/cuda_pathfinder-1.5.3-py3-none-any.whl", hash = "sha256:dff021123aedbb4117cc7ec81717bbfe198fb4e8b5f1ee57e0e084fec5c8577d", size = 49991, upload-time = "2026-04-14T20:09:27.037Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "cuda-toolkit"
|
|
||||||
version = "13.0.2"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/57/b2/453099f5f3b698d7d0eab38916aac44c7f76229f451709e2eb9db6615dcd/cuda_toolkit-13.0.2-py2.py3-none-any.whl", hash = "sha256:b198824cf2f54003f50d64ada3a0f184b42ca0846c1c94192fa269ecd97a66eb", size = 2364, upload-time = "2025-12-19T23:24:07.328Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[package.optional-dependencies]
|
|
||||||
cublas = [
|
|
||||||
{ name = "nvidia-cublas", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
cudart = [
|
|
||||||
{ name = "nvidia-cuda-runtime", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
cufft = [
|
|
||||||
{ name = "nvidia-cufft", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
cufile = [
|
|
||||||
{ name = "nvidia-cufile", marker = "sys_platform == 'linux'" },
|
|
||||||
]
|
|
||||||
cupti = [
|
|
||||||
{ name = "nvidia-cuda-cupti", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
curand = [
|
|
||||||
{ name = "nvidia-curand", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
cusolver = [
|
|
||||||
{ name = "nvidia-cusolver", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
cusparse = [
|
|
||||||
{ name = "nvidia-cusparse", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
nvjitlink = [
|
|
||||||
{ name = "nvidia-nvjitlink", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
nvrtc = [
|
|
||||||
{ name = "nvidia-cuda-nvrtc", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
nvtx = [
|
|
||||||
{ name = "nvidia-nvtx", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "distlib"
|
name = "distlib"
|
||||||
version = "0.4.0"
|
version = "0.4.0"
|
||||||
|
|
@ -530,19 +452,16 @@ name = "infoxtractor"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
source = { editable = "." }
|
source = { editable = "." }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "aiofiles" },
|
|
||||||
{ name = "alembic" },
|
{ name = "alembic" },
|
||||||
{ name = "asyncpg" },
|
{ name = "asyncpg" },
|
||||||
{ name = "fastapi" },
|
{ name = "fastapi" },
|
||||||
{ name = "httpx" },
|
{ name = "httpx" },
|
||||||
{ name = "jinja2" },
|
|
||||||
{ name = "pillow" },
|
{ name = "pillow" },
|
||||||
{ name = "pydantic" },
|
{ name = "pydantic" },
|
||||||
{ name = "pydantic-settings" },
|
{ name = "pydantic-settings" },
|
||||||
{ name = "pymupdf" },
|
{ name = "pymupdf" },
|
||||||
{ name = "python-dateutil" },
|
{ name = "python-dateutil" },
|
||||||
{ name = "python-magic" },
|
{ name = "python-magic" },
|
||||||
{ name = "python-multipart" },
|
|
||||||
{ name = "sqlalchemy", extra = ["asyncio"] },
|
{ name = "sqlalchemy", extra = ["asyncio"] },
|
||||||
{ name = "uvicorn", extra = ["standard"] },
|
{ name = "uvicorn", extra = ["standard"] },
|
||||||
]
|
]
|
||||||
|
|
@ -562,12 +481,10 @@ ocr = [
|
||||||
|
|
||||||
[package.metadata]
|
[package.metadata]
|
||||||
requires-dist = [
|
requires-dist = [
|
||||||
{ name = "aiofiles", specifier = ">=24.1" },
|
|
||||||
{ name = "alembic", specifier = ">=1.14" },
|
{ name = "alembic", specifier = ">=1.14" },
|
||||||
{ name = "asyncpg", specifier = ">=0.30" },
|
{ name = "asyncpg", specifier = ">=0.30" },
|
||||||
{ name = "fastapi", specifier = ">=0.115" },
|
{ name = "fastapi", specifier = ">=0.115" },
|
||||||
{ name = "httpx", specifier = ">=0.27" },
|
{ name = "httpx", specifier = ">=0.27" },
|
||||||
{ name = "jinja2", specifier = ">=3.1" },
|
|
||||||
{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.13" },
|
{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.13" },
|
||||||
{ name = "pillow", specifier = ">=10.2,<11.0" },
|
{ name = "pillow", specifier = ">=10.2,<11.0" },
|
||||||
{ name = "pydantic", specifier = ">=2.9" },
|
{ name = "pydantic", specifier = ">=2.9" },
|
||||||
|
|
@ -578,11 +495,10 @@ requires-dist = [
|
||||||
{ name = "pytest-httpx", marker = "extra == 'dev'", specifier = ">=0.32" },
|
{ name = "pytest-httpx", marker = "extra == 'dev'", specifier = ">=0.32" },
|
||||||
{ name = "python-dateutil", specifier = ">=2.9" },
|
{ name = "python-dateutil", specifier = ">=2.9" },
|
||||||
{ name = "python-magic", specifier = ">=0.4.27" },
|
{ name = "python-magic", specifier = ">=0.4.27" },
|
||||||
{ name = "python-multipart", specifier = ">=0.0.12" },
|
|
||||||
{ name = "ruff", marker = "extra == 'dev'", specifier = ">=0.8" },
|
{ name = "ruff", marker = "extra == 'dev'", specifier = ">=0.8" },
|
||||||
{ name = "sqlalchemy", extras = ["asyncio"], specifier = ">=2.0.36" },
|
{ name = "sqlalchemy", extras = ["asyncio"], specifier = ">=2.0.36" },
|
||||||
{ name = "surya-ocr", marker = "extra == 'ocr'", specifier = ">=0.17,<0.18" },
|
{ name = "surya-ocr", marker = "extra == 'ocr'", specifier = ">=0.9" },
|
||||||
{ name = "torch", marker = "extra == 'ocr'", specifier = ">=2.7" },
|
{ name = "torch", marker = "extra == 'ocr'", specifier = ">=2.4", index = "https://download.pytorch.org/whl/cu124" },
|
||||||
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.32" },
|
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.32" },
|
||||||
]
|
]
|
||||||
provides-extras = ["ocr", "dev"]
|
provides-extras = ["ocr", "dev"]
|
||||||
|
|
@ -884,152 +800,121 @@ wheels = [
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cublas"
|
name = "nvidia-cublas-cu12"
|
||||||
version = "13.1.0.3"
|
version = "12.4.5.8"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/e1/a5/fce49e2ae977e0ccc084e5adafceb4f0ac0c8333cb6863501618a7277f67/nvidia_cublas-13.1.0.3-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:c86fc7f7ae36d7528288c5d88098edcb7b02c633d262e7ddbb86b0ad91be5df2", size = 542851226, upload-time = "2025-10-09T08:59:04.818Z" },
|
{ url = "https://files.pythonhosted.org/packages/ae/71/1c91302526c45ab494c23f61c7a84aa568b8c1f9d196efa5993957faf906/nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl", hash = "sha256:2fc8da60df463fdefa81e323eef2e36489e1c94335b5358bcb38360adf75ac9b", size = 363438805, upload-time = "2024-04-03T20:57:06.025Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/e7/44/423ac00af4dd95a5aeb27207e2c0d9b7118702149bf4704c3ddb55bb7429/nvidia_cublas-13.1.0.3-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:ee8722c1f0145ab246bccb9e452153b5e0515fd094c3678df50b2a0888b8b171", size = 423133236, upload-time = "2025-10-09T08:59:32.536Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cuda-cupti"
|
name = "nvidia-cuda-cupti-cu12"
|
||||||
version = "13.0.85"
|
version = "12.4.127"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/2a/2a/80353b103fc20ce05ef51e928daed4b6015db4aaa9162ed0997090fe2250/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_aarch64.whl", hash = "sha256:796bd679890ee55fb14a94629b698b6db54bcfd833d391d5e94017dd9d7d3151", size = 10310827, upload-time = "2025-09-04T08:26:42.012Z" },
|
{ url = "https://files.pythonhosted.org/packages/67/42/f4f60238e8194a3106d06a058d494b18e006c10bb2b915655bd9f6ea4cb1/nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:9dec60f5ac126f7bb551c055072b69d85392b13311fcc1bcda2202d172df30fb", size = 13813957, upload-time = "2024-04-03T20:55:01.564Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/33/6d/737d164b4837a9bbd202f5ae3078975f0525a55730fe871d8ed4e3b952b0/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl", hash = "sha256:4eb01c08e859bf924d222250d2e8f8b8ff6d3db4721288cf35d14252a4d933c8", size = 10715597, upload-time = "2025-09-04T08:26:51.312Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cuda-nvrtc"
|
name = "nvidia-cuda-nvrtc-cu12"
|
||||||
version = "13.0.88"
|
version = "12.4.127"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/c3/68/483a78f5e8f31b08fb1bb671559968c0ca3a065ac7acabfc7cee55214fd6/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:ad9b6d2ead2435f11cbb6868809d2adeeee302e9bb94bcf0539c7a40d80e8575", size = 90215200, upload-time = "2025-09-04T08:28:44.204Z" },
|
{ url = "https://files.pythonhosted.org/packages/2c/14/91ae57cd4db3f9ef7aa99f4019cfa8d54cb4caa7e00975df6467e9725a9f/nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:a178759ebb095827bd30ef56598ec182b85547f1508941a3d560eb7ea1fbf338", size = 24640306, upload-time = "2024-04-03T20:56:01.463Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/b7/dc/6bb80850e0b7edd6588d560758f17e0550893a1feaf436807d64d2da040f/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d27f20a0ca67a4bb34268a5e951033496c5b74870b868bacd046b1b8e0c3267b", size = 43015449, upload-time = "2025-09-04T08:28:20.239Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cuda-runtime"
|
name = "nvidia-cuda-runtime-cu12"
|
||||||
version = "13.0.96"
|
version = "12.4.127"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/87/4f/17d7b9b8e285199c58ce28e31b5c5bbaa4d8271af06a89b6405258245de2/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ef9bcbe90493a2b9d810e43d249adb3d02e98dd30200d86607d8d02687c43f55", size = 2261060, upload-time = "2025-10-09T08:55:15.78Z" },
|
{ url = "https://files.pythonhosted.org/packages/ea/27/1795d86fe88ef397885f2e580ac37628ed058a92ed2c39dc8eac3adf0619/nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:64403288fa2136ee8e467cdc9c9427e0434110899d07c779f25b5c068934faa5", size = 883737, upload-time = "2024-04-03T20:54:51.355Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/2e/24/d1558f3b68b1d26e706813b1d10aa1d785e4698c425af8db8edc3dced472/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7f82250d7782aa23b6cfe765ecc7db554bd3c2870c43f3d1821f1d18aebf0548", size = 2243632, upload-time = "2025-10-09T08:55:36.117Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cudnn-cu13"
|
name = "nvidia-cudnn-cu12"
|
||||||
version = "9.19.0.56"
|
version = "9.1.0.70"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "nvidia-cublas", marker = "sys_platform != 'darwin'" },
|
{ name = "nvidia-cublas-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" },
|
||||||
]
|
]
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/f1/84/26025437c1e6b61a707442184fa0c03d083b661adf3a3eecfd6d21677740/nvidia_cudnn_cu13-9.19.0.56-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:6ed29ffaee1176c612daf442e4dd6cfeb6a0caa43ddcbeb59da94953030b1be4", size = 433781201, upload-time = "2026-02-03T20:40:53.805Z" },
|
{ url = "https://files.pythonhosted.org/packages/9f/fd/713452cd72343f682b1c7b9321e23829f00b842ceaedcda96e742ea0b0b3/nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl", hash = "sha256:165764f44ef8c61fcdfdfdbe769d687e06374059fbb388b6c89ecb0e28793a6f", size = 664752741, upload-time = "2024-04-22T15:24:15.253Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/a3/22/0b4b932655d17a6da1b92fa92ab12844b053bb2ac2475e179ba6f043da1e/nvidia_cudnn_cu13-9.19.0.56-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:d20e1734305e9d68889a96e3f35094d733ff1f83932ebe462753973e53a572bf", size = 366066321, upload-time = "2026-02-03T20:44:52.837Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cufft"
|
name = "nvidia-cufft-cu12"
|
||||||
version = "12.0.0.61"
|
version = "11.2.1.3"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "nvidia-nvjitlink", marker = "sys_platform != 'darwin'" },
|
{ name = "nvidia-nvjitlink-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" },
|
||||||
]
|
]
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/8b/ae/f417a75c0259e85c1d2f83ca4e960289a5f814ed0cea74d18c353d3e989d/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2708c852ef8cd89d1d2068bdbece0aa188813a0c934db3779b9b1faa8442e5f5", size = 214053554, upload-time = "2025-09-04T08:31:38.196Z" },
|
{ url = "https://files.pythonhosted.org/packages/27/94/3266821f65b92b3138631e9c8e7fe1fb513804ac934485a8d05776e1dd43/nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl", hash = "sha256:f083fc24912aa410be21fa16d157fed2055dab1cc4b6934a0e03cba69eb242b9", size = 211459117, upload-time = "2024-04-03T20:57:40.402Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/a8/2f/7b57e29836ea8714f81e9898409196f47d772d5ddedddf1592eadb8ab743/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:6c44f692dce8fd5ffd3e3df134b6cdb9c2f72d99cf40b62c32dde45eea9ddad3", size = 214085489, upload-time = "2025-09-04T08:31:56.044Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cufile"
|
name = "nvidia-curand-cu12"
|
||||||
version = "1.15.1.6"
|
version = "10.3.5.147"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/3f/70/4f193de89a48b71714e74602ee14d04e4019ad36a5a9f20c425776e72cd6/nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:08a3ecefae5a01c7f5117351c64f17c7c62efa5fffdbe24fc7d298da19cd0b44", size = 1223672, upload-time = "2025-09-04T08:32:22.779Z" },
|
{ url = "https://files.pythonhosted.org/packages/8a/6d/44ad094874c6f1b9c654f8ed939590bdc408349f137f9b98a3a23ccec411/nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl", hash = "sha256:a88f583d4e0bb643c49743469964103aa59f7f708d862c3ddb0fc07f851e3b8b", size = 56305206, upload-time = "2024-04-03T20:58:08.722Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/ab/73/cc4a14c9813a8a0d509417cf5f4bdaba76e924d58beb9864f5a7baceefbf/nvidia_cufile-1.15.1.6-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:bdc0deedc61f548bddf7733bdc216456c2fdb101d020e1ab4b88d232d5e2f6d1", size = 1136992, upload-time = "2025-09-04T08:32:14.119Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-curand"
|
name = "nvidia-cusolver-cu12"
|
||||||
version = "10.4.0.35"
|
version = "11.6.1.9"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/1e/72/7c2ae24fb6b63a32e6ae5d241cc65263ea18d08802aaae087d9f013335a2/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:133df5a7509c3e292aaa2b477afd0194f06ce4ea24d714d616ff36439cee349a", size = 61962106, upload-time = "2025-08-04T10:21:41.128Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a5/9f/be0a41ca4a4917abf5cb9ae0daff1a6060cc5de950aec0396de9f3b52bc5/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:1aee33a5da6e1db083fe2b90082def8915f30f3248d5896bcec36a579d941bfc", size = 59544258, upload-time = "2025-08-04T10:22:03.992Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "nvidia-cusolver"
|
|
||||||
version = "12.0.4.66"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "nvidia-cublas", marker = "sys_platform != 'darwin'" },
|
{ name = "nvidia-cublas-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" },
|
||||||
{ name = "nvidia-cusparse", marker = "sys_platform != 'darwin'" },
|
{ name = "nvidia-cusparse-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" },
|
||||||
{ name = "nvidia-nvjitlink", marker = "sys_platform != 'darwin'" },
|
{ name = "nvidia-nvjitlink-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" },
|
||||||
]
|
]
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/c8/c3/b30c9e935fc01e3da443ec0116ed1b2a009bb867f5324d3f2d7e533e776b/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:02c2457eaa9e39de20f880f4bd8820e6a1cfb9f9a34f820eb12a155aa5bc92d2", size = 223467760, upload-time = "2025-09-04T08:33:04.222Z" },
|
{ url = "https://files.pythonhosted.org/packages/3a/e1/5b9089a4b2a4790dfdea8b3a006052cfecff58139d5a4e34cb1a51df8d6f/nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl", hash = "sha256:19e33fa442bcfd085b3086c4ebf7e8debc07cfe01e11513cc6d332fd918ac260", size = 127936057, upload-time = "2024-04-03T20:58:28.735Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/5f/67/cba3777620cdacb99102da4042883709c41c709f4b6323c10781a9c3aa34/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:0a759da5dea5c0ea10fd307de75cdeb59e7ea4fcb8add0924859b944babf1112", size = 200941980, upload-time = "2025-09-04T08:33:22.767Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cusparse"
|
name = "nvidia-cusparse-cu12"
|
||||||
version = "12.6.3.3"
|
version = "12.3.1.170"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "nvidia-nvjitlink", marker = "sys_platform != 'darwin'" },
|
{ name = "nvidia-nvjitlink-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" },
|
||||||
]
|
]
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/f8/94/5c26f33738ae35276672f12615a64bd008ed5be6d1ebcb23579285d960a9/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:80bcc4662f23f1054ee334a15c72b8940402975e0eab63178fc7e670aa59472c", size = 162155568, upload-time = "2025-09-04T08:33:42.864Z" },
|
{ url = "https://files.pythonhosted.org/packages/db/f7/97a9ea26ed4bbbfc2d470994b8b4f338ef663be97b8f677519ac195e113d/nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl", hash = "sha256:ea4f11a2904e2a8dc4b1833cc1b5181cde564edd0d5cd33e3c168eff2d1863f1", size = 207454763, upload-time = "2024-04-03T20:58:59.995Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/fa/18/623c77619c31d62efd55302939756966f3ecc8d724a14dab2b75f1508850/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2b3c89c88d01ee0e477cb7f82ef60a11a4bcd57b6b87c33f789350b59759360b", size = 145942937, upload-time = "2025-09-04T08:33:58.029Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-cusparselt-cu13"
|
name = "nvidia-cusparselt-cu12"
|
||||||
version = "0.8.0"
|
version = "0.6.2"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/46/10/8dcd1175260706a2fc92a16a52e306b71d4c1ea0b0cc4a9484183399818a/nvidia_cusparselt_cu13-0.8.0-py3-none-manylinux2014_aarch64.whl", hash = "sha256:400c6ed1cf6780fc6efedd64ec9f1345871767e6a1a0a552a1ea0578117ea77c", size = 220791277, upload-time = "2025-08-13T19:22:40.982Z" },
|
{ url = "https://files.pythonhosted.org/packages/78/a8/bcbb63b53a4b1234feeafb65544ee55495e1bb37ec31b999b963cbccfd1d/nvidia_cusparselt_cu12-0.6.2-py3-none-manylinux2014_x86_64.whl", hash = "sha256:df2c24502fd76ebafe7457dbc4716b2fec071aabaed4fb7691a201cde03704d9", size = 150057751, upload-time = "2024-07-23T02:35:53.074Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/fd/53/43b0d71f4e702fa9733f8b4571fdca50a8813f1e450b656c239beff12315/nvidia_cusparselt_cu13-0.8.0-py3-none-manylinux2014_x86_64.whl", hash = "sha256:25e30a8a7323935d4ad0340b95a0b69926eee755767e8e0b1cf8dd85b197d3fd", size = 169884119, upload-time = "2025-08-13T19:23:41.967Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-nccl-cu13"
|
name = "nvidia-nccl-cu12"
|
||||||
version = "2.28.9"
|
version = "2.21.5"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/39/55/1920646a2e43ffd4fc958536b276197ed740e9e0c54105b4bb3521591fc7/nvidia_nccl_cu13-2.28.9-py3-none-manylinux_2_18_aarch64.whl", hash = "sha256:01c873ba1626b54caa12272ed228dc5b2781545e0ae8ba3f432a8ef1c6d78643", size = 196561677, upload-time = "2025-11-18T05:49:03.45Z" },
|
{ url = "https://files.pythonhosted.org/packages/df/99/12cd266d6233f47d00daf3a72739872bdc10267d0383508b0b9c84a18bb6/nvidia_nccl_cu12-2.21.5-py3-none-manylinux2014_x86_64.whl", hash = "sha256:8579076d30a8c24988834445f8d633c697d42397e92ffc3f63fa26766d25e0a0", size = 188654414, upload-time = "2024-04-03T15:32:57.427Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/b0/b4/878fefaad5b2bcc6fcf8d474a25e3e3774bc5133e4b58adff4d0bca238bc/nvidia_nccl_cu13-2.28.9-py3-none-manylinux_2_18_x86_64.whl", hash = "sha256:e4553a30f34195f3fa1da02a6da3d6337d28f2003943aa0a3d247bbc25fefc42", size = 196493177, upload-time = "2025-11-18T05:49:17.677Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-nvjitlink"
|
name = "nvidia-nvjitlink-cu12"
|
||||||
version = "13.0.88"
|
version = "12.4.127"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/56/7a/123e033aaff487c77107195fa5a2b8686795ca537935a24efae476c41f05/nvidia_nvjitlink-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:13a74f429e23b921c1109976abefacc69835f2f433ebd323d3946e11d804e47b", size = 40713933, upload-time = "2025-09-04T08:35:43.553Z" },
|
{ url = "https://files.pythonhosted.org/packages/ff/ff/847841bacfbefc97a00036e0fce5a0f086b640756dc38caea5e1bb002655/nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:06b3b9b25bf3f8af351d664978ca26a16d2c5127dbd53c0497e28d1fb9611d57", size = 21066810, upload-time = "2024-04-03T20:59:46.957Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/ab/2c/93c5250e64df4f894f1cbb397c6fd71f79813f9fd79d7cd61de3f97b3c2d/nvidia_nvjitlink-13.0.88-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:e931536ccc7d467a98ba1d8b89ff7fa7f1fa3b13f2b0069118cd7f47bff07d0c", size = 38768748, upload-time = "2025-09-04T08:35:20.008Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nvidia-nvshmem-cu13"
|
name = "nvidia-nvtx-cu12"
|
||||||
version = "3.4.5"
|
version = "12.4.127"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/dc/0f/05cc9c720236dcd2db9c1ab97fff629e96821be2e63103569da0c9b72f19/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:6dc2a197f38e5d0376ad52cd1a2a3617d3cdc150fd5966f4aee9bcebb1d68fe9", size = 60215947, upload-time = "2025-09-06T00:32:20.022Z" },
|
{ url = "https://files.pythonhosted.org/packages/87/20/199b8713428322a2f22b722c62b8cc278cc53dffa9705d744484b5035ee9/nvidia_nvtx_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:781e950d9b9f60d8241ccea575b32f5105a5baf4c2351cab5256a24869f12a1a", size = 99144, upload-time = "2024-04-03T20:56:12.406Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/3c/35/a9bf80a609e74e3b000fef598933235c908fcefcef9026042b8e6dfde2a9/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:290f0a2ee94c9f3687a02502f3b9299a9f9fe826e6d0287ee18482e78d495b80", size = 60412546, upload-time = "2025-09-06T00:32:41.564Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "nvidia-nvtx"
|
|
||||||
version = "13.0.85"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c2/f3/d86c845465a2723ad7e1e5c36dcd75ddb82898b3f53be47ebd429fb2fa5d/nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:4936d1d6780fbe68db454f5e72a42ff64d1fd6397df9f363ae786930fd5c1cd4", size = 148047, upload-time = "2025-09-04T08:29:01.761Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/a8/64/3708a90d1ebe202ffdeb7185f878a3c84d15c2b2c31858da2ce0583e2def/nvidia_nvtx-13.0.85-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:cb7780edb6b14107373c835bf8b72e7a178bac7367e23da7acb108f973f157a6", size = 148878, upload-time = "2025-09-04T08:28:53.627Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
|
|
@ -1365,15 +1250,6 @@ wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/6c/73/9f872cb81fc5c3bb48f7227872c28975f998f3e7c2b1c16e95e6432bbb90/python_magic-0.4.27-py2.py3-none-any.whl", hash = "sha256:c212960ad306f700aa0d01e5d7a325d20548ff97eb9920dcd29513174f0294d3", size = 13840, upload-time = "2022-06-07T20:16:57.763Z" },
|
{ url = "https://files.pythonhosted.org/packages/6c/73/9f872cb81fc5c3bb48f7227872c28975f998f3e7c2b1c16e95e6432bbb90/python_magic-0.4.27-py2.py3-none-any.whl", hash = "sha256:c212960ad306f700aa0d01e5d7a325d20548ff97eb9920dcd29513174f0294d3", size = 13840, upload-time = "2022-06-07T20:16:57.763Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "python-multipart"
|
|
||||||
version = "0.0.26"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/88/71/b145a380824a960ebd60e1014256dbb7d2253f2316ff2d73dfd8928ec2c3/python_multipart-0.0.26.tar.gz", hash = "sha256:08fadc45918cd615e26846437f50c5d6d23304da32c341f289a617127b081f17", size = 43501, upload-time = "2026-04-10T14:09:59.473Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/9a/22/f1925cdda983ab66fc8ec6ec8014b959262747e58bdca26a4e3d1da29d56/python_multipart-0.0.26-py3-none-any.whl", hash = "sha256:c0b169f8c4484c13b0dcf2ef0ec3a4adb255c4b7d18d8e420477d2b1dd03f185", size = 28847, upload-time = "2026-04-10T14:09:58.131Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "pyyaml"
|
name = "pyyaml"
|
||||||
version = "6.0.3"
|
version = "6.0.3"
|
||||||
|
|
@ -1654,7 +1530,7 @@ wheels = [
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "surya-ocr"
|
name = "surya-ocr"
|
||||||
version = "0.17.1"
|
version = "0.14.1"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "click" },
|
{ name = "click" },
|
||||||
|
|
@ -1671,21 +1547,21 @@ dependencies = [
|
||||||
{ name = "torch" },
|
{ name = "torch" },
|
||||||
{ name = "transformers" },
|
{ name = "transformers" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/b3/b4/b0f3afd024b4e6fcb4db6fdd17ee963d081e960c907aebd0556ff948abb6/surya_ocr-0.17.1.tar.gz", hash = "sha256:349d78d854c1ed5f816e583545ed6451aa0bc6992e283a805034799aacee8c24", size = 161854, upload-time = "2026-01-30T21:52:59.361Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/11/98/88019a7b7ad86c0ba0c8d19c7a68da637fedaf9e785df1f9746120d583e9/surya_ocr-0.14.1.tar.gz", hash = "sha256:655cf9df26e79901791dd590b2c14dffb2adee461669f2de9ff12adc1d7049c6", size = 147105, upload-time = "2025-05-16T21:52:52.744Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/8d/39/458fb9bc111e123faa2b5a0aade151c663f79dac4ad7e74f9c468b4b7786/surya_ocr-0.17.1-py3-none-any.whl", hash = "sha256:74c331ccb9be2d0c6a774122e572b85abdddf2982ecf6d11c1d83b3a0c9ae19d", size = 189881, upload-time = "2026-01-30T21:53:00.524Z" },
|
{ url = "https://files.pythonhosted.org/packages/bb/e1/f5dc11abf0fbc6f7774f1c4c37b5ee2433d34692c80f55f47281d3265d5a/surya_ocr-0.14.1-py3-none-any.whl", hash = "sha256:e1d5435fc4dc354bf4a21b48e7552c4ed3840da01c89d57fa6c2e2c49005c296", size = 174260, upload-time = "2025-05-16T21:52:51.345Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "sympy"
|
name = "sympy"
|
||||||
version = "1.14.0"
|
version = "1.13.1"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "mpmath" },
|
{ name = "mpmath" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/83/d3/803453b36afefb7c2bb238361cd4ae6125a569b4db67cd9e79846ba2d68c/sympy-1.14.0.tar.gz", hash = "sha256:d3d3fe8df1e5a0b42f0e7bdf50541697dbe7d23746e894990c030e2b05e72517", size = 7793921, upload-time = "2025-04-27T18:05:01.611Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/ca/99/5a5b6f19ff9f083671ddf7b9632028436167cd3d33e11015754e41b249a4/sympy-1.13.1.tar.gz", hash = "sha256:9cebf7e04ff162015ce31c9c6c9144daa34a93bd082f54fd8f12deca4f47515f", size = 7533040, upload-time = "2024-07-19T09:26:51.238Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" },
|
{ url = "https://files.pythonhosted.org/packages/b2/fe/81695a1aa331a842b582453b605175f419fe8540355886031328089d840a/sympy-1.13.1-py3-none-any.whl", hash = "sha256:db36cdc64bf61b9b24578b6f7bab1ecdd2452cf008f34faa33776680c26d66f8", size = 6189177, upload-time = "2024-07-19T09:26:48.863Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
|
|
@ -1716,45 +1592,37 @@ wheels = [
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "torch"
|
name = "torch"
|
||||||
version = "2.11.0"
|
version = "2.6.0+cu124"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://download.pytorch.org/whl/cu124" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "cuda-bindings", marker = "sys_platform == 'linux'" },
|
|
||||||
{ name = "cuda-toolkit", extra = ["cublas", "cudart", "cufft", "cufile", "cupti", "curand", "cusolver", "cusparse", "nvjitlink", "nvrtc", "nvtx"], marker = "sys_platform == 'linux'" },
|
|
||||||
{ name = "filelock" },
|
{ name = "filelock" },
|
||||||
{ name = "fsspec" },
|
{ name = "fsspec" },
|
||||||
{ name = "jinja2" },
|
{ name = "jinja2" },
|
||||||
{ name = "networkx" },
|
{ name = "networkx" },
|
||||||
{ name = "nvidia-cudnn-cu13", marker = "sys_platform == 'linux'" },
|
{ name = "nvidia-cublas-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
{ name = "nvidia-cusparselt-cu13", marker = "sys_platform == 'linux'" },
|
{ name = "nvidia-cuda-cupti-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
{ name = "nvidia-nccl-cu13", marker = "sys_platform == 'linux'" },
|
{ name = "nvidia-cuda-nvrtc-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
{ name = "nvidia-nvshmem-cu13", marker = "sys_platform == 'linux'" },
|
{ name = "nvidia-cuda-runtime-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-cufft-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-curand-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-cusolver-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-cusparse-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-cusparselt-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-nccl-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-nvjitlink-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
|
{ name = "nvidia-nvtx-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
{ name = "setuptools" },
|
{ name = "setuptools" },
|
||||||
{ name = "sympy" },
|
{ name = "sympy" },
|
||||||
{ name = "triton", marker = "sys_platform == 'linux'" },
|
{ name = "triton", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
|
||||||
{ name = "typing-extensions" },
|
{ name = "typing-extensions" },
|
||||||
]
|
]
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/6f/8b/69e3008d78e5cee2b30183340cc425081b78afc5eff3d080daab0adda9aa/torch-2.11.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:4b5866312ee6e52ea625cd211dcb97d6a2cdc1131a5f15cc0d87eec948f6dd34", size = 80606338, upload-time = "2026-03-23T18:11:34.781Z" },
|
{ url = "https://download-r2.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp312-cp312-linux_x86_64.whl", hash = "sha256:a393b506844035c0dac2f30ea8478c343b8e95a429f06f3b3cadfc7f53adb597" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/13/16/42e5915ebe4868caa6bac83a8ed59db57f12e9a61b7d749d584776ed53d5/torch-2.11.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:f99924682ef0aa6a4ab3b1b76f40dc6e273fca09f367d15a524266db100a723f", size = 419731115, upload-time = "2026-03-23T18:11:06.944Z" },
|
{ url = "https://download-r2.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp312-cp312-win_amd64.whl", hash = "sha256:3313061c1fec4c7310cf47944e84513dcd27b6173b72a349bb7ca68d0ee6e9c0" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/1a/c9/82638ef24d7877510f83baf821f5619a61b45568ce21c0a87a91576510aa/torch-2.11.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:0f68f4ac6d95d12e896c3b7a912b5871619542ec54d3649cf48cc1edd4dd2756", size = 530712279, upload-time = "2026-03-23T18:10:31.481Z" },
|
{ url = "https://download-r2.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp313-cp313-linux_x86_64.whl", hash = "sha256:0f3bc53c988ce9568cd876a2a5316761e84a8704135ec8068f5f81b4417979cb" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/1c/ff/6756f1c7ee302f6d202120e0f4f05b432b839908f9071157302cedfc5232/torch-2.11.0-cp312-cp312-win_amd64.whl", hash = "sha256:fbf39280699d1b869f55eac536deceaa1b60bd6788ba74f399cc67e60a5fab10", size = 114556047, upload-time = "2026-03-23T18:10:55.931Z" },
|
{ url = "https://download-r2.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp313-cp313-win_amd64.whl", hash = "sha256:519330eef09534acad8110b6f423d2fe58c1d8e9ada999ed077a637a0021f908" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/87/89/5ea6722763acee56b045435fb84258db7375c48165ec8be7880ab2b281c5/torch-2.11.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:1e6debd97ccd3205bbb37eb806a9d8219e1139d15419982c09e23ef7d4369d18", size = 80606801, upload-time = "2026-03-23T18:10:18.649Z" },
|
{ url = "https://download-r2.pytorch.org/whl/cu124/torch-2.6.0%2Bcu124-cp313-cp313t-linux_x86_64.whl", hash = "sha256:35cba404c0d742406cdcba1609085874bc60facdfbc50e910c47a92405fef44c" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/32/d1/8ed2173589cbfe744ed54e5a73efc107c0085ba5777ee93a5f4c1ab90553/torch-2.11.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:63a68fa59de8f87acc7e85a5478bb2dddbb3392b7593ec3e78827c793c4b73fd", size = 419732382, upload-time = "2026-03-23T18:08:30.835Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/3d/e1/b73f7c575a4b8f87a5928f50a1e35416b5e27295d8be9397d5293e7e8d4c/torch-2.11.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:cc89b9b173d9adfab59fd227f0ab5e5516d9a52b658ae41d64e59d2e55a418db", size = 530711509, upload-time = "2026-03-23T18:08:47.213Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/66/82/3e3fcdd388fbe54e29fd3f991f36846ff4ac90b0d0181e9c8f7236565f82/torch-2.11.0-cp313-cp313-win_amd64.whl", hash = "sha256:4dda3b3f52d121063a731ddb835f010dc137b920d7fec2778e52f60d8e4bf0cd", size = 114555842, upload-time = "2026-03-23T18:09:52.111Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/db/38/8ac78069621b8c2b4979c2f96dc8409ef5e9c4189f6aac629189a78677ca/torch-2.11.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:8b394322f49af4362d4f80e424bcaca7efcd049619af03a4cf4501520bdf0fb4", size = 80959574, upload-time = "2026-03-23T18:10:14.214Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/6d/6c/56bfb37073e7136e6dd86bfc6af7339946dd684e0ecf2155ac0eee687ae1/torch-2.11.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:2658f34ce7e2dabf4ec73b45e2ca68aedad7a5be87ea756ad656eaf32bf1e1ea", size = 419732324, upload-time = "2026-03-23T18:09:36.604Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/07/f4/1b666b6d61d3394cca306ea543ed03a64aad0a201b6cd159f1d41010aeb1/torch-2.11.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:98bb213c3084cfe176302949bdc360074b18a9da7ab59ef2edc9d9f742504778", size = 530596026, upload-time = "2026-03-23T18:09:20.842Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/48/6b/30d1459fa7e4b67e9e3fe1685ca1d8bb4ce7c62ef436c3a615963c6c866c/torch-2.11.0-cp313-cp313t-win_amd64.whl", hash = "sha256:a97b94bbf62992949b4730c6cd2cc9aee7b335921ee8dc207d930f2ed09ae2db", size = 114793702, upload-time = "2026-03-23T18:09:47.304Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/26/0d/8603382f61abd0db35841148ddc1ffd607bf3100b11c6e1dab6d2fc44e72/torch-2.11.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:01018087326984a33b64e04c8cb5c2795f9120e0d775ada1f6638840227b04d7", size = 80573442, upload-time = "2026-03-23T18:09:10.117Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/c7/86/7cd7c66cb9cec6be330fff36db5bd0eef386d80c031b581ec81be1d4b26c/torch-2.11.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:2bb3cc54bd0dea126b0060bb1ec9de0f9c7f7342d93d436646516b0330cd5be7", size = 419749385, upload-time = "2026-03-23T18:07:33.77Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/47/e8/b98ca2d39b2e0e4730c0ee52537e488e7008025bc77ca89552ff91021f7c/torch-2.11.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:4dc8b3809469b6c30b411bb8c4cad3828efd26236153d9beb6a3ec500f211a60", size = 530716756, upload-time = "2026-03-23T18:07:50.02Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/78/88/d4a4cda8362f8a30d1ed428564878c3cafb0d87971fbd3947d4c84552095/torch-2.11.0-cp314-cp314-win_amd64.whl", hash = "sha256:2b4e811728bd0cc58fb2b0948fe939a1ee2bf1422f6025be2fca4c7bd9d79718", size = 114552300, upload-time = "2026-03-23T18:09:05.617Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/bf/46/4419098ed6d801750f26567b478fc185c3432e11e2cad712bc6b4c2ab0d0/torch-2.11.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:8245477871c3700d4370352ffec94b103cfcb737229445cf9946cddb7b2ca7cd", size = 80959460, upload-time = "2026-03-23T18:09:00.818Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/fd/66/54a56a4a6ceaffb567231994a9745821d3af922a854ed33b0b3a278e0a99/torch-2.11.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:ab9a8482f475f9ba20e12db84b0e55e2f58784bdca43a854a6ccd3fd4b9f75e6", size = 419735835, upload-time = "2026-03-23T18:07:18.974Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b1/e7/0b6665f533aa9e337662dc190425abc0af1fe3234088f4454c52393ded61/torch-2.11.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:563ed3d25542d7e7bbc5b235ccfacfeb97fb470c7fee257eae599adb8005c8a2", size = 530613405, upload-time = "2026-03-23T18:08:07.014Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/cf/bf/c8d12a2c86dbfd7f40fb2f56fbf5a505ccf2d9ce131eb559dfc7c51e1a04/torch-2.11.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b2a43985ff5ef6ddd923bbcf99943e5f58059805787c5c9a2622bf05ca2965b0", size = 114792991, upload-time = "2026-03-23T18:08:19.216Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
|
|
@ -1792,19 +1660,11 @@ wheels = [
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "triton"
|
name = "triton"
|
||||||
version = "3.6.0"
|
version = "3.2.0"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
wheels = [
|
wheels = [
|
||||||
{ url = "https://files.pythonhosted.org/packages/17/5d/08201db32823bdf77a0e2b9039540080b2e5c23a20706ddba942924ebcd6/triton-3.6.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:374f52c11a711fd062b4bfbb201fd9ac0a5febd28a96fb41b4a0f51dde3157f4", size = 176128243, upload-time = "2026-01-20T16:16:07.857Z" },
|
{ url = "https://files.pythonhosted.org/packages/06/00/59500052cb1cf8cf5316be93598946bc451f14072c6ff256904428eaf03c/triton-3.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8d9b215efc1c26fa7eefb9a157915c92d52e000d2bf83e5f69704047e63f125c", size = 253159365, upload-time = "2025-01-22T19:13:24.648Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/ab/a8/cdf8b3e4c98132f965f88c2313a4b493266832ad47fb52f23d14d4f86bb5/triton-3.6.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:74caf5e34b66d9f3a429af689c1c7128daba1d8208df60e81106b115c00d6fca", size = 188266850, upload-time = "2026-01-20T16:00:43.041Z" },
|
{ url = "https://files.pythonhosted.org/packages/c7/30/37a3384d1e2e9320331baca41e835e90a3767303642c7a80d4510152cbcf/triton-3.2.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e5dfa23ba84541d7c0a531dfce76d8bcd19159d50a4a8b14ad01e91734a5c1b0", size = 253154278, upload-time = "2025-01-22T19:13:54.221Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/3c/12/34d71b350e89a204c2c7777a9bba0dcf2f19a5bfdd70b57c4dbc5ffd7154/triton-3.6.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:448e02fe6dc898e9e5aa89cf0ee5c371e99df5aa5e8ad976a80b93334f3494fd", size = 176133521, upload-time = "2026-01-20T16:16:13.321Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f9/0b/37d991d8c130ce81a8728ae3c25b6e60935838e9be1b58791f5997b24a54/triton-3.6.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:10c7f76c6e72d2ef08df639e3d0d30729112f47a56b0c81672edc05ee5116ac9", size = 188289450, upload-time = "2026-01-20T16:00:49.136Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/ce/4e/41b0c8033b503fd3cfcd12392cdd256945026a91ff02452bef40ec34bee7/triton-3.6.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1722e172d34e32abc3eb7711d0025bb69d7959ebea84e3b7f7a341cd7ed694d6", size = 176276087, upload-time = "2026-01-20T16:16:18.989Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/35/f8/9c66bfc55361ec6d0e4040a0337fb5924ceb23de4648b8a81ae9d33b2b38/triton-3.6.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d002e07d7180fd65e622134fbd980c9a3d4211fb85224b56a0a0efbd422ab72f", size = 188400296, upload-time = "2026-01-20T16:00:56.042Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/49/55/5ecf0dcaa0f2fbbd4420f7ef227ee3cb172e91e5fede9d0ecaddc43363b4/triton-3.6.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ef5523241e7d1abca00f1d240949eebdd7c673b005edbbce0aca95b8191f1d43", size = 176138577, upload-time = "2026-01-20T16:16:25.426Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/df/3d/9e7eee57b37c80cec63322c0231bb6da3cfe535a91d7a4d64896fcb89357/triton-3.6.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a17a5d5985f0ac494ed8a8e54568f092f7057ef60e1b0fa09d3fd1512064e803", size = 188273063, upload-time = "2026-01-20T16:01:07.278Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/48/db/56ee649cab5eaff4757541325aca81f52d02d4a7cd3506776cad2451e060/triton-3.6.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0b3a97e8ed304dfa9bd23bb41ca04cdf6b2e617d5e782a8653d616037a5d537d", size = 176274804, upload-time = "2026-01-20T16:16:31.528Z" },
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/f6/56/6113c23ff46c00aae423333eb58b3e60bdfe9179d542781955a5e1514cb3/triton-3.6.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:46bd1c1af4b6704e554cad2eeb3b0a6513a980d470ccfa63189737340c7746a7", size = 188397994, upload-time = "2026-01-20T16:01:14.236Z" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue