feat(provenance): mapper + verifier (spec §9.4, §6) #8

Merged

goldstein merged 1 commit from feat/provenance-mapper-verifier into main

2026-04-18 09:01:35 +00:00

goldstein commented

2026-04-18 09:01:29 +00:00

Owner

Completes the provenance subsystem.

mapper.py — map_segment_refs_to_provenance resolves LLM SegmentCitations to FieldProvenance entries, honours max_sources_per_field, counts invalid references, and resolves field values via dot-path (items[0].name → items.0.name).

verify.py — verify_field + apply_reliability_flags dispatch by Pydantic field type and write the provenance_verified / text_agreement flags in place. Handles PEP 604 X | None unions correctly (types.UnionType on 3.12+), scans numeric tokens so labels do not mask balance matches, recognises IBANs by field name.

CI trigger is flaky for now; local tests green (103 passed, ruff clean).

Completes the provenance subsystem. `mapper.py` — `map_segment_refs_to_provenance` resolves LLM SegmentCitations to `FieldProvenance` entries, honours `max_sources_per_field`, counts invalid references, and resolves field values via dot-path (`items[0].name` → `items.0.name`). `verify.py` — `verify_field` + `apply_reliability_flags` dispatch by Pydantic field type and write the `provenance_verified` / `text_agreement` flags in place. Handles PEP 604 `X | None` unions correctly (types.UnionType on 3.12+), scans numeric tokens so labels do not mask balance matches, recognises IBANs by field name. CI trigger is flaky for now; local tests green (103 passed, ruff clean).

goldstein added 1 commit 2026-04-18 09:01:29 +00:00

feat(provenance): mapper + verifier for ReliabilityStep (spec §9.4, §6)

tests / test (pull_request) Successful in 1m10s

Details

tests / test (push) Successful in 1m11s

Details

1e340c82fa

Lands the two remaining provenance-subsystem pieces:

mapper.py — map_segment_refs_to_provenance:
- For each LLM SegmentCitation, pick seg-ids per source_type
  (`value` vs `value_and_context`), cap at max_sources_per_field,
  resolve each via SegmentIndex, track invalid references.
- Resolve field values by dot-path (`result.items[0].name` supported —
  `[N]` bracket notation is normalised to `.N` before traversal).
- Skip fields that resolve to zero valid sources (spec §9.4).
- Write quality_metrics with fields_with_provenance / total_fields /
  coverage_rate / invalid_references.

verify.py — verify_field + apply_reliability_flags:
- Dispatches per Pydantic field type: date → parse-both-sides compare;
  int/float/Decimal → normalize + whole-snippet / numeric-token scan;
  IBAN (detected via `iban` in field name) → upper+strip compare;
  Literal / None → flags stay None; else string substring.
- _unwrap_optional handles BOTH typing.Union AND types.UnionType so
  `Decimal | None` (PEP 604, what get_type_hints emits on 3.12+) resolves
  correctly — caught by the integration-style test_writes_flags_and_counters.
- Number comparator scans numeric tokens in the snippet so labels
  ("Closing balance CHF 1'234.56") don't mask the match.
- apply_reliability_flags mutates the passed ProvenanceData in place and
  writes verified_fields / text_agreement_fields to quality_metrics.

Tests cover each comparator, Literal/None skip, short-value skip (strings
and numerics), Decimal via optional union, and end-to-end flag+counter
writing against a Pydantic use-case schema that mirrors bank_statement_header.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

goldstein merged commit b397a80c0b into main

2026-04-18 09:01:35 +00:00

goldstein referenced this pull request from a commit

2026-04-18 09:01:36 +00:00

feat(provenance): mapper + verifier (spec §9.4, §6) (#8)

No reviewers

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: goldstein/infoxtractor#8

No description provided.