JiriOndrusek opened a new pull request, #9008: URL: https://github.com/apache/camel-quarkus/pull/9008
based on https://github.com/apache/camel-quarkus/pull/9006 Second groundwork PR toward declarative AI document ingestion, building directly on the `IngestionTracker` from #9003: the tracker records what was ingested — this adds the engine that acts on it, as Camel-free support code in `org.apache.camel.quarkus.component.support.langchain4j.ingest`. - `IngestService` — the write path for one document: two-tier change detection (cheap source fingerprint, then content hash — both tenant-aware, so re-assigning a document to another tenant is never skipped with stale metadata), deterministic UUIDv5 segment ids, and the crash-safe `writeIntent` → store write → shrink-remove → `commit` protocol. Failures are typed by where they happen: content-attributable failures (fetch, split) are dead-lettered and retried only when the content changes, while infrastructure failures (embedding provider, store, tracker I/O) leave the row `in_progress` and are retried on the next pass — a transient outage can never freeze or drop a document. Includes a per-store write strategy (`upsert` vs `remove-then-add`, from measured store behaviour), a startup capability probe (a store without id-addressed removal fails fast with its name instead of dead-lettering the corpus), model-scoped embedding batching/rate limiting, and per-document wri te serialization. - `SyncPassRunner` — one bounded synchronisation pass ending in tracker-versus-source reconciliation (the store is never enumerated), behind the safety interlock: a pass with any failure deletes nothing, a bulk-delete floor (measured against the pre-pass corpus) refuses mass deletion without explicit consent, and only source-origin documents are candidates. - `IngestIds` — deterministic RFC 4122 v5 segment ids over a length-prefixed name (two pipelines sharing a store cannot collide); golden-value tests freeze the derivation, since these ids live in users' vector stores permanently. - `IngestResult` — the per-document outcome. The module README documents the full document-state machine and the write-path decision diagram. Invariants are pinned by 40 new unit tests, including transient-outage recovery for both write strategies and crash-convergence scenarios. This work is not plugged into any existing code: no current extension or code path changes behaviour, the module is purely additive and is the second piece of groundwork for the upcoming declarative `langchain4j-ingest` extension (`sync` mode). Like #9003, it is **experimental, internal SPI** with no compatibility promise — together with the tracker it forms the unit that could be replaced by (or proposed to) an upstream LangChain4j record-manager mechanism (langchain4j/langchain4j#2931). Dependency note: `java-uuid-generator` (UUIDv5; the JDK only offers v3) is `<optional>` in the support module, so no existing extension inherits it — the future `langchain4j-ingest` extension declares it explicitly. Depends on #9006 — this branch stacks on it and should be rebased/merged after it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- Uncomment and fill this section if your PR is not trivial [ ] An issue should be filed for the change unless this is a trivial change (fixing a typo or similar). One issue should ideally be fixed by not more than one commit and the other way round, each commit should fix just one issue, without pulling in other changes. [ ] Each commit in the pull request should have a meaningful and properly spelled subject line and body. Copying the title of the associated issue is typically enough. Please include the issue number in the commit message prefixed by #. [ ] The pull request description should explain what the pull request does, how, and why. If the info is available in the associated issue or some other external document, a link is enough. [ ] Phrases like Fix #<issueNumber> or Fixes #<issueNumber> will auto-close the named issue upon merging the pull request. Using them is typically a good idea. [ ] Please run mvn process-resources -Pformat (and amend the changes if necessary) before sending the pull request. [ ] Contributor guide is your good friend: https://camel.apache.org/camel-quarkus/latest/contributor-guide.html --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
