zbendhiba opened a new pull request, #9374:
URL: https://github.com/apache/camel-quarkus/pull/9374
…assistants
Fixes #9370. Follow-up to #8706 / #9336, which shipped the built-in skills
and the Dev MCP tools.
### The problem
An assistant asked for a Camel Quarkus route answers from training data. It
guesses endpoint options, invents configuration properties, and has no idea
which version the project uses. We already ship a skill that teaches the right
patterns, but the 334 extension reference pages, with their configuration
tables and their Dev Services behaviour, were not reachable.
The Quarkus Agent MCP server already serves the core Quarkus guides through
`quarkus_searchDocs`. This PR puts our pages in there too.
### What it does
```
docs/modules/ROOT/pages/reference/extensions/*.adoc (334 generated pages)
|
| quarkus-documentation-rag-maven-plugin, run once
| by the new documentation-rag module
v
camel-quarkus-documentation-rag.jar (1000 chunks, 5.68 MB
SQL)
^
| META-INF/quarkus-rag-artifact.properties
|
camel-quarkus-*-deployment.jar --> Quarkus Agent MCP server
```
Each deployment JAR carries a 2 line pointer naming the artifact. The server
reads it, resolves that artifact at the version of the extension that carried
the pointer, downloads it once, and loads the embeddings. No coordinate of ours
is compiled into the server, and users declare nothing.
### Why one shared artifact
The plugin supports three mechanisms. Only the third fits a repository with
334 extensions.
| Mechanism | Verdict |
|---|---|
| Inline `META-INF/quarkus-rag.sql` in every deployment JAR | Works.
Implemented and measured first. 271 SQL files, 4.8 MB spread over the extension
JARs, and +29.3 s on the full reactor. |
| Central aggregated artifact | Impossible. The server resolves aggregates
only under the hardcoded coordinate
`io.quarkus:quarkus-documentation-core-rag`. |
| Pointer to a separate artifact | Chosen. One artifact, one generation,
nothing added to the extension JARs beyond the pointer. |
How upstream does it, for context:
- **Quarkiverse** (`quarkiverse-parent` PR #235) declares the plugin in the
parent and has each deployment pom opt in with `<quarkus-rag.guide>`, behind an
opt-in `-Drag` profile.
- **Quarkus core** uses a central `devtools/extension-rag` module gated on
the `release` profile, publishing the aggregated artifact.
We are closer to the Quarkus core shape, one module producing one artifact,
but with a pointer instead of the hardcoded coordinate we cannot use.
### Build cost
Generation runs in one module instead of 334. That module goes from 9.3 s to
24.8 s, so +15.5 s on a full build. It is skipped by `-Dquickly`, like the rest
of the documentation tooling.
### The `:extensions:` attribute
`update-extension-doc-page` now writes one line per page:
```asciidoc
:extensions: org.apache.camel.quarkus:camel-quarkus-kafka
```
It is in the FreeMarker template, so a new extension gets it for free, and
the 334 page changes in this PR are regenerated output.
It is required twice over, and the second reason is the decisive one:
1. Without it, the plugin names the source after the file name, which yields
`quarkus-core`, `quarkus-jolokia` and other real core Quarkus source names. The
server deletes a source before reloading it, so we would wipe genuine Quarkus
documentation.
2. The server rewrites each row's source to the artifactId of the dependency
that pointed at the artifact, unless the row declares its own extension in the
same groupId. That declaration is what the attribute produces. Without it, all
1000 chunks collapse onto a single source.
This is also the upstream convention, not a workaround of ours. Both forms
coexist in `quarkus-documentation-core-rag`: `kafka-getting-started.adoc` is
published as `quarkus-messaging-kafka` from the attribute, `all-config.adoc` as
`quarkus-all-config` from the file name.
### Minimum server version
`quarkus-agent-mcp` 1.2.8 or newer. Pointer support starts at 1.1.5, but
versions below 1.2.8 track a single source per artifact and purge with a
hardcoded `DELETE ... = 'quarkus-documentation'` that matches nothing, so the
shared artifact would duplicate every chunk on reload. 1.2.8 added
`extractSources` and `DELETE ... = ANY(?)`.
Documented directly, since the agent side is not released yet.
### What I verified
- 334/334 pages carry the attribute. The artifact has 334 distinct
`camel-quarkus-*` sources and zero `quarkus-*`.
- Deployment JARs carry the pointer only, no inline SQL. `-Dquickly` skips
generation.
- End to end against the real 1.2.12 `RagSqlLoader`: 122 transitive
dependencies resolved, pointer read from `camel-quarkus-kafka-deployment`,
artifact resolved in 35 ms, 334 sources reported. Deduplication means it is
resolved once per project, not 334 times.
- No BOM entry is needed, and the module has to sit in the root `<modules>`
rather than the `standard-build` profile, which is disabled during a release.
### One bug found along the way, not a blocker here
End to end search kept returning nothing of ours, while an exact vector
search ranked our chunks first (kafka 0.9041, jms 0.8749, ftp 0.8757). The
cause is on the server side: the `ivfflat` index is queried with the pgvector
default `ivfflat.probes = 1`, which searches one cluster.
| probes | recall@10 | latency |
|---|---|---|
| 1 | 54.4% | 1 ms |
| 10 | 87.5% | 2 ms |
| 50 | 100% | ~5 ms |
Measured over 16 queries. One core Quarkus query ("configure logging
categories and levels") scored 0/10. This affects core Quarkus documentation as
much as ours, and is still present in 1.2.12. To be reported upstream, with
these numbers.
### Documentation
- New contributor guide page, `contributor-guide/documentation-rag.adoc`:
the pipeline, source naming, the module, the pointer, a walkthrough of an
assistant adding Camel to a project that has none, how the server finds and
downloads the artifact, local build and inspection commands, and the known
limitations.
- `user-guide/quarkus-agent-mcp.adoc`: what users get and the minimum
version.
### Open for discussion
- **Gating.** Generation runs on `!quickly`, so a normal build produces it.
Both upstream projects gate it off by default instead. Should we follow them
and only build it on release?
- **Upstream.** Worth proposing a `sourceNamePrefix` parameter on the
plugin, so the file name fallback yields `camel-quarkus-<file>` instead of
`quarkus-<file>`. Three lines in `RagPipeline.resolveSourceName`. We do not
need it, but it removes a collision trap for the next project in our position.
- **Scope.** The whole reference corpus is loaded as soon as one Camel
Quarkus extension is on the class path. Per extension artifacts would scope it
tighter, at the price of 334 artifacts. Worth revisiting once recall is fixed.
- Other approaches welcome. I am not attached to this one, it is the one
that works with the server as it is today.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
