davsclaus opened a new pull request, #3230:
URL: https://github.com/apache/camel-kamelets/pull/3230
Part of #3170: `tika-extract-text-action` and `docling-convert-action` can
give HTML or other formats instead of plain text or Markdown;
`docling-convert-action` can log in to a Docling Serve server with an API key,
and takes OCR options and timeouts; `pqc-signature-action` and `pqc-kem-action`
can use a key pair from a key store, so that a signature or an encapsulation
still works after a restart; `langchain4j-ingest-file-source` takes
`initialDelay`, `maxMessagesPerPoll` and `readLockMinAge`.
### The options
| Kamelet | Option | Template mapping | What it does |
|---|---|---|---|
| `tika-extract-text-action` | `outputFormat` (`text`, `html`, `xml`) |
`tikaParseOutputFormat: "{{outputFormat}}"`, `default: text` | `html` gives an
XHTML page with the metadata as `meta` elements; `xml` the same with an XML
declaration. |
| `docling-convert-action` | `operation` (`CONVERT_TO_MARKDOWN`,
`CONVERT_TO_HTML`, `EXTRACT_TEXT`) | `operation: "{{operation}}"`, `default:
CONVERT_TO_MARKDOWN` | What the document is turned into. |
| | `enableOcr` | `enableOCR: "{{?enableOcr}}"` and `doOcr:
"{{?enableOcr}}"` | false: no OCR; true: OCR in `ocrLanguage`. |
| | `ocrLanguage` | `ocrLanguage: "{{?ocrLanguage}}"`, `pattern:
"[A-Za-z_:-]+"` | The language of the text in images. |
| | `forceOcr` | `forceOcr: "{{?forceOcr}}"` | OCR every page, replacing its
text layer. |
| | `ocrEngine` (`AUTO`, `EASYOCR`, `RAPIDOCR`, `TESSERACT`, `TESSEROCR`,
`OCRMAC`) | `ocrEngine: "{{?ocrEngine}}"` | The OCR engine of the server. |
| | `processTimeout` | `processTimeout: "{{?processTimeout}}"` | How long to
wait for a conversion, in ms (30000 when unset). |
| | `documentTimeout` | `documentTimeout: "{{?documentTimeout}}"` | How long
the server may work on a document, in seconds. |
| | `authenticationScheme` (`API_KEY`, `BEARER`) | `authenticationScheme:
"{{?authenticationScheme}}"` | `X-Api-Key` header, or `Authorization: Bearer`. |
| | `authenticationToken` (secret) | `authenticationToken:
"RAW({{?authenticationToken}})"` | The API key or token. |
| `pqc-signature-action`, `pqc-kem-action` | `keyStore` | `keyStore:
"{{?keyStore}}"` | A Java `KeyStore` bean, as `#bean:name`. |
| | `keyPairAlias` | `keyPairAlias: "{{?keyPairAlias}}"` | The alias of the
key pair in it. |
| | `keyStorePassword` (secret) | `keyStorePassword:
"RAW({{?keyStorePassword}})"` | The password of the key store and of the key
pair. |
| | `keyPair` | `keyPair: "{{?keyPair}}"` | A Java `KeyPair` bean, as
`#bean:name`, instead of a key store. |
| `pqc-kem-action` | `symmetricKeyLength` | `symmetricKeyLength:
"{{?symmetricKeyLength}}"` | The length of the secret key in bits (256 when
unset); 128 makes `SEED` and `SM4` usable. |
| `langchain4j-ingest-file-source` | `initialDelay` | `initialDelay:
"{{?initialDelay}}"` | Wait before the first look at the directory, in ms. |
| | `maxMessagesPerPoll` | `maxMessagesPerPoll: "{{?maxMessagesPerPoll}}"` |
The most files read per look. |
| | `readLockMinAge` | `readLockMinAge: "{{?readLockMinAge}}"` | Read a file
only when it last changed at least this long ago. |
- **Defaults:** every new option is optional. Only `outputFormat` and
`operation` have a `default`, set to the value the template had hard-coded, so
that the endpoint URI stays the same. All others have no `default`, as in
#3174: unset, the optional placeholders drop them. `camel get endpoint` shows
the same five endpoint URIs on `main` and on this branch with the options unset.
- **Docling `operation` instead of `outputFormat`:** the component's
`outputFormat` applies only to `SUBMIT_ASYNC_CONVERSION`; the format of a
normal conversion comes from the operation. `CONVERT_TO_JSON` and
`EXTRACT_STRUCTURED_DATA` are left out: they give a Java `DoclingDocument`, not
text.
- **Docling `enableOcr`:** one option maps to both `enableOCR`, which only
the `docling` command line reads, and `doOcr`, which only Docling Serve reads.
With Docling Serve, the component sends `ocrLanguage` only when `doOcr` is
true, so `ocrLanguage` says it needs `enableOcr: true` there.
- **PQC key store:** `keyStore` and `keyPair` are objects, so they need a
bean. A key store can be made in YAML with a one-line Groovy `script` bean
(`KeyStore.getInstance(new File(...), password)`), which the partial pages
show; no Java class. A bean-free option, such as a key store file and type,
would need a change in camel-pqc. `keyPairAlias` and `keyStorePassword` alone
do nothing without a key store.
- **PQC autowiring:** camel-pqc autowires `keyStore` and `keyPair`. When the
application has exactly one `KeyStore`, every PQC endpoint uses it, even
without `keyStore`; without an alias it then has no key pair, and every message
fails with a `NullPointerException`. This happens on `main` too. The
descriptions now say so, and `keyPairAlias` with `keyStorePassword` is the fix.
- **Not changed:** `langchain4j-ingest-sink` already exposes every option of
the `langchain4j-ingest` endpoint, so it needed nothing.
- **Descriptions:** the summaries, `Output:` and `Good to know` paragraphs
of the changed Kamelets mention the new choices. Docling's "Good to know" says
that the 30-second default timeout is too short for the first conversion of a
Docling Serve server on CPU. Tika's `outputFormat` says that `html` and `xml`
fail on a document whose metadata holds a character that XML does not allow,
such as a NUL, while `text` reads it. This happened with 3 of the 4 test PDFs.
- **Partial pages:** new for all five Kamelets, each with an example that
was run.
### Verification
Tested with Camel JBang 4.24.0-SNAPSHOT on Java 25 (arm64 Mac). Docling
against `quay.io/docling-project/docling-serve:v1.38.2` (arm64), started with
`DOCLING_SERVE_API_KEY`. Tika in-process with `tika-parser-pdf-module:4.1.0`.
PQC with a PKCS12 key store made by `keytool` (ML-DSA-65 and ML-KEM-768), and
checked against the JDK's own ML-DSA. LangChain4j file source against a local
directory, with a log step as the sink, so no embedding model was needed.
| Kamelet | Case | Result |
|---|---|---|
| all five | Options unset, `main` and this branch | `camel get endpoint`
shows identical `tika:`, `docling:`, `pqc:` (both) and `file:` URIs |
| all five | All new options set | Each one appears on the endpoint URI; the
secrets as `RAW(...)` |
| tika | `outputFormat` unset / `text`, PDF and HTML | Same plain text as
`main` |
| tika | `html`, invoice PDF | XHTML: `head` with `meta` elements
(`dc:title`, `pdf:producer`...), `body` with a `div class="page"` of `p` |
| tika | `html`, HTML page | `h1`, `h2`, `ul`/`li`, `table`/`tr`/`th` |
| tika | `xml` | The same page with `<?xml version="1.0" encoding="UTF-8"?>`
on top |
| tika | `html` / `xml`, two `cupsfilter` PDFs and the docling test
`multi_page.pdf` | Fails: `An invalid XML character (Unicode: 0x0)` while
writing the `head`; `text` reads them |
| docling | No login, server with an API key | `{"detail":"Unauthorized"}` |
| docling | `API_KEY` + right key | Converted (Markdown with tables) |
| docling | `API_KEY` + wrong key; token without scheme; `BEARER` + right
key | `Unauthorized` |
| docling | Against a header-echo server, token `k+y&z=1%2 #x` | `API_KEY`
sends `X-Api-Key: k+y&z=1%2 #x`; `BEARER` sends `Authorization: Bearer
k+y&z=1%2 #x`; unset sends neither |
| docling | `processTimeout` unset, first conversion after the server
started | Timed out after 30000 ms; the same document with `processTimeout:
300000` took 56 s and converted |
| docling | `processTimeout: 500` | Fails: `request timed out` after 506 ms |
| docling | `operation: CONVERT_TO_HTML` | A full HTML page, `<!DOCTYPE
html>` with styles |
| docling | `operation: EXTRACT_TEXT` | Text without `##` heading marks or
`<!-- image -->`; tables still drawn with pipes |
| docling | Scanned PDF (text only in an image), OCR options unset |
`Scanned invoice 4711 / Total due: 99 EUR` |
| docling | Scanned PDF, `enableOcr: false` | Empty body |
| docling | `enableOcr: true`, `ocrLanguage: de` | Text read |
| docling | `enableOcr: true`, `ocrLanguage: xx` | Fails; the server log
says no engine has a model for `xx` |
| docling | `ocrLanguage: xx` without `enableOcr` | Ignored, converted |
| docling | `ocrEngine: OCRMAC` | Fails; server: `OcrMac is only supported
on Mac` |
| docling | `ocrEngine: TESSERACT`, no language / `en` | Fails; server: no
Tesseract model for `spa` / `en` |
| docling | `ocrEngine: TESSERACT`, `enableOcr: true`, `ocrLanguage: eng` or
`iso:en` | Text read |
| docling | `forceOcr: true`, digital invoice PDF | Text from OCR: `#36258`
and `Order ID:` where the text layer has `# 36258` and `Order ID :` |
| docling | `documentTimeout: 1`, 9430-character PDF | No failure; the body
holds 1519 characters, a part of the document |
| docling | Partial example (API key, OCR in German, a directory with a
subdirectory) | Markdown files written, the scan's text read |
| pqc-signature | `sign` with key store + alias + password | The JDK's
`Signature("ML-DSA")` verifies the signature with the certificate's public key
from `pqc.p12`: true |
| pqc-signature | New JVM, `verify` with the key store | true; changed body:
false |
| pqc-signature | `keyPair: "#bean:..."` (Groovy bean from the key store) |
Signed; JDK verify true |
| pqc-signature | One `KeyPair` bean, `keyPair` unset | Autowired: signed
with it, JDK verify true |
| pqc-signature | One `KeyStore` bean, no alias, `main` and this branch |
Both fail: `NullPointerException ... this.keyPair is null` |
| pqc-signature | No key store anywhere, sign then verify | true, as on
`main` |
| pqc-kem | `generateSecretKeyEncapsulation` then
`extractSecretKeyEncapsulation`, key store `kem`, `symmetricKeyLength: 128` |
16-byte AES key; the extracted key equals the generated one |
| pqc-kem | New JVM: the saved encapsulation, opened with only
`keyPairAlias` + `keyStorePassword` (the key store autowired) | The same key as
in the first JVM |
| pqc-kem | New JVM, no key store | Fails: `encapsulation wrong length`
(another key pair) |
| pqc-kem | `SEED` with `symmetricKeyLength: 128` | A 16-byte `SEED` key |
| pqc-kem | `SEED` with 256, or unset | Does not start: `Invalid symmetric
key length 256 for algorithm SEED. Valid key lengths: [128]` |
| pqc | Partial examples, both Kamelets | Run as written |
| file source | Options unset, 5 old files | One file a second (the
`changed` read lock waits 1 s per file); first read 1.9 s after start |
| file source | `initialDelay: 6000` | First read 6.9 s after start |
| file source | `maxMessagesPerPoll: 2`, `delay: 4000` | Files read 2, then
2, then 1, the groups about 6 s apart (4 s delay plus the 1 s read lock per
file) |
| file source | `readLockMinAge: 3000` | The 5 old files read within 6 ms; a
new file read 3.4 s after it was written |
| file source | `readLockMinAge: 15000`, longer than the 10 s read lock
timeout | The new file read 16 s after it was written, with no warning in the
log |
Not tested: the `docling` command line (not installed), so `enableOcr` and
`processTimeout` on that path are checked only by reading the component source.
Also not tested: the `langchain4j-ingest-file-source` partial example with a
real embedding model; its options were tested with a log step.
`./mvnw clean verify -DskipTests` and then `./mvnw clean verify` passed from
the root. Before building, the library copies of the five Kamelets were
refreshed with `./mvnw -pl library/camel-kamelets
resources:copy-resources@copy-resource-one`. The build regenerated
`kamelets-index.json`. `KameletsCatalogTest` counts no option of these Kamelets.
_Claude Code on behalf of Claus Ibsen_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]