This is an automated email from the ASF dual-hosted git repository. tballison pushed a commit to branch TIKA-4809-stage-2 in repository https://gitbox.apache.org/repos/asf/tika.git
commit 24a31879201fb37e5b24bbb7f8b747a0eb04b951 Author: tallison <[email protected]> AuthorDate: Fri Aug 7 11:15:08 2026 -0400 TIKA-4809: Correct docs -- 3 independent pipes groups, not 2 --- docs/modules/ROOT/pages/pipes/cpu-sizing.adoc | 33 ++++++++++++---------- .../ROOT/pages/using-tika/server/index.adoc | 27 +++++++++++------- 2 files changed, 35 insertions(+), 25 deletions(-) diff --git a/docs/modules/ROOT/pages/pipes/cpu-sizing.adoc b/docs/modules/ROOT/pages/pipes/cpu-sizing.adoc index 7c1d066d8d..bd23334323 100644 --- a/docs/modules/ROOT/pages/pipes/cpu-sizing.adoc +++ b/docs/modules/ROOT/pages/pipes/cpu-sizing.adoc @@ -119,17 +119,20 @@ Everything above describes sizing for *one* `PipesParser` — one `pipes` config section, one set of forked workers. The auto-sizer has no visibility into anything else running in the same JVM. -This matters concretely for tika-server: `/tika`+`/rmeta`+`/unpack` and -`/pipes`+`/async` are backed by two *independent* `PipesParser` groups when -both are enabled in the same server. Each group's auto-sizer computes its -slice from `Runtime.availableProcessors()` as if it were the only consumer on -the host — it does not know a sibling group in the same process is about to -fork its own `numClients` workers too. The result: with `numClients=2` on -both, you get 4 total forked JVMs, each capped assuming exclusive access to -the whole host. Whether that's *actually* oversubscribed depends on your -host's real core count relative to those combined `numClients` values — it's -not automatic, but the auto-sizer also won't warn you, because each group -looks correctly sized from its own perspective alone. See +This matters concretely for tika-server: `/tika`+`/rmeta`+`/unpack`, `/pipes`, +and `/async` are backed by *three independent* groups when all are enabled in +the same server — `/tika`+`/rmeta`+`/unpack` and `/pipes` are each their own +`PipesParser` instance, and `/async` manages its own forked-worker pool +directly (not via `PipesParser` at all, though it uses the same underlying +auto-sizer). Each group's auto-sizer computes its slice from +`Runtime.availableProcessors()` as if it were the only consumer on the host — +it does not know sibling groups in the same process are about to fork their +own `numClients` workers too. The result: with `numClients=2` on all three, +you get 6 total forked JVMs, each capped assuming exclusive access to the +whole host. Whether that's *actually* oversubscribed depends on your host's +real core count relative to those combined `numClients` values — it's not +automatic, but the auto-sizer also won't warn you, because each group looks +correctly sized from its own perspective alone. See xref:using-tika/server/index.adoc#_endpoints_and_forked_process_groups[Endpoints and Forked-Process Groups] for the tika-server-specific guidance. @@ -138,10 +141,10 @@ directly and constructing more than one instance in a single JVM — the auto-sizer will size each independently, with the same caveat. There is no automatic fix for this today: unlike the single-group case, where -Tika detects and warns about bad provisioning, a *second* group has no way to -learn what a sibling group already claimed. Mitigate it explicitly — either -run only one group per process, or set `-XX:ActiveProcessorCount` yourself -(next section) with the combined total in mind. +Tika detects and warns about bad provisioning, each group has no way to learn +what its siblings already claimed. Mitigate it explicitly — either run fewer +groups per process, or set `-XX:ActiveProcessorCount` yourself (next section) +with the combined total in mind. == Disabling or overriding diff --git a/docs/modules/ROOT/pages/using-tika/server/index.adoc b/docs/modules/ROOT/pages/using-tika/server/index.adoc index b04868f137..241ca60680 100644 --- a/docs/modules/ROOT/pages/using-tika/server/index.adoc +++ b/docs/modules/ROOT/pages/using-tika/server/index.adoc @@ -298,14 +298,20 @@ xref:migration-to-4x/migrating-tika-server-4x.adoc#_configuration_changes[Config [#_endpoints_and_forked_process_groups] == Endpoints and Forked-Process Groups -Two independent pipes-backed process groups exist, plus one endpoint that +Three independent pipes-backed process groups exist, plus one endpoint that isn't pipes-backed at all: * **`/tika` + `/rmeta` + `/unpack`** share one group — all three go through the same `PipesParsingHelper`/`PipesParser`, sized by `pipes.numClients`. -* **`/pipes` + `/async`** share a separate group (gated behind `allowPipes`), -sized by the same `pipes.numClients` setting in the same config, but as an -independent set of forked processes. +* **`/pipes`** is its own separate group (gated behind `allowPipes`) — it +builds its own independent `PipesParser`, not shared with `/tika`/`/rmeta`/ +`/unpack`, sized by the same `pipes.numClients` setting in the same config +but as a wholly separate set of forked processes. +* **`/async`** is a *third*, independent group (also gated behind +`allowPipes`) — it doesn't share a `PipesParser` with `/pipes` either. It +manages its own forked-worker pool directly (queued/background processing, +results delivered via a configured `PipesReporter` rather than in the HTTP +response), sized by its own read of `pipes.numClients` from the same config. * **`/meta` is not pipes-backed** — it still parses in-process, in the request-handling JVM, as in 3.x. It isn't bound by `numClients` and doesn't participate in anything below, but it also has no crash/OOM isolation: a @@ -341,13 +347,14 @@ Independently of the above, each group also auto-sizes its forked JVMs' xref:pipes/cpu-sizing.adoc[Forked-JVM CPU Sizing] for the full mechanics. This part *can* go wrong across groups: the auto-sizer for one group has no visibility into another group running in the same process, so if you enable -both `/tika`/`/rmeta`/`/unpack` *and* `/pipes`/`/async` together — a config +`/tika`/`/rmeta`/`/unpack` together with `/pipes` and/or `/async` — a config listing all of them, or simply leaving `endpoints` unset while -`allowPipes=true` — each group's auto-sizer computes its slice as if it owned -the whole host. Whether that actually causes oversubscription depends on your -`numClients` values relative to the host's core count; it's not automatic, but -it's also not something the auto-sizer will warn you about, because from -either group's perspective alone the sizing looks fine. See +`allowPipes=true` gives you all three groups at once — each group's auto-sizer +computes its slice as if it owned the whole host. Whether that actually causes +oversubscription depends on your `numClients` values relative to the host's +core count; it's not automatic, but it's also not something the auto-sizer +will warn you about, because from any one group's perspective alone the sizing +looks fine. See xref:pipes/cpu-sizing.adoc#_known_limitation_multiple_pipes_groups_in_one_process[Known limitation: multiple Pipes groups in one process] for the mechanics and mitigation (scope `endpoints` to what you actually use, or set
