This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tika.git
The following commit(s) were added to refs/heads/main by this push:
new 265c4a4199 TIKA-4984 (#3415)
265c4a4199 is described below
commit 265c4a4199a682fc6d2169eab98646a9151ff107
Author: Tim Allison <[email protected]>
AuthorDate: Sat Oct 24 13:00:02 2026 -0400
TIKA-4984 (#3415)
* TIKA-4984: tell tika-app users to drop plugin-roots from the docs example
configs
* TIKA-4984: treat the docs' PLUGINS_PATHS placeholder as an unset
plugin-roots
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
CHANGES.txt | 6 +++++-
docs/modules/ROOT/pages/pipes/configuration.adoc | 2 +-
docs/modules/ROOT/pages/pipes/getting-started.adoc | 8 +++++++-
docs/modules/ROOT/pages/pipes/plugins/filesystem.adoc | 2 +-
.../ROOT/pages/pipes/plugins/writing-a-plugin.adoc | 5 +++++
.../java/org/apache/tika/async/cli/PluginsWriter.java | 3 +++
.../java/org/apache/tika/async/cli/TikaAsyncCLI.java | 2 +-
.../tika/async/cli/TikaConfigAsyncWriterTest.java | 17 +++++++++++++++++
.../apache/tika/pipes/core/config/ConfigMerger.java | 2 +-
.../tika/pipes/core/config/DefaultPluginsDir.java | 15 +++++++++++++++
.../tika/pipes/core/config/ConfigMergerTest.java | 18 ++++++++++++++++++
.../tika/pipes/core/config/DefaultPluginsDirTest.java | 13 +++++++++++++
12 files changed, 87 insertions(+), 6 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index f16a7204f8..3ebd06513b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -77,7 +77,11 @@ Release 4.2.0 - 10/23/2026
* Pipes plugins load only from the zips in plugin-roots; a plugin on the
classpath
needs -Dtika.plugins.classpath=true, which PipesForkParser sets when it
finds no
- plugins directory holding tika-pipes-file-system (TIKA-4918, TIKA-4985).
+ plugins directory holding tika-pipes-file-system (TIKA-4918, TIKA-4985). A
+ plugin-roots naming a directory without the plugin zips, which the
classpath used
+ to mask, now fails with "Unknown type: ... Available: []"; tika-app and
+ tika-server treat the docs' PLUGINS_PATHS placeholder as unset and use
their
+ bundled plugins directory (TIKA-4984).
* image/vnd.microsoft.icon moves from ImageParser to the new ICOParser,
which also
parses cursors (TIKA-4937); animated cursors (RIFF ACON) are
diff --git a/docs/modules/ROOT/pages/pipes/configuration.adoc
b/docs/modules/ROOT/pages/pipes/configuration.adoc
index 832565aa9d..c16926767b 100644
--- a/docs/modules/ROOT/pages/pipes/configuration.adoc
+++ b/docs/modules/ROOT/pages/pipes/configuration.adoc
@@ -293,7 +293,7 @@ include::example$pipes-fs-pipeline.json[]
icon:github[]
https://github.com/apache/tika/blob/main/tika-pipes/tika-pipes-integration-tests/src/test/resources/configs/tika-config-basic.json[View
source on GitHub]
-Tokens (`FETCHER_BASE_PATH`, `EMITTER_BASE_PATH`, `PLUGINS_PATHS`,
`EMIT_INTERMEDIATE_RESULTS`) are substituted by the test harness — replace them
with real values in production configs. The first three are paths;
`EMIT_INTERMEDIATE_RESULTS` is the boolean `emitIntermediateResults` flag.
+Tokens (`FETCHER_BASE_PATH`, `EMITTER_BASE_PATH`, `PLUGINS_PATHS`,
`EMIT_INTERMEDIATE_RESULTS`) are substituted by the test harness — replace them
with real values in production configs. The first two are paths;
`EMIT_INTERMEDIATE_RESULTS` is the boolean `emitIntermediateResults` flag.
`PLUGINS_PATHS` must name a directory holding the plugin zips; with tika-app or
tika-server, delete the `plugin-roots` line to use the bundled `plugins/`
directory (see xref:pipes/plugins/writing-a-plugin. [...]
[#emit-all]
=== Emit-all variant
diff --git a/docs/modules/ROOT/pages/pipes/getting-started.adoc
b/docs/modules/ROOT/pages/pipes/getting-started.adoc
index a9976a32e2..7af489cf3e 100644
--- a/docs/modules/ROOT/pages/pipes/getting-started.adoc
+++ b/docs/modules/ROOT/pages/pipes/getting-started.adoc
@@ -69,7 +69,13 @@ include::example$pipes-fs-pipeline.json[]
----
icon:github[]
https://github.com/apache/tika/blob/main/tika-pipes/tika-pipes-integration-tests/src/test/resources/configs/tika-config-basic.json[View
source on GitHub]
-NOTE: The values shown like `FETCHER_BASE_PATH`, `EMITTER_BASE_PATH`,
`PLUGINS_PATHS`, and `EMIT_INTERMEDIATE_RESULTS` are placeholders the
integration tests substitute at runtime. Replace them with real paths (or, for
`EMIT_INTERMEDIATE_RESULTS`, the boolean `true`/`false`) in your own config.
+NOTE: `FETCHER_BASE_PATH`, `EMITTER_BASE_PATH`, `PLUGINS_PATHS` and
`EMIT_INTERMEDIATE_RESULTS` are
+placeholders the integration tests substitute at runtime. Replace the two base
paths with real
+paths and `EMIT_INTERMEDIATE_RESULTS` with `true` or `false`. With tika-app,
**delete the
+`plugin-roots` line**: tika-app then loads the plugins from the `plugins/`
directory beside its jar.
+A `plugin-roots` you set replaces that default, and if it names a directory
without the plugin zips
+the run fails with `Unknown type: file-system-pipes-iterator. Available: []`.
See
+xref:pipes/plugins/writing-a-plugin.adoc#plugin-roots[plugin-roots].
Run it with:
diff --git a/docs/modules/ROOT/pages/pipes/plugins/filesystem.adoc
b/docs/modules/ROOT/pages/pipes/plugins/filesystem.adoc
index e1ec32d54a..50ef19efc9 100644
--- a/docs/modules/ROOT/pages/pipes/plugins/filesystem.adoc
+++ b/docs/modules/ROOT/pages/pipes/plugins/filesystem.adoc
@@ -48,7 +48,7 @@ The File System plugin (`tika-pipes-file-system`) is the most
common starting po
== Complete Pipeline Example
-The canonical filesystem-to-filesystem integration-test config.
`FETCHER_BASE_PATH`, `EMITTER_BASE_PATH` and `PLUGINS_PATHS` are placeholders
the test harness substitutes with real paths; `EMIT_INTERMEDIATE_RESULTS` with
the boolean `true` or `false`. See xref:pipes/configuration.adoc[Pipes
Configuration] for what each setting does.
+The canonical filesystem-to-filesystem integration-test config.
`FETCHER_BASE_PATH`, `EMITTER_BASE_PATH` and `PLUGINS_PATHS` are placeholders
the test harness substitutes with real paths; `EMIT_INTERMEDIATE_RESULTS` with
the boolean `true` or `false`. With tika-app or tika-server, delete the
`plugin-roots` line rather than filling it in, so the bundled `plugins/`
directory is used (see
xref:pipes/plugins/writing-a-plugin.adoc#plugin-roots[plugin-roots]). See
xref:pipes/configuration.adoc [...]
[source,json,subs=none]
----
diff --git a/docs/modules/ROOT/pages/pipes/plugins/writing-a-plugin.adoc
b/docs/modules/ROOT/pages/pipes/plugins/writing-a-plugin.adoc
index ce36b0c7ec..deca84bea7 100644
--- a/docs/modules/ROOT/pages/pipes/plugins/writing-a-plugin.adoc
+++ b/docs/modules/ROOT/pages/pipes/plugins/writing-a-plugin.adoc
@@ -297,6 +297,7 @@ Everything else — your own transitive libraries — belongs
in `lib/`.
== Installing and configuring
+[#plugin-roots]
=== Deployment
`plugin-roots` is a directory of **zip files**, not of unpacked plugin
directories:
@@ -316,6 +317,10 @@ all fill it in when you do not: a `plugins` directory
beside the running jar, el
working directory. Loading through `TikaPluginManager` directly with no
`plugin-roots` fails with
`plugin-roots must be specified`.
+A `plugin-roots` you set replaces that default rather than adding to it. If it
names a directory
+with none of the needed zips, loading fails with `Unknown type: <component>.
Available: []`; the
+empty list is the sign that no plugins were found.
+
Zips in `plugin-roots` are the only source of plugins. A plugin jar on the
application classpath
is not scanned, even though it carries the same `META-INF/extensions.idx`,
unless the JVM runs
with `-Dtika.plugins.classpath=true`. That switch exists so `PipesForkParser`
can embed without a
diff --git
a/tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/PluginsWriter.java
b/tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/PluginsWriter.java
index 8b4785d41e..77f66ec003 100644
---
a/tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/PluginsWriter.java
+++
b/tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/PluginsWriter.java
@@ -87,6 +87,9 @@ public class PluginsWriter {
Path userConfigPath =
Paths.get(simpleAsyncConfig.getTikaConfig());
JsonNode userRoot =
objectMapper.readTree(userConfigPath.toFile());
mergeUserConfig(root, (ObjectNode) userRoot);
+ if (DefaultPluginsDir.needsDefault(root.get("plugin-roots"))) {
+ root.put("plugin-roots", pluginString);
+ }
}
// Resolve any unfilled placeholders left over from
diff --git
a/tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/TikaAsyncCLI.java
b/tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/TikaAsyncCLI.java
index 227136e7ce..387a7acfc0 100644
---
a/tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/TikaAsyncCLI.java
+++
b/tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/TikaAsyncCLI.java
@@ -448,7 +448,7 @@ public class TikaAsyncCLI {
ObjectMapper mapper = TikaObjectMapperFactory.getMapper();
JsonNode rootNode = mapper.readTree(originalConfigPath.toFile());
- if (rootNode.has("plugin-roots")) {
+ if (!DefaultPluginsDir.needsDefault(rootNode.get("plugin-roots"))) {
// plugin-roots already set, use original config
return originalConfigPath;
}
diff --git
a/tika-pipes/tika-async-cli/src/test/java/org/apache/tika/async/cli/TikaConfigAsyncWriterTest.java
b/tika-pipes/tika-async-cli/src/test/java/org/apache/tika/async/cli/TikaConfigAsyncWriterTest.java
index 4738fbc362..4aa5fdef7f 100644
---
a/tika-pipes/tika-async-cli/src/test/java/org/apache/tika/async/cli/TikaConfigAsyncWriterTest.java
+++
b/tika-pipes/tika-async-cli/src/test/java/org/apache/tika/async/cli/TikaConfigAsyncWriterTest.java
@@ -141,4 +141,21 @@ public class TikaConfigAsyncWriterTest {
.path("onExists").asText(), pair[0]);
}
}
+
+ @Test
+ public void testPluginRootsPlaceholderGetsDefault(@TempDir Path dir)
throws Exception {
+ // the docs' example configs carry this token (TIKA-4984)
+ Path config = dir.resolve("config.json");
+ Files.writeString(config, "{ \"plugin-roots\": \"PLUGINS_PATHS\" }");
+ Path plugins = Files.createDirectories(dir.resolve("my-plugins"));
+ SimpleAsyncConfig simpleAsyncConfig = new SimpleAsyncConfig("input",
"output", 4,
+ null, null, null,
config.toAbsolutePath().toString().replace("\\", "/"),
+ BasicContentHandlerFactory.HANDLER_TYPE.TEXT,
+ SimpleAsyncConfig.ExtractBytesMode.NONE, plugins.toString());
+
+ Path tmp = Files.createTempFile(dir, "plugins-", ".json");
+ new PluginsWriter(simpleAsyncConfig, null).write(tmp);
+ assertEquals(plugins.toAbsolutePath().toString(),
+ new
ObjectMapper().readTree(tmp.toFile()).path("plugin-roots").asText());
+ }
}
diff --git
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/config/ConfigMerger.java
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/config/ConfigMerger.java
index 7f6b00bf83..8916ba5329 100644
---
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/config/ConfigMerger.java
+++
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/config/ConfigMerger.java
@@ -250,7 +250,7 @@ public class ConfigMerger {
}
// Apply plugin roots if not already set
- if (overrides.getPluginRoots() != null && !root.has("plugin-roots")) {
+ if (overrides.getPluginRoots() != null &&
DefaultPluginsDir.needsDefault(root.get("plugin-roots"))) {
root.put("plugin-roots", overrides.getPluginRoots());
LOG.debug("Set plugin-roots: {}", overrides.getPluginRoots());
}
diff --git
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/config/DefaultPluginsDir.java
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/config/DefaultPluginsDir.java
index bc4400541a..2fb481897b 100644
---
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/config/DefaultPluginsDir.java
+++
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/config/DefaultPluginsDir.java
@@ -19,7 +19,9 @@ package org.apache.tika.pipes.core.config;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Optional;
+import java.util.Set;
+import com.fasterxml.jackson.databind.JsonNode;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -48,6 +50,9 @@ public final class DefaultPluginsDir {
*/
public static final String PLUGINS_DIR_NAME = "plugins";
+ // tokens the example configs shown in the docs carry for plugin-roots
+ private static final Set<String> PLACEHOLDERS = Set.of("PLUGINS_PATHS",
"PLUGIN_ROOTS");
+
private DefaultPluginsDir() {
}
@@ -73,6 +78,16 @@ public final class DefaultPluginsDir {
return find(codeSourceDir(anchor), Path.of(""));
}
+ /**
+ * @param pluginRoots the configured {@code plugin-roots} node, or null
+ * @return true when it is absent or still a placeholder token copied from
an example
+ * config, so the caller should install the default
+ */
+ public static boolean needsDefault(JsonNode pluginRoots) {
+ return pluginRoots == null || pluginRoots.isNull()
+ || (pluginRoots.isTextual() &&
PLACEHOLDERS.contains(pluginRoots.asText()));
+ }
+
private static Path codeSourceDir(Class<?> anchor) {
try {
return
Path.of(anchor.getProtectionDomain().getCodeSource().getLocation()
diff --git
a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/config/ConfigMergerTest.java
b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/config/ConfigMergerTest.java
index b0a4f9f1d8..b8f5624010 100644
---
a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/config/ConfigMergerTest.java
+++
b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/config/ConfigMergerTest.java
@@ -322,6 +322,24 @@ public class ConfigMergerTest {
Files.deleteIfExists(result.configPath());
}
+ @Test
+ public void testPluginRootsPlaceholderReplaced() throws IOException {
+ // the docs' example configs carry this token (TIKA-4984)
+ Path existingPath = tempDir.resolve("config-with-placeholder.json");
+ Files.writeString(existingPath, "{ \"plugin-roots\": \"PLUGINS_PATHS\"
}");
+
+ ConfigOverrides overrides = ConfigOverrides.builder()
+ .setPluginRoots("default-plugins")
+ .build();
+
+ ConfigMerger.MergeResult result =
ConfigMerger.mergeOrCreate(existingPath, overrides);
+
+ JsonNode root = new
ObjectMapper().readTree(result.configPath().toFile());
+ assertEquals("default-plugins", root.get("plugin-roots").asText());
+
+ Files.deleteIfExists(result.configPath());
+ }
+
@Test
public void testPluginRootsNotOverriddenIfExists() throws IOException {
// Create config with existing plugin-roots
diff --git
a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/config/DefaultPluginsDirTest.java
b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/config/DefaultPluginsDirTest.java
index a7df4d1a6c..86fe4cace4 100644
---
a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/config/DefaultPluginsDirTest.java
+++
b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/config/DefaultPluginsDirTest.java
@@ -17,11 +17,13 @@
package org.apache.tika.pipes.core.config;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.nio.file.Files;
import java.nio.file.Path;
+import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
@@ -65,4 +67,15 @@ public class DefaultPluginsDirTest {
//the forked pipes server must not re-resolve the path against its own
cwd
assertTrue(DefaultPluginsDir.resolve(null, cwd).isAbsolute());
}
+
+ @Test
+ public void needsDefaultForAbsentOrPlaceholder() throws Exception {
+ ObjectMapper mapper = new ObjectMapper();
+ assertTrue(DefaultPluginsDir.needsDefault(null));
+ assertTrue(DefaultPluginsDir.needsDefault(mapper.readTree("null")));
+
assertTrue(DefaultPluginsDir.needsDefault(mapper.readTree("\"PLUGINS_PATHS\"")));
+
assertTrue(DefaultPluginsDir.needsDefault(mapper.readTree("\"PLUGIN_ROOTS\"")));
+
assertFalse(DefaultPluginsDir.needsDefault(mapper.readTree("\"/opt/tika/plugins\"")));
+
assertFalse(DefaultPluginsDir.needsDefault(mapper.readTree("[\"/opt/tika/plugins\"]")));
+ }
}
