This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tika.git
The following commit(s) were added to refs/heads/main by this push:
new e2f8e12b4d TIKA-4975 (#3353)
e2f8e12b4d is described below
commit e2f8e12b4d8f9b711cdfa778d96d495d37ecfb5a
Author: Tim Allison <[email protected]>
AuthorDate: Mon Oct 19 21:12:02 2026 -0400
TIKA-4975 (#3353)
* TIKA-4975: enforce @OperatorOnly wherever a request builds a component
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-authored-by: Tilman Hausherr <[email protected]>
---
.skills/devs/development/SKILL.md | 4 ++
CHANGES.txt | 10 +++++
.../ROOT/pages/developers/serialization.adoc | 6 +++
docs/modules/ROOT/pages/pipes/plugins/http.adoc | 11 +++++
.../org/apache/tika/config/ConfigDeserializer.java | 3 ++
.../apache/tika/pipes/grpc/TikaGrpcServerImpl.java | 2 +
.../apache/tika/pipes/grpc/TikaGrpcServerTest.java | 15 +++++++
tika-parent/forbidden-apis-json-reader-bans.txt | 22 ++++++++++
tika-parent/pom.xml | 3 ++
.../core/serialization/JsonFetchEmitTupleList.java | 2 +
.../serialization/JsonFetchEmitTupleListTest.java | 10 +++++
.../tika/pipes/emitter/fs/FileSystemEmitter.java | 11 ++---
.../emitter/fs/FileSystemEmitterRuntimeConfig.java | 8 ----
.../fs/FileSystemEmitterRuntimeConfigTest.java | 5 +--
.../tika/pipes/fetcher/http/HttpFetcher.java | 17 ++++----
.../http/config/HttpFetcherRequestConfig.java | 26 ++++++-----
.../tika/pipes/fetcher/http/HttpFetcherTest.java | 13 +++++-
.../org/apache/tika/plugins/ExtensionConfig.java | 7 ++-
.../java/org/apache/tika/plugins/PluginJson.java | 48 +++++++++++++--------
tika-serialization/pom.xml | 5 +++
.../config/loader/TikaObjectMapperFactory.java | 50 +++++++++++++---------
.../config/loader/TikaObjectMapperFactoryTest.java | 11 +++++
.../tika/server/core/resource/TikaResource.java | 3 +-
.../core/resource/TikaResourceConfigMergeTest.java | 28 +++++++-----
24 files changed, 225 insertions(+), 95 deletions(-)
diff --git a/.skills/devs/development/SKILL.md
b/.skills/devs/development/SKILL.md
index 31407ae236..b02b6f08a2 100644
--- a/.skills/devs/development/SKILL.md
+++ b/.skills/devs/development/SKILL.md
@@ -177,6 +177,10 @@ edits are included.
invariant, workaround, spec quirk). Never restate the code, narrate the
next line, justify the change to a reviewer, or describe past states of
the code.
+- **A `JsonConfig` is read only through `ConfigDeserializer` or
`PluginJson.read(JsonConfig, Class)`**,
+ which apply request restrictions and `@OperatorOnly`; forbidden-apis rejects
+ `JsonConfig.json()` in main code outside tika-serialization. Per-request
plugin config is
+ read into a class holding only what a request may set.
- **XML is parsed only through `XMLReaderUtils.parseSAX` / `buildDOM`**
(forbidden-apis
rejects the JAXP and Commons Secure XML factories and the raw `getSAXParser`
/
`getXMLReader` / `getDocumentBuilder` in main code). Never install an
`EntityResolver`;
diff --git a/CHANGES.txt b/CHANGES.txt
index 3156ae198b..fc0e7f6a6c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,15 @@
Release 4.2.0 - unreleased
+ * A JsonConfig's raw text is read only through ConfigDeserializer or
+ PluginJson.read(JsonConfig, Class), which apply the request restrictions
and
+ @OperatorOnly; forbidden-apis rejects JsonConfig.json() elsewhere in
Tika's main code.
+ Projects that inherit tika-parent inherit the ban; set
json.forbiddenapis.signaturesFile
+ to forbidden-apis-none.txt to opt out. ExtensionConfig no longer
implements
+ JsonConfig. A request's http-fetcher config may now set only
httpRequestHeaders; other
+ keys, previously ignored, are an error. A file-system emitter's runtime
config that
+ names basePath fails as an unknown key. A duplicate key in configuration
JSON,
+ previously last-wins, is now an error (TIKA-4975).
+
* @OperatorOnly is now enforced on every component a request builds,
wherever
it is named -- a metadata-filters entry, a nested property, or a bare
string -- not just on self-configuring parsers. On a class, it keeps a
diff --git a/docs/modules/ROOT/pages/developers/serialization.adoc
b/docs/modules/ROOT/pages/developers/serialization.adoc
index aad277e085..d9f928ae67 100644
--- a/docs/modules/ROOT/pages/developers/serialization.adoc
+++ b/docs/modules/ROOT/pages/developers/serialization.adoc
@@ -352,6 +352,12 @@ Java code that builds a component from a `JsonConfig` it
wrote itself is operato
says so with `JsonConfig.trusted(json)`; a plain lambda is untrusted, so it
cannot set an
operator-only property.
+A `JsonConfig` may be per-request input, so its raw text is read only by the
readers that
+apply these checks: `ConfigDeserializer` and the rest of tika-serialization,
and
+`PluginJson.read(JsonConfig, Class)` for plugins. forbidden-apis rejects
`JsonConfig.json()`
+anywhere else in Tika's main code. A plugin reads its per-request config into
a class holding
+only the keys a request may set.
+
Writing a config class: annotate the setter of anything a caller must not be
able to point
elsewhere. A rule the annotation cannot express — a ceiling a request may
lower but not raise,
a field that unlocks when the operator opts in — belongs in the component
after the merge, as
diff --git a/docs/modules/ROOT/pages/pipes/plugins/http.adoc
b/docs/modules/ROOT/pages/pipes/plugins/http.adoc
index fea9d0e2a7..17e2701934 100644
--- a/docs/modules/ROOT/pages/pipes/plugins/http.adoc
+++ b/docs/modules/ROOT/pages/pipes/plugins/http.adoc
@@ -128,6 +128,17 @@ include::example$pipes-http-fetcher.json[]
|Base64-encoded private key for asymmetric (RSA/ECDSA) JWT signing. Mutually
exclusive with `jwtSecret`.
|===
+==== Per-request headers
+
+A request can add headers to one fetch through an `http-fetcher` entry in its
parse context
+(gRPC: `additional_fetch_config_json`). Only `httpRequestHeaders` may be set
there; any other
+key is an error, since credentials, proxy, TLS and JWT settings belong to the
operator.
+
+[source,json]
+----
+{ "httpRequestHeaders": { "map": { "X-Request-Id": ["abc123"] } } }
+----
+
[#security-notes]
== Security Notes
diff --git
a/tika-core/src/main/java/org/apache/tika/config/ConfigDeserializer.java
b/tika-core/src/main/java/org/apache/tika/config/ConfigDeserializer.java
index 18aefa2f16..e28cafd866 100644
--- a/tika-core/src/main/java/org/apache/tika/config/ConfigDeserializer.java
+++ b/tika-core/src/main/java/org/apache/tika/config/ConfigDeserializer.java
@@ -16,6 +16,8 @@
*/
package org.apache.tika.config;
+import org.apache.tika.utils.SuppressForbidden;
+
/**
* Deserializes JSON configuration without a compile-time dependency on
Jackson.
* <p>
@@ -61,6 +63,7 @@ public class ConfigDeserializer {
* @throws RuntimeException if tika-serialization is not on the classpath,
deserialization
* fails, or untrusted JSON sets an operator-only property
*/
+ @SuppressForbidden
public static <T> T buildConfig(JsonConfig jsonConfig, Class<T>
configClass) {
ConfigReader reader = ConfigReader.find();
if (reader == null) {
diff --git
a/tika-grpc/src/main/java/org/apache/tika/pipes/grpc/TikaGrpcServerImpl.java
b/tika-grpc/src/main/java/org/apache/tika/pipes/grpc/TikaGrpcServerImpl.java
index 7323a01c54..7ce28f672b 100644
--- a/tika-grpc/src/main/java/org/apache/tika/pipes/grpc/TikaGrpcServerImpl.java
+++ b/tika-grpc/src/main/java/org/apache/tika/pipes/grpc/TikaGrpcServerImpl.java
@@ -24,6 +24,7 @@ import java.util.Objects;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.core.StreamReadFeature;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.json.JsonMapper;
@@ -85,6 +86,7 @@ class TikaGrpcServerImpl extends TikaGrpc.TikaImplBase {
private static final Logger LOG =
LoggerFactory.getLogger(TikaGrpcServerImpl.class);
private static final ObjectMapper OBJECT_MAPPER = JsonMapper.builder()
.defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL,
JsonInclude.Include.NON_NULL))
+ .enable(StreamReadFeature.STRICT_DUPLICATE_DETECTION)
.build();
public static final JsonSchemaGenerator JSON_SCHEMA_GENERATOR = new
JsonSchemaGenerator(OBJECT_MAPPER);
diff --git
a/tika-grpc/src/test/java/org/apache/tika/pipes/grpc/TikaGrpcServerTest.java
b/tika-grpc/src/test/java/org/apache/tika/pipes/grpc/TikaGrpcServerTest.java
index 1911cad2f5..8090113fff 100644
--- a/tika-grpc/src/test/java/org/apache/tika/pipes/grpc/TikaGrpcServerTest.java
+++ b/tika-grpc/src/test/java/org/apache/tika/pipes/grpc/TikaGrpcServerTest.java
@@ -568,6 +568,21 @@ public class TikaGrpcServerTest {
"the caller must be told which entry: " +
ex.getStatus().getDescription());
}
+ @Test
+ public void testDuplicateParseContextKeyIsRejected() throws Exception {
+ StatusRuntimeException ex =
Assertions.assertThrows(StatusRuntimeException.class, () ->
+ blockingStub.fetchAndParse(FetchAndParseRequest
+ .newBuilder()
+ .setFetcherId(createFetcherId(1))
+ .setFetchKey("no-such-file")
+
.setParseContextJson("{\"basic-content-handler-factory\":{\"type\":\"XML\"},"
+ +
"\"basic-content-handler-factory\":{\"type\":\"TEXT\"}}")
+ .build()));
+ assertEquals(Status.Code.INVALID_ARGUMENT,
ex.getStatus().getCode());
+
assertTrue(String.valueOf(ex.getStatus().getDescription()).contains("Duplicate
field"),
+ String.valueOf(ex.getStatus().getDescription()));
+ }
+
/**
* TIKA-4804: the server-streaming variant must close the call. With
the in-process
* transport and a direct executor the whole handler runs inside the
stub call, so
diff --git a/tika-parent/forbidden-apis-json-reader-bans.txt
b/tika-parent/forbidden-apis-json-reader-bans.txt
new file mode 100644
index 0000000000..0231dcfaf2
--- /dev/null
+++ b/tika-parent/forbidden-apis-json-reader-bans.txt
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# TIKA-4975: a JsonConfig may be per-request input. Its raw text is read only
by the
+# readers that apply the request restrictions and @OperatorOnly
(tika-serialization,
+# ConfigDeserializer, PluginJson); reading it with any other mapper bypasses
them.
+
+@defaultMessage read a JsonConfig through ConfigDeserializer or
PluginJson.read(JsonConfig, Class)
+
+org.apache.tika.config.JsonConfig#json()
diff --git a/tika-parent/pom.xml b/tika-parent/pom.xml
index 518889d3d9..2acfabe790 100644
--- a/tika-parent/pom.xml
+++ b/tika-parent/pom.xml
@@ -326,6 +326,8 @@
<!-- TIKA-4848: default-on in every module's main sources (see
forbiddenapis-check below) -->
<exception.forbiddenapis.signaturesFile>${maven.multiModuleProjectDirectory}/tika-parent/forbidden-apis-exception-reporting-bans.txt</exception.forbiddenapis.signaturesFile>
<xml.forbiddenapis.signaturesFile>${maven.multiModuleProjectDirectory}/tika-parent/forbidden-apis-xml-factory-bans.txt</xml.forbiddenapis.signaturesFile>
+ <!-- TIKA-4975: default-on; configuration JSON only through
tika-serialization -->
+
<json.forbiddenapis.signaturesFile>${maven.multiModuleProjectDirectory}/tika-parent/forbidden-apis-json-reader-bans.txt</json.forbiddenapis.signaturesFile>
<!-- Per-module extra main-source bans; a module overrides this (e.g. to
${metadata.forbiddenapis.signaturesFile}) instead of overriding the
forbiddenapis-check execution, whose child list would REPLACE the
parent's
@@ -1486,6 +1488,7 @@
<signaturesFiles>
<signaturesFile>${exception.forbiddenapis.signaturesFile}</signaturesFile>
<signaturesFile>${xml.forbiddenapis.signaturesFile}</signaturesFile>
+
<signaturesFile>${json.forbiddenapis.signaturesFile}</signaturesFile>
<signaturesFile>${forbiddenapis.module.signaturesFile}</signaturesFile>
</signaturesFiles>
</configuration>
diff --git
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/JsonFetchEmitTupleList.java
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/JsonFetchEmitTupleList.java
index 060e75861e..a5948b8ef4 100644
---
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/JsonFetchEmitTupleList.java
+++
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/JsonFetchEmitTupleList.java
@@ -22,6 +22,7 @@ import java.io.StringWriter;
import java.io.Writer;
import java.util.List;
+import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
@@ -50,6 +51,7 @@ public class JsonFetchEmitTupleList {
module.addSerializer(ParseContext.class, new ParseContextSerializer());
module.addDeserializer(ParseContext.class, new
ParseContextDeserializer());
OBJECT_MAPPER.registerModule(module);
+ OBJECT_MAPPER.enable(JsonParser.Feature.STRICT_DUPLICATE_DETECTION);
}
public static List<FetchEmitTuple> fromJson(Reader reader) throws
IOException {
diff --git
a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/JsonFetchEmitTupleListTest.java
b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/JsonFetchEmitTupleListTest.java
index 8290637dab..6beebd3e3b 100644
---
a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/JsonFetchEmitTupleListTest.java
+++
b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/JsonFetchEmitTupleListTest.java
@@ -65,6 +65,16 @@ public class JsonFetchEmitTupleListTest {
assertTrue(e.getMessage().contains(JsonFetchEmitTupleList.TUPLES),
e.getMessage());
}
+ @Test
+ public void testDuplicateKeyIsRejected() {
+ // single-tuple /pipes rejects this; a batch must too, not keep the
last value
+ String body =
"{\"tuples\":[{\"id\":\"x\",\"fetcher\":\"f\",\"fetchKey\":\"a\"," +
+ "\"fetchKey\":\"b\",\"emitter\":\"e\"}]}";
+ IOException e = assertThrows(IOException.class,
+ () -> JsonFetchEmitTupleList.fromJson(new StringReader(body)));
+ assertTrue(e.getMessage().contains("fetchKey"), e.getMessage());
+ }
+
/** Unknown tuple fields must surface as IOException, not Jackson's
IllegalArgumentException. */
@Test
public void testUnknownTupleFieldIsRejectedAsIOException() {
diff --git
a/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitter.java
b/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitter.java
index 5e12739827..c5e3621113 100644
---
a/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitter.java
+++
b/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitter.java
@@ -40,6 +40,7 @@ import org.apache.tika.parser.ParseContext;
import org.apache.tika.pipes.api.ParseMode;
import org.apache.tika.pipes.api.emitter.AbstractStreamEmitter;
import org.apache.tika.plugins.ExtensionConfig;
+import org.apache.tika.plugins.PluginJson;
import org.apache.tika.serialization.JsonMetadataList;
import org.apache.tika.utils.StringUtils;
@@ -274,13 +275,9 @@ public class FileSystemEmitter extends
AbstractStreamEmitter {
if (parseContext.hasJsonConfig(configKey)) {
JsonConfig configJson = parseContext.getJsonConfig(configKey);
if (configJson != null) {
- // Check if basePath is present in runtime config - this is
not allowed for security
- if (configJson.json().contains("\"basePath\"")) {
- throw new TikaConfigException("Cannot change 'basePath' at
runtime for security reasons. " + "basePath can only be set during
initialization.");
- }
-
- // Load runtime config (excludes basePath for security)
- FileSystemEmitterRuntimeConfig runtimeConfig =
FileSystemEmitterRuntimeConfig.load(configJson.json());
+ // basePath, allowAbsolutePaths and atomicWrites are not
runtime keys; naming one fails
+ FileSystemEmitterRuntimeConfig runtimeConfig =
+ PluginJson.read(configJson,
FileSystemEmitterRuntimeConfig.class);
// Merge runtime config into default config while preserving
basePath and the
// init-time allowAbsolutePaths -- neither may be changed at
runtime.
diff --git
a/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitterRuntimeConfig.java
b/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitterRuntimeConfig.java
index 2ceffcea01..45951049fc 100644
---
a/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitterRuntimeConfig.java
+++
b/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitterRuntimeConfig.java
@@ -16,9 +16,6 @@
*/
package org.apache.tika.pipes.emitter.fs;
-import org.apache.tika.exception.TikaConfigException;
-import org.apache.tika.plugins.PluginJson;
-
/**
* Runtime configuration for FileSystemEmitter.
* Only includes fields that are safe to update at runtime.
@@ -26,11 +23,6 @@ import org.apache.tika.plugins.PluginJson;
*/
public class FileSystemEmitterRuntimeConfig {
- public static FileSystemEmitterRuntimeConfig load(final String json)
- throws TikaConfigException {
- return PluginJson.read(json, FileSystemEmitterRuntimeConfig.class);
- }
-
private String fileExtension;
private FileSystemEmitterConfig.ON_EXISTS onExists;
private boolean prettyPrint;
diff --git
a/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/test/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitterRuntimeConfigTest.java
b/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/test/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitterRuntimeConfigTest.java
index f3abd3dd67..d644be393b 100644
---
a/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/test/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitterRuntimeConfigTest.java
+++
b/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/test/java/org/apache/tika/pipes/emitter/fs/FileSystemEmitterRuntimeConfigTest.java
@@ -71,8 +71,7 @@ public class FileSystemEmitterRuntimeConfigTest {
Path output1 = dir1.resolve("test1.json");
assertTrue(Files.exists(output1), "File should be created in dir1");
- // Try to override basePath at runtime to point to dir2
- // This should throw an exception for security reasons
+ // basePath is not a runtime key, so naming it fails
String runtimeConfig = String.format(Locale.ROOT,
"{\"basePath\":\"%s\", \"onExists\":\"REPLACE\"}",
dir2.toString().replace("\\", "\\\\"));
@@ -90,7 +89,7 @@ public class FileSystemEmitterRuntimeConfigTest {
emitter.emit("test2.json", metadataList2, context2);
});
assertTrue(exception.getCause() != null &&
- exception.getCause().getMessage().contains("Cannot change
'basePath' at runtime"),
+ exception.getCause().getMessage().contains("basePath"),
"Should throw exception when attempting to change basePath at
runtime");
}
diff --git
a/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/main/java/org/apache/tika/pipes/fetcher/http/HttpFetcher.java
b/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/main/java/org/apache/tika/pipes/fetcher/http/HttpFetcher.java
index 9d4cffcc54..68e2535d2b 100644
---
a/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/main/java/org/apache/tika/pipes/fetcher/http/HttpFetcher.java
+++
b/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/main/java/org/apache/tika/pipes/fetcher/http/HttpFetcher.java
@@ -39,8 +39,6 @@ import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.atomic.AtomicBoolean;
-import com.fasterxml.jackson.core.JsonProcessingException;
-import com.fasterxml.jackson.databind.ObjectMapper;
import com.nimbusds.jose.JOSEException;
import org.apache.commons.io.IOUtils;
import org.apache.commons.io.output.UnsynchronizedByteArrayOutputStream;
@@ -74,11 +72,13 @@ import org.apache.tika.parser.ParseContext;
import org.apache.tika.pipes.api.fetcher.Fetcher;
import org.apache.tika.pipes.api.fetcher.RangeFetcher;
import org.apache.tika.pipes.fetcher.http.config.HttpFetcherConfig;
+import org.apache.tika.pipes.fetcher.http.config.HttpFetcherRequestConfig;
import org.apache.tika.pipes.fetcher.http.jwt.JwtGenerator;
import org.apache.tika.pipes.fetcher.http.jwt.JwtPrivateKeyCreds;
import org.apache.tika.pipes.fetcher.http.jwt.JwtSecretCreds;
import org.apache.tika.plugins.AbstractTikaExtension;
import org.apache.tika.plugins.ExtensionConfig;
+import org.apache.tika.plugins.PluginJson;
import org.apache.tika.utils.StringUtils;
/**
@@ -94,7 +94,6 @@ public class HttpFetcher extends AbstractTikaExtension
implements Fetcher, Range
}
- private static final ObjectMapper OM = new ObjectMapper();
private HttpFetcherConfig httpFetcherConfig = new HttpFetcherConfig();
private HttpClientFactory httpClientFactory = new HttpClientFactory();
@@ -160,7 +159,7 @@ public class HttpFetcher extends AbstractTikaExtension
implements Fetcher, Range
@Override
public TikaInputStream fetch(String fetchKey, Metadata metadata,
ParseContext parseContext) throws IOException, TikaException {
- HttpFetcherConfig additionalHttpFetcherConfig =
getAdditionalHttpFetcherConfig(parseContext);
+ HttpFetcherRequestConfig additionalHttpFetcherConfig =
getAdditionalHttpFetcherConfig(parseContext);
HttpGet get = new HttpGet(fetchKey);
get.setConfig(buildRequestConfig());
setHttpRequestHeaders(metadata, get);
@@ -205,12 +204,12 @@ public class HttpFetcher extends AbstractTikaExtension
implements Fetcher, Range
}
}
- private HttpFetcherConfig getAdditionalHttpFetcherConfig(ParseContext
parseContext) throws JsonProcessingException {
- // Check for JSON config in ParseContext (new pattern)
+ private HttpFetcherRequestConfig
getAdditionalHttpFetcherConfig(ParseContext parseContext)
+ throws TikaConfigException {
if (parseContext.hasJsonConfig("http-fetcher")) {
JsonConfig jsonConfig = parseContext.getJsonConfig("http-fetcher");
if (jsonConfig != null) {
- return OM.readValue(jsonConfig.json(),
HttpFetcherConfig.class);
+ return PluginJson.read(jsonConfig,
HttpFetcherRequestConfig.class);
}
}
return null;
@@ -219,7 +218,7 @@ public class HttpFetcher extends AbstractTikaExtension
implements Fetcher, Range
@Override
public TikaInputStream fetch(String fetchKey, long startRange, long
endRange, Metadata metadata,
ParseContext parseContext) throws IOException,
TikaException {
- HttpFetcherConfig additionalHttpFetcherConfig =
getAdditionalHttpFetcherConfig(parseContext);
+ HttpFetcherRequestConfig additionalHttpFetcherConfig =
getAdditionalHttpFetcherConfig(parseContext);
HttpGet get = new HttpGet(fetchKey);
// Same RequestConfig as the whole-document fetch: maxRedirects must
bind range fetches too.
get.setConfig(buildRequestConfig());
@@ -230,7 +229,7 @@ public class HttpFetcher extends AbstractTikaExtension
implements Fetcher, Range
return execute(get, metadata, httpClient, true);
}
- private void putAdditionalHeadersOnRequest(HttpFetcherConfig
additionalFetcherConfig, HttpGet httpGet) throws TikaException {
+ private void putAdditionalHeadersOnRequest(HttpFetcherRequestConfig
additionalFetcherConfig, HttpGet httpGet) throws TikaException {
if (!StringUtils.isBlank(httpFetcherConfig.getUserAgent())) {
httpGet.setHeader(USER_AGENT, httpFetcherConfig.getUserAgent());
}
diff --git
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
b/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/main/java/org/apache/tika/pipes/fetcher/http/config/HttpFetcherRequestConfig.java
similarity index 55%
copy from
tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
copy to
tika-pipes/tika-pipes-plugins/tika-pipes-http/src/main/java/org/apache/tika/pipes/fetcher/http/config/HttpFetcherRequestConfig.java
index 52bc5d3c09..04e8b7b655 100644
---
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
+++
b/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/main/java/org/apache/tika/pipes/fetcher/http/config/HttpFetcherRequestConfig.java
@@ -14,19 +14,23 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.tika.plugins;
-
-import java.io.Serializable;
-
-import org.apache.tika.config.JsonConfig;
+package org.apache.tika.pipes.fetcher.http.config;
/**
- * Configuration for a plugin extension.
- *
- * @param id unique instance identifier
- * @param name the plugin type name
- * @param json the raw JSON configuration string for the plugin to parse
+ * What a request may set on the http-fetcher (its {@code "http-fetcher"}
parse-context entry):
+ * extra headers only. Everything else in {@link HttpFetcherConfig} is
operator config, and
+ * naming it here is an unknown-key error.
*/
-public record ExtensionConfig(String id, String name, String json) implements
JsonConfig, Serializable {
+public class HttpFetcherRequestConfig {
+
+ private HttpHeaders httpRequestHeaders = new HttpHeaders();
+
+ public HttpHeaders getHttpRequestHeaders() {
+ return httpRequestHeaders;
+ }
+ public HttpFetcherRequestConfig setHttpRequestHeaders(HttpHeaders
httpRequestHeaders) {
+ this.httpRequestHeaders = httpRequestHeaders;
+ return this;
+ }
}
diff --git
a/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/test/java/org/apache/tika/pipes/fetcher/http/HttpFetcherTest.java
b/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/test/java/org/apache/tika/pipes/fetcher/http/HttpFetcherTest.java
index fb02a28da3..fde8f185eb 100644
---
a/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/test/java/org/apache/tika/pipes/fetcher/http/HttpFetcherTest.java
+++
b/tika-pipes/tika-pipes-plugins/tika-pipes-http/src/test/java/org/apache/tika/pipes/fetcher/http/HttpFetcherTest.java
@@ -63,6 +63,7 @@ import org.mockito.Mockito;
import org.apache.tika.TikaTest;
import org.apache.tika.client.HttpClientFactory;
import org.apache.tika.config.loader.TikaJsonConfig;
+import org.apache.tika.exception.TikaConfigException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.io.TemporaryResources;
import org.apache.tika.metadata.Metadata;
@@ -71,6 +72,7 @@ import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.pipes.core.fetcher.FetcherManager;
import org.apache.tika.pipes.fetcher.http.config.HttpFetcherConfig;
+import org.apache.tika.pipes.fetcher.http.config.HttpFetcherRequestConfig;
import org.apache.tika.pipes.fetcher.http.config.HttpHeaders;
import org.apache.tika.pipes.fetcher.http.jwt.JwtGenerator;
import org.apache.tika.plugins.ExtensionConfig;
@@ -169,6 +171,15 @@ class HttpFetcherTest extends TikaTest {
.jwt();
}
+ @Test
+ public void testRequestConfigMaySetOnlyHeaders() {
+ ParseContext parseContext = new ParseContext();
+ parseContext.setJsonConfig("http-fetcher",
"{\"userName\":\"x\",\"password\":\"y\"}");
+ TikaConfigException e =
Assertions.assertThrows(TikaConfigException.class,
+ () -> httpFetcher.fetch("http://localhost";, new Metadata(),
parseContext));
+ Assertions.assertTrue(e.getMessage().contains("userName"),
e.getMessage());
+ }
+
@Test
public void testHttpRequestHeaders() throws Exception {
HttpClient httpClient = Mockito.mock(HttpClient.class);
@@ -201,7 +212,7 @@ class HttpFetcherTest extends TikaTest {
Metadata metadata = new Metadata();
ParseContext parseContext = new ParseContext();
- HttpFetcherConfig additionalHttpFetcherConfig = new
HttpFetcherConfig();
+ HttpFetcherRequestConfig additionalHttpFetcherConfig = new
HttpFetcherRequestConfig();
additionalHttpFetcherConfig.setHttpRequestHeaders(new HttpHeaders());
HashMap<String, Collection<String>> headersMap = new HashMap<>();
headersMap.put("fromFetchRequestHeader1",
List.of("fromFetchRequestValue1"));
diff --git
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
b/tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
index 52bc5d3c09..4ba4b31dae 100644
---
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
+++
b/tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
@@ -18,15 +18,14 @@ package org.apache.tika.plugins;
import java.io.Serializable;
-import org.apache.tika.config.JsonConfig;
-
/**
- * Configuration for a plugin extension.
+ * Configuration for a plugin extension. Not a {@link
org.apache.tika.config.JsonConfig}: that
+ * type may carry per-request input, and its raw text is read only under the
request checks.
*
* @param id unique instance identifier
* @param name the plugin type name
* @param json the raw JSON configuration string for the plugin to parse
*/
-public record ExtensionConfig(String id, String name, String json) implements
JsonConfig, Serializable {
+public record ExtensionConfig(String id, String name, String json) implements
Serializable {
}
diff --git
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/PluginJson.java
b/tika-plugins-core/src/main/java/org/apache/tika/plugins/PluginJson.java
index 03d9093add..406fe56999 100644
--- a/tika-plugins-core/src/main/java/org/apache/tika/plugins/PluginJson.java
+++ b/tika-plugins-core/src/main/java/org/apache/tika/plugins/PluginJson.java
@@ -16,34 +16,27 @@
*/
package org.apache.tika.plugins;
-import com.fasterxml.jackson.core.JsonParser;
+import java.io.IOException;
+
import com.fasterxml.jackson.core.JsonProcessingException;
-import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
-import com.fasterxml.jackson.databind.SerializationFeature;
+import org.apache.tika.config.JsonConfig;
+import org.apache.tika.config.loader.TikaObjectMapperFactory;
import org.apache.tika.exception.TikaConfigException;
+import org.apache.tika.serialization.OperatorOnlyGuard;
+import org.apache.tika.utils.SuppressForbidden;
/**
- * The one {@link ObjectMapper} for plugin configuration JSON ({@link
ExtensionConfig#json()}).
- * <p>
- * Fails fast on what a hand-written config gets wrong: unknown keys, a number
where an
- * enum name belongs, duplicate keys. Comments are
- * accepted, as in the main config loader. Plain JSON only -- no Tika
component serializers
- * -- so it is safe to share across the plugin classloader boundary, provided
the plugin
- * does not bundle its own Jackson (the plugins parent pom enforces that).
+ * Reads plugin configuration JSON ({@link ExtensionConfig#json()}) with
+ * {@link TikaObjectMapperFactory#createStrictPlainMapper()}: the main config
loader's checks,
+ * without Tika component serializers, so it is safe to share across the
plugin classloader
+ * boundary, provided the plugin does not bundle its own Jackson (the plugins
parent pom
+ * enforces that).
*/
public final class PluginJson {
- private static final ObjectMapper MAPPER = new ObjectMapper()
- .enable(JsonParser.Feature.ALLOW_COMMENTS)
- // parser-level: FAIL_ON_READING_DUP_TREE_KEY only covers
readTree, not readValue
- .enable(JsonParser.Feature.STRICT_DUPLICATE_DETECTION)
- .enable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
- // Not FAIL_ON_NULL_FOR_PRIMITIVES: it also fires for a primitive
record component
- // that is simply absent, and optional primitives with defaults
are the norm here.
- .enable(DeserializationFeature.FAIL_ON_NUMBERS_FOR_ENUMS)
- .disable(SerializationFeature.FAIL_ON_EMPTY_BEANS);
+ private static final ObjectMapper MAPPER =
TikaObjectMapperFactory.createStrictPlainMapper();
private PluginJson() {
}
@@ -57,4 +50,21 @@ public final class PluginJson {
"Failed to parse " + type.getSimpleName() + ": " +
e.getOriginalMessage(), e);
}
}
+
+ /**
+ * As {@link #read(String, Class)}, but untrusted JSON (per-request
config, see
+ * {@link JsonConfig#trusted()}) may not set an {@link
org.apache.tika.config.OperatorOnly}
+ * property of {@code type}.
+ */
+ @SuppressForbidden
+ public static <T> T read(JsonConfig config, Class<T> type) throws
TikaConfigException {
+ if (!config.trusted()) {
+ try {
+ OperatorOnlyGuard.check(MAPPER, type.getSimpleName(),
config.json(), type);
+ } catch (IOException e) {
+ throw new TikaConfigException(e.getMessage(), e);
+ }
+ }
+ return read(config.json(), type);
+ }
}
diff --git a/tika-serialization/pom.xml b/tika-serialization/pom.xml
index cfff7ed06b..bb9a0c4e93 100644
--- a/tika-serialization/pom.xml
+++ b/tika-serialization/pom.xml
@@ -34,6 +34,11 @@
<name>Apache Tika serialization</name>
<url>https://tika.apache.org</url>
+ <properties>
+ <!-- TIKA-4975: this module reads JsonConfigs under the request
restrictions -->
+
<json.forbiddenapis.signaturesFile>${maven.multiModuleProjectDirectory}/tika-parent/forbidden-apis-none.txt</json.forbiddenapis.signaturesFile>
+ </properties>
+
<dependencies>
<!-- Optional OSGi dependency, used only when running within OSGi -->
<dependency>
diff --git
a/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaObjectMapperFactory.java
b/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaObjectMapperFactory.java
index 1977005f7c..6e88ca7c94 100644
---
a/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaObjectMapperFactory.java
+++
b/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaObjectMapperFactory.java
@@ -151,30 +151,10 @@ public class TikaObjectMapperFactory {
* @return configured ObjectMapper
*/
public static ObjectMapper createMapper(JsonFactory factory) {
- ObjectMapper mapper = (factory != null) ? new ObjectMapper(factory) :
new ObjectMapper();
-
- // Allow comments in JSON config files (// and /* */ style)
- mapper.configure(JsonParser.Feature.ALLOW_COMMENTS, true);
-
- // Fail on unknown properties to catch configuration errors early
- mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES,
true);
-
+ ObjectMapper mapper = strict((factory != null) ? new
ObjectMapper(factory) : new ObjectMapper());
// Prevent null values being assigned to primitive fields (int,
boolean, etc.)
mapper.configure(DeserializationFeature.FAIL_ON_NULL_FOR_PRIMITIVES,
true);
- // Ensure enums are properly validated (not just numeric values)
- mapper.configure(DeserializationFeature.FAIL_ON_NUMBERS_FOR_ENUMS,
true);
-
- // Accept enum values in any case ("no_ocr" as well as "NO_OCR"):
hand-written
- // requests and older examples spell them either way
- mapper.configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_ENUMS, true);
-
- // Catch duplicate keys in JSON objects
- mapper.configure(DeserializationFeature.FAIL_ON_READING_DUP_TREE_KEY,
true);
-
- // Need to allow creation of classes without setters/getters -- we may
want to revisit this
- mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
-
sparseOverlays(mapper);
// Load component registries for name resolution
@@ -190,6 +170,34 @@ public class TikaObjectMapperFactory {
return mapper;
}
+ /**
+ * A strict mapper without Tika's component serializers, for plugin
configuration
+ * ({@code PluginJson}). Same checks as {@link #getMapper()} except
+ * {@code FAIL_ON_NULL_FOR_PRIMITIVES}, which also rejects an absent
primitive record
+ * component, and optional primitives with defaults are the norm in plugin
configs.
+ */
+ public static ObjectMapper createStrictPlainMapper() {
+ return strict(new ObjectMapper());
+ }
+
+ private static ObjectMapper strict(ObjectMapper mapper) {
+ // Allow comments in JSON config files (// and /* */ style)
+ mapper.configure(JsonParser.Feature.ALLOW_COMMENTS, true);
+ // Duplicate keys: the parser feature covers readValue; the tree
feature only readTree
+ mapper.configure(JsonParser.Feature.STRICT_DUPLICATE_DETECTION, true);
+ mapper.configure(DeserializationFeature.FAIL_ON_READING_DUP_TREE_KEY,
true);
+ // Fail on unknown properties to catch configuration errors early
+ mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES,
true);
+ // Ensure enums are properly validated (not just numeric values)
+ mapper.configure(DeserializationFeature.FAIL_ON_NUMBERS_FOR_ENUMS,
true);
+ // Accept enum values in any case ("no_ocr" as well as "NO_OCR"):
hand-written
+ // requests and older examples spell them either way
+ mapper.configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_ENUMS, true);
+ // Need to allow creation of classes without setters/getters -- we may
want to revisit this
+ mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
+ return mapper;
+ }
+
/**
* Loads component registries for name resolution.
* Registries are loaded from index files and registered with the
ComponentNameResolver.
diff --git
a/tika-serialization/src/test/java/org/apache/tika/config/loader/TikaObjectMapperFactoryTest.java
b/tika-serialization/src/test/java/org/apache/tika/config/loader/TikaObjectMapperFactoryTest.java
index d58e5e2c19..60100f028a 100644
---
a/tika-serialization/src/test/java/org/apache/tika/config/loader/TikaObjectMapperFactoryTest.java
+++
b/tika-serialization/src/test/java/org/apache/tika/config/loader/TikaObjectMapperFactoryTest.java
@@ -19,6 +19,7 @@ package org.apache.tika.config.loader;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
+import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.exc.InvalidFormatException;
import org.junit.jupiter.api.Test;
@@ -50,4 +51,14 @@ public class TikaObjectMapperFactoryTest {
assertThrows(InvalidFormatException.class,
() -> mapper.readValue("{\"strategy\":\"nope\"}",
Config.class));
}
+
+ @Test
+ public void testDuplicateKeyFailsOnReadValue() {
+ // the last value would otherwise win silently
+ String json = "{\"strategy\":\"AUTO\",\"strategy\":\"NO_OCR\"}";
+ assertThrows(JsonParseException.class,
+ () -> TikaObjectMapperFactory.getMapper().readValue(json,
Config.class));
+ assertThrows(JsonParseException.class,
+ () -> TikaObjectMapperFactory.readConfig(json, Config.class,
true));
+ }
}
diff --git
a/tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/TikaResource.java
b/tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/TikaResource.java
index f905cc15ce..89defdb838 100644
---
a/tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/TikaResource.java
+++
b/tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/TikaResource.java
@@ -26,6 +26,7 @@ import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Map;
+import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import jakarta.ws.rs.BadRequestException;
@@ -264,7 +265,7 @@ public class TikaResource {
* @throws IOException if parsing fails
*/
public static void mergeParseContextFromConfig(String configJson,
ParseContext context) throws IOException, TikaConfigException {
- ObjectMapper mapper = new ObjectMapper();
+ ObjectMapper mapper = new
ObjectMapper().enable(JsonParser.Feature.STRICT_DUPLICATE_DETECTION);
JsonNode root = mapper.readTree(configJson);
// Request-supplied config: restrict so it cannot bind wire-blocked
components.
ParseContext configuredContext =
ParseContextDeserializer.readParseContext(root, true);
diff --git
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
b/tika-server/tika-server-core/src/test/java/org/apache/tika/server/core/resource/TikaResourceConfigMergeTest.java
similarity index 51%
copy from
tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
copy to
tika-server/tika-server-core/src/test/java/org/apache/tika/server/core/resource/TikaResourceConfigMergeTest.java
index 52bc5d3c09..65e76832df 100644
---
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/ExtensionConfig.java
+++
b/tika-server/tika-server-core/src/test/java/org/apache/tika/server/core/resource/TikaResourceConfigMergeTest.java
@@ -14,19 +14,25 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.tika.plugins;
+package org.apache.tika.server.core.resource;
-import java.io.Serializable;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
-import org.apache.tika.config.JsonConfig;
+import java.io.IOException;
-/**
- * Configuration for a plugin extension.
- *
- * @param id unique instance identifier
- * @param name the plugin type name
- * @param json the raw JSON configuration string for the plugin to parse
- */
-public record ExtensionConfig(String id, String name, String json) implements
JsonConfig, Serializable {
+import org.junit.jupiter.api.Test;
+
+import org.apache.tika.parser.ParseContext;
+
+public class TikaResourceConfigMergeTest {
+ @Test
+ public void testDuplicateKeyIsRejected() {
+ String json = "{\"basic-content-handler-factory\":{\"type\":\"XML\"},"
+ + "\"basic-content-handler-factory\":{\"type\":\"TEXT\"}}";
+ IOException e = assertThrows(IOException.class,
+ () -> TikaResource.mergeParseContextFromConfig(json, new
ParseContext()));
+ assertTrue(e.getMessage().contains("Duplicate field"), e.getMessage());
+ }
}
