This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tika.git
The following commit(s) were added to refs/heads/main by this push:
new 47d2f35b03 TIKA-5060 -- add a jazzer fuzzer (#3569)
47d2f35b03 is described below
commit 47d2f35b03f63c5a315ab28dfbe10dba145a532d
Author: Tim Allison <[email protected]>
AuthorDate: Sat Nov 7 20:38:37 2026 -0500
TIKA-5060 -- add a jazzer fuzzer (#3569)
* TIKA-5060: add tika-fuzzing module with Jazzer AutoDetectParser fuzz
tests and corpus replay
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.skills/devs/oss-fuzz/SKILL.md | 4 +
pom.xml | 1 +
tika-fuzzing/.gitignore | 7 ++
tika-fuzzing/README.md | 85 +++++++++++++
tika-fuzzing/pom.xml | 92 +++++++++++++++
.../tika/fuzzing/AutoDetectParserFuzzTest.java | 131 +++++++++++++++++++++
.../org/apache/tika/fuzzing/CorpusReplayTest.java | 65 ++++++++++
.../src/test/resources/junit-platform.properties | 18 +++
8 files changed, 403 insertions(+)
diff --git a/.skills/devs/oss-fuzz/SKILL.md b/.skills/devs/oss-fuzz/SKILL.md
index 7b88478dc6..528daf7945 100644
--- a/.skills/devs/oss-fuzz/SKILL.md
+++ b/.skills/devs/oss-fuzz/SKILL.md
@@ -30,6 +30,10 @@ read after this file, wins on conflict.
# Tika OSS-Fuzz — local fuzzing
+For fuzzing or replaying a directory of files without Docker, use the in-repo
+`tika-fuzzing` module (Jazzer `@FuzzTest`, see `tika-fuzzing/README.md`). This
skill
+covers the OSS-Fuzz project, whose targets still live in the oss-fuzz repo.
+
Tika is already in OSS-Fuzz as the **`apache-tika`** project (not `tika`).
It is Jazzer-based (coverage-guided, in-process JVM fuzzing), not the old
`tika-fuzzing` seed-mutation module (removed in TIKA-4506). The fuzz targets
diff --git a/pom.xml b/pom.xml
index 1992c6671b..b7490cfe22 100644
--- a/pom.xml
+++ b/pom.xml
@@ -64,6 +64,7 @@
<module>tika-example</module>
<module>tika-java7</module>
<module>tika-handlers</module>
+ <module>tika-fuzzing</module>
</modules>
<profiles>
diff --git a/tika-fuzzing/.gitignore b/tika-fuzzing/.gitignore
new file mode 100644
index 0000000000..f01f699255
--- /dev/null
+++ b/tika-fuzzing/.gitignore
@@ -0,0 +1,7 @@
+# Jazzer-generated corpus
+.cifuzz-corpus/
+# Jazzer findings: replayed locally, committed only with `git add -f` once
fixed and disclosed
+src/test/resources/**/crash-*
+src/test/resources/**/timeout-*
+src/test/resources/**/oom-*
+src/test/resources/**/slow-unit-*
diff --git a/tika-fuzzing/README.md b/tika-fuzzing/README.md
new file mode 100644
index 0000000000..99f187b1a6
--- /dev/null
+++ b/tika-fuzzing/README.md
@@ -0,0 +1,85 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# tika-fuzzing
+
+[Jazzer](https://github.com/CodeIntelligenceTesting/jazzer) `@FuzzTest`
targets.
+`AutoDetectParserFuzzTest` runs `AutoDetectParser` (via
`RecursiveParserWrapper`) on
+`TikaInputStream.get(byte[])`, `get(InputStream)` and `get(Path)`. The
classpath holds
+`tika-parsers-standard-package` plus `tika-parser-sqlite3-module`; add a
parser module to
+the pom to fuzz it. Jazzer gets no coverage from native code (sqlite-jdbc),
and a native
+crash kills the JVM.
+
+The targets look for OOM, stack overflow, hangs, and exceptions other than
+`TikaException`/`IOException`/`SAXException`. `CompositeParser` wraps parser
+`RuntimeException`s in `TikaException`; add `-Dtika.fuzz.strict=true` to
report those too.
+
+| Property | Default | |
+|---|---|---|
+| `-Dtika.fuzz.timeout=<s>` | `60` | per-input parse timeout, all modes; a
hang is reported with the stuck stack |
+| `-Dtika.fuzz.xmx=<size>` | `3g` | test JVM heap (OSS-Fuzz uses `-Xmx3000m`) |
+| `JAZZER_MAX_DURATION=<d>` | Jazzer's | length of a fuzzing session |
+
+Do not use JUnit's `junit.jupiter.execution.timeout.*`: in fuzzing mode it
ends the
+whole session, not one input.
+
+## Regression mode (every build)
+
+`./mvnw clean test -pl tika-fuzzing` replays each method's inputs from
+`src/test/resources/org/apache/tika/fuzzing/AutoDetectParserFuzzTestInputs/<method>/`
+and `.cifuzz-corpus/`. An input that exceeds `tika.fuzz.timeout` fails the
build.
+
+## Replaying a directory
+
+For a new parser or a refactor, run every file under a directory through all
three
+input modes, with no fuzzing:
+
+```bash
+./mvnw clean install -pl tika-fuzzing -am -Pfast
+./mvnw clean test -pl tika-fuzzing -Dtest=CorpusReplayTest
-Dtika.fuzz.corpus=<dir> \
+ -Dtika.fuzz.timeout=10
+grep -o 'testcase name="[^"]*"'
tika-fuzzing/target/surefire-reports/TEST-*CorpusReplayTest.xml
+```
+
+The report names the mode and file of each failure; the console shows only an
index.
+
+One JVM runs the whole directory. An OOM ends the run, and a timed-out parse
cannot be
+stopped: it keeps a core and its memory, so files after it can time out
falsely. Trust
+the first timeout; re-run later ones alone.
+
+## Fuzzing
+
+```bash
+./mvnw clean install -pl tika-fuzzing -am -Pfast
+JAZZER_FUZZ=1 JAZZER_MAX_DURATION=10m \
+ ./mvnw clean test -pl tika-fuzzing -Dtest='AutoDetectParserFuzzTest#path'
+```
+
+Fuzzing mode runs one method at a time. To seed it, **copy** real files into
+`.cifuzz-corpus/org.apache.tika.fuzzing.AutoDetectParserFuzzTest/<method>/`
+(git-ignored). libFuzzer adds new inputs to that directory, so never point it
at the
+original corpus. A plain test run also replays everything in it.
+
+Jazzer stops at the first finding and writes it to
+`src/test/resources/org/apache/tika/fuzzing/AutoDetectParserFuzzTestInputs/<method>/`
as
+`crash-*` (exceptions, OOM and hangs alike). Later plain test runs replay it.
Finding
+files are git-ignored. Commit one with `git add -f` only after the bug is
fixed and
+disclosed (see `SECURITY.md`). Commit only small, synthesized files, never
corpus
+documents.
+
+After a session, empty `.cifuzz-corpus/` and remove the finding files you are
not
+keeping, or every later build replays them.
diff --git a/tika-fuzzing/pom.xml b/tika-fuzzing/pom.xml
new file mode 100644
index 0000000000..b84f7c136e
--- /dev/null
+++ b/tika-fuzzing/pom.xml
@@ -0,0 +1,92 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-parent</artifactId>
+ <version>${revision}</version>
+ <relativePath>../tika-parent/pom.xml</relativePath>
+ </parent>
+ <artifactId>tika-fuzzing</artifactId>
+ <name>Apache Tika fuzzing</name>
+ <description>
+ Jazzer fuzz targets. A plain test run replays the committed inputs; set
JAZZER_FUZZ=1 to fuzz.
+ </description>
+
+ <properties>
+ <jazzer.version>0.30.0</jazzer.version>
+ <maven.deploy.skip>true</maven.deploy.skip>
+ <!-- OSS-Fuzz runs Tika at -Xmx3000m -->
+ <tika.fuzz.xmx>3g</tika.fuzz.xmx>
+ <tika.fuzz.timeout>60</tika.fuzz.timeout>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>${project.groupId}</groupId>
+ <artifactId>tika-core</artifactId>
+ <version>${project.version}</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>${project.groupId}</groupId>
+ <artifactId>tika-parsers-standard-package</artifactId>
+ <version>${project.version}</version>
+ <type>pom</type>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>${project.groupId}</groupId>
+ <artifactId>tika-parser-sqlite3-module</artifactId>
+ <version>${project.version}</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.code-intelligence</groupId>
+ <artifactId>jazzer-junit</artifactId>
+ <version>${jazzer.version}</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter</artifactId>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <argLine>-Xmx${tika.fuzz.xmx} -Djava.awt.headless=true
"-Djava.io.tmpdir=${tika.test.tmpdir}" @{surefireArgLine}</argLine>
+ <systemPropertyVariables>
+ <tika.fuzz.timeout>${tika.fuzz.timeout}</tika.fuzz.timeout>
+ </systemPropertyVariables>
+ <!-- report the file and mode of a failing corpus replay, not its
index -->
+ <statelessTestsetReporter
implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
+ <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
+ </statelessTestsetReporter>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+</project>
diff --git
a/tika-fuzzing/src/test/java/org/apache/tika/fuzzing/AutoDetectParserFuzzTest.java
b/tika-fuzzing/src/test/java/org/apache/tika/fuzzing/AutoDetectParserFuzzTest.java
new file mode 100644
index 0000000000..c1cc4beea2
--- /dev/null
+++
b/tika-fuzzing/src/test/java/org/apache/tika/fuzzing/AutoDetectParserFuzzTest.java
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.fuzzing;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+import com.code_intelligence.jazzer.junit.FuzzTest;
+import org.xml.sax.SAXException;
+
+import org.apache.tika.exception.TikaException;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.parser.AutoDetectParser;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.RecursiveParserWrapper;
+import org.apache.tika.sax.BasicContentHandlerFactory;
+import org.apache.tika.sax.RecursiveParserWrapperHandler;
+
+/**
+ * AutoDetectParser over the three ways a caller hands Tika bytes. Findings
are Errors
+ * (OOM, stack overflow), hangs, and anything not a
TikaException/IOException/SAXException.
+ * With {@code -Dtika.fuzz.strict=true}, a parser RuntimeException that
CompositeParser
+ * wrapped in a TikaException is a finding too. A parse longer than {@code
tika.fuzz.timeout}
+ * seconds (default 60) is reported with the stuck thread's stack.
+ */
+class AutoDetectParserFuzzTest {
+
+ private static final Parser PARSER = new RecursiveParserWrapper(new
AutoDetectParser());
+ private static final boolean STRICT =
Boolean.getBoolean("tika.fuzz.strict");
+ private static final long TIMEOUT_SECONDS =
Long.getLong("tika.fuzz.timeout", 60);
+
+ private static ExecutorService worker;
+ private static volatile Thread workerThread;
+
+ @FuzzTest
+ void bytes(byte[] data) throws Exception {
+ try (TikaInputStream tis = TikaInputStream.get(data)) {
+ parse(tis);
+ }
+ }
+
+ @FuzzTest
+ void stream(byte[] data) throws Exception {
+ try (TikaInputStream tis = TikaInputStream.get(new
ByteArrayInputStream(data))) {
+ parse(tis);
+ }
+ }
+
+ @FuzzTest
+ void path(byte[] data) throws Exception {
+ Path tmp = Files.createTempFile("tika-fuzz-", ".bin");
+ try {
+ Files.write(tmp, data);
+ try (TikaInputStream tis = TikaInputStream.get(tmp)) {
+ parse(tis);
+ }
+ } finally {
+ Files.deleteIfExists(tmp);
+ }
+ }
+
+ // A JUnit timeout would also cap a whole fuzzing session, so each parse
is timed here.
+ static synchronized void parse(TikaInputStream tis) throws Exception {
+ if (worker == null) {
+ worker = Executors.newSingleThreadExecutor(r -> {
+ Thread t = new Thread(r, "tika-fuzz-parse");
+ t.setDaemon(true);
+ workerThread = t;
+ return t;
+ });
+ }
+ Future<Void> f = worker.submit(() -> {
+ parseInline(tis);
+ return null;
+ });
+ try {
+ f.get(TIMEOUT_SECONDS, TimeUnit.SECONDS);
+ } catch (TimeoutException e) {
+ // a spinning parse ignores interrupts; abandon its thread
+ Thread stuck = workerThread;
+ worker = null;
+ AssertionError hang = new AssertionError(
+ "parse did not finish in " + TIMEOUT_SECONDS + " s");
+ hang.setStackTrace(stuck.getStackTrace());
+ throw hang;
+ } catch (ExecutionException e) {
+ if (e.getCause() instanceof Error err) {
+ throw err;
+ }
+ throw (Exception) e.getCause();
+ }
+ }
+
+ private static void parseInline(TikaInputStream tis) throws Exception {
+ RecursiveParserWrapperHandler handler = new
RecursiveParserWrapperHandler(
+ new
BasicContentHandlerFactory(BasicContentHandlerFactory.HANDLER_TYPE.TEXT, -1));
+ try {
+ PARSER.parse(tis, handler, new Metadata(), new ParseContext());
+ } catch (TikaException e) {
+ if (STRICT && e.getCause() instanceof RuntimeException) {
+ throw e;
+ }
+ } catch (IOException | SAXException e) {
+ // expected on malformed input
+ }
+ }
+}
diff --git
a/tika-fuzzing/src/test/java/org/apache/tika/fuzzing/CorpusReplayTest.java
b/tika-fuzzing/src/test/java/org/apache/tika/fuzzing/CorpusReplayTest.java
new file mode 100644
index 0000000000..d664376475
--- /dev/null
+++ b/tika-fuzzing/src/test/java/org/apache/tika/fuzzing/CorpusReplayTest.java
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.fuzzing;
+
+import static org.junit.jupiter.api.Assumptions.assumeTrue;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.stream.Stream;
+
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import org.apache.tika.io.TikaInputStream;
+
+/**
+ * Replays every file under {@code -Dtika.fuzz.corpus=<dir>} through the same
parse as
+ * {@link AutoDetectParserFuzzTest}, once per input mode. Skipped when the
property is unset.
+ */
+class CorpusReplayTest {
+
+ private static final String CORPUS =
System.getProperty("tika.fuzz.corpus", "");
+
+ enum Mode { BYTES, STREAM, PATH }
+
+ static Stream<Arguments> files() throws IOException {
+ if (CORPUS.isEmpty()) {
+ return Stream.of(Arguments.of(Mode.BYTES, null));
+ }
+ Path root = Paths.get(CORPUS);
+ // collect eagerly: Files.walk holds a directory handle until closed
+ return
Files.walk(root).filter(Files::isRegularFile).sorted().toList().stream()
+ .flatMap(p -> Stream.of(Mode.values()).map(m ->
Arguments.of(m, p)));
+ }
+
+ @ParameterizedTest(name = "{0} {1}")
+ @MethodSource("files")
+ void replay(Mode mode, Path file) throws Exception {
+ assumeTrue(file != null, "set -Dtika.fuzz.corpus=<dir> to replay a
corpus");
+ try (TikaInputStream tis = switch (mode) {
+ case BYTES -> TikaInputStream.get(Files.readAllBytes(file));
+ case STREAM -> TikaInputStream.get(Files.newInputStream(file));
+ case PATH -> TikaInputStream.get(file);
+ }) {
+ AutoDetectParserFuzzTest.parse(tis);
+ }
+ }
+}
diff --git a/tika-fuzzing/src/test/resources/junit-platform.properties
b/tika-fuzzing/src/test/resources/junit-platform.properties
new file mode 100644
index 0000000000..6ce0abcdf8
--- /dev/null
+++ b/tika-fuzzing/src/test/resources/junit-platform.properties
@@ -0,0 +1,18 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Jazzer instruments only directory classes by default; parsers come from jars.
+jazzer.instrument=org.apache.tika.**
