This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tika.git
The following commit(s) were added to refs/heads/main by this push:
new b5c52d4426 TIKA-4877: improve tmp file handling (#3125)
b5c52d4426 is described below
commit b5c52d4426705e275fc6b062544ec2575bd9e087
Author: Tim Allison <[email protected]>
AuthorDate: Thu Sep 3 17:20:14 2026 -0400
TIKA-4877: improve tmp file handling (#3125)
---
CHANGES.txt | 11 +
.../migration-to-4x/migrating-tika-server-4x.adoc | 2 +-
docs/modules/ROOT/pages/pipes/configuration.adoc | 35 ++-
.../microsoft/rtf/jflex/RTFEmbeddedHandler.java | 12 +-
.../microsoft/rtf/jflex/RTFHtmlDecapsulator.java | 243 +++++++++++----------
.../rtf/jflex/RTFHtmlDecapsulatorTest.java | 30 +++
.../tika/renderer/pdf/pdfbox/PDFBoxRenderer.java | 8 +
.../tika/renderer/pdf/poppler/PopplerRenderer.java | 14 +-
.../renderer/pdf/pdfbox/PDFBoxRendererTest.java | 36 +++
.../tika/pipes/core/PerClientServerManager.java | 7 +-
.../org/apache/tika/pipes/core/PipesConfig.java | 41 ++--
.../tika/pipes/core/SharedServerManager.java | 7 +-
.../apache/tika/pipes/core/server/PipesServer.java | 35 ++-
.../pipes/core/server/PipesServerTempDirTest.java | 49 +++++
.../org/apache/tika/config/loader/TikaLoader.java | 30 +++
.../apache/tika/config/loader/TikaLoaderTest.java | 22 ++
.../language/translate/impl/MarianTranslator.java | 33 ++-
17 files changed, 452 insertions(+), 163 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 2e739e0f21..fe2cc602cb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,16 @@
Release 4.1.0 - unreleased
+ * Temp files follow -Djava.io.tmpdir on the parent JVM (Tika, its
+ libraries, and forks all honor it); TikaLoader fails at config load
+ if it is unusable. pipes.tempDirectory is deprecated for removal in
+ 5.0: it only covered the forks. Do not use tmpfs: spool size is
+ bounded by input, and an orphaned fork dir pins RAM (TIKA-4877).
+
+ * A fork whose parent dies deletes its own temp dir; the parent
+ surfaces a fork's hs_err log before every delete. Failure-path temp
+ file leaks fixed in PDFBoxRenderer, PopplerRenderer, truncated RTF,
+ and MarianTranslator (TIKA-4877).
+
* tika-eval Profile/Compare speedups: single-pass URL/mail stripping
replaces the bounded regexes in langdetect preprocessing (same output,
17-290x faster on web text), the default H2 db URL drops MVStore chunk
diff --git
a/docs/modules/ROOT/pages/migration-to-4x/migrating-tika-server-4x.adoc
b/docs/modules/ROOT/pages/migration-to-4x/migrating-tika-server-4x.adoc
index e1859febae..ac4afd013f 100644
--- a/docs/modules/ROOT/pages/migration-to-4x/migrating-tika-server-4x.adoc
+++ b/docs/modules/ROOT/pages/migration-to-4x/migrating-tika-server-4x.adoc
@@ -576,7 +576,7 @@ xref:pipes/cpu-sizing.adoc[Forked-JVM CPU and Heap Sizing].
=== Performance Optimizations
* TCP_NODELAY enabled for reduced latency on small requests
-* Configurable temp directory for RAM disk optimization (`pipes.tempDirectory`)
+* Temporary files go to `-Djava.io.tmpdir`; forks get private subdirectories
under it (see xref:pipes/configuration.adoc#_where_temporary_files_go[Where
temporary files go]; `pipes.tempDirectory` is deprecated, and do not use a RAM
disk)
== Advanced: Shared Server Mode
diff --git a/docs/modules/ROOT/pages/pipes/configuration.adoc
b/docs/modules/ROOT/pages/pipes/configuration.adoc
index fb1c205117..d8691bf7a8 100644
--- a/docs/modules/ROOT/pages/pipes/configuration.adoc
+++ b/docs/modules/ROOT/pages/pipes/configuration.adoc
@@ -57,14 +57,37 @@ how many forked JVMs to run, timeouts, memory management,
and parse behavior.
|`tempDirectory`
|_system default_
-|Directory for temporary files. Each fork gets a subdirectory here, and the
fork's whole `java.io.tmpdir` points at it, so anything the fork spools while
parsing, unpacked embedded files and JVM crash logs all land inside. It does
*not* cover the host side: tika-server spools over-threshold request bodies
into its own input temp directory, and `PipesForkParser` into the calling JVM's
`java.io.tmpdir`. Consider a RAM-backed filesystem (e.g., `/dev/shm`) for
better performance -- but see t [...]
+|*Deprecated since 4.1, removal planned for 5.0.* Set `-Djava.io.tmpdir` on
the parent JVM instead; see below. Still honored: each fork gets a subdirectory
here, and the fork's whole `java.io.tmpdir` points at it. It never covered the
host side -- tika-server's request-body spool and `PipesForkParser` use the
parent's `java.io.tmpdir` -- nor any library that calls `File.createTempFile`
itself.
|===
-The parent deletes a fork's subdirectory when that fork is torn down or fails
to start, so a
-crashing fork does not accumulate them. A parent killed abruptly (`SIGKILL`,
container stop)
-cannot, and its subdirectories survive. On a RAM-backed filesystem those leaks
consume memory
-rather than disk, and `/dev/shm` is commonly sized at half of RAM -- so if you
point
-`tempDirectory` at one, sweep it on service start.
+=== Where temporary files go
+
+Set `-Djava.io.tmpdir` on the parent JVM. It has to be a launch flag because
of Tika's
+dependencies: POI, PDFBox and the rest create temp files through the JDK,
which reads
+`java.io.tmpdir` once at JVM start, so nothing Tika sets at runtime reaches
them.
+Tika, every library it calls, and its forked pipes servers all honor the flag:
each fork gets a private subdirectory under it (its whole
+`java.io.tmpdir` points there, so spooled input, unpacked embedded files and
JVM crash logs
+all land inside), and the parent deletes that subdirectory when the fork is
torn down.
+Tika checks the directory exists and is writable at config load and refuses to
start
+otherwise, rather than failing on the first document that spools.
+
+If the parent dies, a surviving fork deletes its own subdirectory as it exits.
Only when the
+whole process family is killed at once (`SIGKILL` of the group, container
stop) do
+subdirectories survive, with whatever the forks were spooling inside; Tika
never deletes
+directories another process created. Point `java.io.tmpdir` at a disk-backed
directory on a
+volume where filling it does not take out the OS, and apply your own retention
to
+`pipes-server-*` there.
+
+[WARNING]
+====
+*DO NOT USE tmpfs* (`/dev/shm`, any RAM-backed filesystem) for
`java.io.tmpdir`.
+
+Spool size is bounded by the input, not by any Tika setting: one large archive
expanding
+into tmpfs can exhaust memory for every process on the host or get a container
evicted,
+and a fork directory orphaned by a killed parent pins that RAM until someone
deletes it.
+A slow run is recoverable; a lost host is not. If you must, give tmpfs a
dedicated mount
+with `size=` so the failure is a refused write rather than a lost host.
+====
=== Embedded-object cache memory budget
diff --git
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFEmbeddedHandler.java
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFEmbeddedHandler.java
index 9889a4d245..e5ab1241f4 100644
---
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFEmbeddedHandler.java
+++
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFEmbeddedHandler.java
@@ -16,6 +16,7 @@
*/
package org.apache.tika.parser.microsoft.rtf.jflex;
+import java.io.Closeable;
import java.io.IOException;
import java.util.concurrent.atomic.AtomicInteger;
@@ -26,6 +27,7 @@ import org.xml.sax.SAXException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.extractor.EmbeddedDocumentExtractor;
import org.apache.tika.extractor.EmbeddedDocumentUtil;
+import org.apache.tika.io.TemporaryResources;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.HttpHeaders;
import org.apache.tika.metadata.Metadata;
@@ -41,7 +43,7 @@ import org.apache.tika.sax.EmbeddedContentHandler;
* {@link RTFPictStreamParser}) so that large embedded objects are written
* to temp files rather than buffered entirely in memory.</p>
*/
-public class RTFEmbeddedHandler {
+public class RTFEmbeddedHandler implements Closeable {
private final ContentHandler handler;
private final ParseContext context;
@@ -254,4 +256,12 @@ public class RTFEmbeddedHandler {
return 10 + (ch - 'A');
}
}
+
+ /** Closes a stream parser left open by a document that ended inside its
group. */
+ @Override
+ public void close() throws IOException {
+ TemporaryResources.closeAll(objParser, pictParser);
+ objParser = null;
+ pictParser = null;
+ }
}
diff --git
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFHtmlDecapsulator.java
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFHtmlDecapsulator.java
index 7ef06f1ebe..cf4c9f729e 100644
---
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFHtmlDecapsulator.java
+++
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFHtmlDecapsulator.java
@@ -75,150 +75,155 @@ public class RTFHtmlDecapsulator {
boolean inHtmlTag = false;
RTFToken tok;
- while ((tok = tokenizer.yylex()) != null) {
- RTFTokenType type = tok.getType();
- if (type == RTFTokenType.EOF) {
- break;
- }
+ // EOF inside {\object or {\pict leaves a stream parser open
+ try {
+ while ((tok = tokenizer.yylex()) != null) {
+ RTFTokenType type = tok.getType();
+ if (type == RTFTokenType.EOF) {
+ break;
+ }
- // Flush pending bytes before charset-changing events
- if (type == RTFTokenType.GROUP_CLOSE
- || (type == RTFTokenType.CONTROL_WORD &&
"f".equals(tok.getName())
- && tok.hasParameter())) {
- flushPendingBytes(pendingBytes, html, state);
- }
+ // Flush pending bytes before charset-changing events
+ if (type == RTFTokenType.GROUP_CLOSE
+ || (type == RTFTokenType.CONTROL_WORD &&
"f".equals(tok.getName())
+ && tok.hasParameter())) {
+ flushPendingBytes(pendingBytes, html, state);
+ }
- boolean consumed = state.processToken(tok);
+ boolean consumed = state.processToken(tok);
- // Embedded handler processes objdata/pict/sp in the same pass
- if (!consumed) {
- RTFGroupState closingGroup =
- (type == RTFTokenType.GROUP_CLOSE) ?
state.getLastClosedGroup() : null;
- try {
- embHandler.processToken(tok, state, closingGroup);
- } catch (TikaException | IOException e) {
- // don't let a bad embedded object kill decapsulation
+ // Embedded handler processes objdata/pict/sp in the same pass
+ if (!consumed) {
+ RTFGroupState closingGroup =
+ (type == RTFTokenType.GROUP_CLOSE) ?
state.getLastClosedGroup() : null;
+ try {
+ embHandler.processToken(tok, state, closingGroup);
+ } catch (TikaException | IOException e) {
+ // don't let a bad embedded object kill decapsulation
+ }
}
- }
- RTFGroupState group = state.getCurrentGroup();
+ RTFGroupState group = state.getCurrentGroup();
- // Skip tokens that are part of objdata/pict hex streams
- if (!consumed && (group.objdata || group.pictDepth > 0)) {
- continue;
- }
-
- switch (type) {
- case GROUP_OPEN:
- sawIgnorable = false;
- break;
-
- case GROUP_CLOSE:
- if (inHtmlTag && state.getDepth() < htmlTagDepth) {
- flushPendingBytes(pendingBytes, html, state);
- inHtmlTag = false;
- htmlTagDepth = -1;
- }
- break;
+ // Skip tokens that are part of objdata/pict hex streams
+ if (!consumed && (group.objdata || group.pictDepth > 0)) {
+ continue;
+ }
- case CONTROL_SYMBOL:
- if (tok.getChar() == '*') {
- sawIgnorable = true;
- }
- if (!foundHtmlTag || inHtmlRtfSkip) {
+ switch (type) {
+ case GROUP_OPEN:
+ sawIgnorable = false;
break;
- }
- if (inHtmlTag || htmlTagDepth == -1) {
- char sym = tok.getChar();
- if (sym == '{' || sym == '}' || sym == '\\') {
+
+ case GROUP_CLOSE:
+ if (inHtmlTag && state.getDepth() < htmlTagDepth) {
flushPendingBytes(pendingBytes, html, state);
- html.append(sym);
+ inHtmlTag = false;
+ htmlTagDepth = -1;
}
- }
- break;
-
- case CONTROL_WORD:
- if (consumed) {
break;
- }
- String name = tok.getName();
- if ("fromhtml".equals(name)) {
- foundFromHtml = true;
- break;
- }
- if ("htmltag".equals(name) && sawIgnorable) {
- if (!foundFromHtml) {
+ case CONTROL_SYMBOL:
+ if (tok.getChar() == '*') {
+ sawIgnorable = true;
+ }
+ if (!foundHtmlTag || inHtmlRtfSkip) {
break;
}
- foundHtmlTag = true;
- flushPendingBytes(pendingBytes, html, state);
- inHtmlTag = true;
- htmlTagDepth = state.getDepth();
- break;
- }
- if ("htmlrtf".equals(name)) {
- flushPendingBytes(pendingBytes, html, state);
- inHtmlRtfSkip = !(tok.hasParameter() &&
tok.getParameter() == 0);
- break;
- }
- if (!foundHtmlTag || inHtmlRtfSkip) {
+ if (inHtmlTag || htmlTagDepth == -1) {
+ char sym = tok.getChar();
+ if (sym == '{' || sym == '}' || sym == '\\') {
+ flushPendingBytes(pendingBytes, html, state);
+ html.append(sym);
+ }
+ }
break;
- }
- if (inHtmlTag || htmlTagDepth == -1) {
- flushPendingBytes(pendingBytes, html, state);
- switch (name) {
- case "par":
- case "pard":
- html.append('\n');
- break;
- case "tab":
- html.append('\t');
- break;
- case "line":
- html.append("<br>");
- break;
- default:
- break;
+
+ case CONTROL_WORD:
+ if (consumed) {
+ break;
}
- }
- break;
+ String name = tok.getName();
- case HEX_ESCAPE:
- if (consumed || !foundHtmlTag || inHtmlRtfSkip) {
+ if ("fromhtml".equals(name)) {
+ foundFromHtml = true;
+ break;
+ }
+ if ("htmltag".equals(name) && sawIgnorable) {
+ if (!foundFromHtml) {
+ break;
+ }
+ foundHtmlTag = true;
+ flushPendingBytes(pendingBytes, html, state);
+ inHtmlTag = true;
+ htmlTagDepth = state.getDepth();
+ break;
+ }
+ if ("htmlrtf".equals(name)) {
+ flushPendingBytes(pendingBytes, html, state);
+ inHtmlRtfSkip = !(tok.hasParameter() &&
tok.getParameter() == 0);
+ break;
+ }
+ if (!foundHtmlTag || inHtmlRtfSkip) {
+ break;
+ }
+ if (inHtmlTag || htmlTagDepth == -1) {
+ flushPendingBytes(pendingBytes, html, state);
+ switch (name) {
+ case "par":
+ case "pard":
+ html.append('\n');
+ break;
+ case "tab":
+ html.append('\t');
+ break;
+ case "line":
+ html.append("<br>");
+ break;
+ default:
+ break;
+ }
+ }
break;
- }
- if (inHtmlTag || htmlTagDepth == -1) {
- pendingBytes.write(tok.getHexValue());
- }
- break;
- case UNICODE_ESCAPE:
- if (!foundHtmlTag || inHtmlRtfSkip) {
+ case HEX_ESCAPE:
+ if (consumed || !foundHtmlTag || inHtmlRtfSkip) {
+ break;
+ }
+ if (inHtmlTag || htmlTagDepth == -1) {
+ pendingBytes.write(tok.getHexValue());
+ }
break;
- }
- if (inHtmlTag || htmlTagDepth == -1) {
- flushPendingBytes(pendingBytes, html, state);
- int cp = tok.getParameter();
- if (Character.isValidCodePoint(cp)) {
- html.appendCodePoint(cp);
+
+ case UNICODE_ESCAPE:
+ if (!foundHtmlTag || inHtmlRtfSkip) {
+ break;
}
- }
- break;
+ if (inHtmlTag || htmlTagDepth == -1) {
+ flushPendingBytes(pendingBytes, html, state);
+ int cp = tok.getParameter();
+ if (Character.isValidCodePoint(cp)) {
+ html.appendCodePoint(cp);
+ }
+ }
+ break;
- case TEXT:
- if (consumed || !foundHtmlTag || inHtmlRtfSkip) {
+ case TEXT:
+ if (consumed || !foundHtmlTag || inHtmlRtfSkip) {
+ break;
+ }
+ if (inHtmlTag || htmlTagDepth == -1) {
+ flushPendingBytes(pendingBytes, html, state);
+ html.append(tok.getChar());
+ }
break;
- }
- if (inHtmlTag || htmlTagDepth == -1) {
- flushPendingBytes(pendingBytes, html, state);
- html.append(tok.getChar());
- }
- break;
- default:
- break;
+ default:
+ break;
+ }
}
+ } finally {
+ embHandler.close();
}
flushPendingBytes(pendingBytes, html, state);
diff --git
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFHtmlDecapsulatorTest.java
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFHtmlDecapsulatorTest.java
index 72235f36a3..9d22a40935 100644
---
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFHtmlDecapsulatorTest.java
+++
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/java/org/apache/tika/parser/microsoft/rtf/jflex/RTFHtmlDecapsulatorTest.java
@@ -23,8 +23,15 @@ import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.IOException;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.HashSet;
+import java.util.Set;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.parallel.Isolated;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;
@@ -35,6 +42,7 @@ import org.apache.tika.parser.ParseContext;
* Tests for {@link RTFHtmlDecapsulator}, mirroring the original
* RTFEncapsulatedHTMLExtractorTest to verify parity.
*/
+@Isolated // testTruncatedPictLeavesNoTempFile snapshots java.io.tmpdir
public class RTFHtmlDecapsulatorTest {
private static String extract(byte[] rtfBytes)
@@ -257,4 +265,26 @@ public class RTFHtmlDecapsulatorTest {
assertNotNull(html);
assertEquals("A\u2014B", html);
}
+
+ /** EOF inside {\pict: the temp file already exists and no group close
will hand it off. */
+ @Test
+ public void testTruncatedPictLeavesNoTempFile() throws Exception {
+ String rtf = "{\\rtf1\\ansi\\ansicpg1252\\fromhtml1
\\deff0{\\pict\\wmetafile8 0102";
+ Set<String> before = tikaTempEntries();
+ extract(rtf.getBytes(US_ASCII));
+ Set<String> leaked = tikaTempEntries();
+ leaked.removeAll(before);
+ assertTrue(leaked.isEmpty(), "temp files left behind: " + leaked);
+ }
+
+ private static Set<String> tikaTempEntries() throws IOException {
+ Set<String> names = new HashSet<>();
+ Path tmpDir = Paths.get(System.getProperty("java.io.tmpdir"));
+ try (DirectoryStream<Path> entries = Files.newDirectoryStream(tmpDir,
"apache-tika-*.bin")) {
+ for (Path p : entries) {
+ names.add(p.getFileName().toString());
+ }
+ }
+ return names;
+ }
}
diff --git
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRenderer.java
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRenderer.java
index 3e26a13dda..17d194abea 100644
---
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRenderer.java
+++
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRenderer.java
@@ -130,6 +130,14 @@ public class PDFBoxRenderer implements PDDocumentRenderer {
for (RenderRequest renderRequest : requests) {
processRequest(renderRequest, pdDocument, metadata,
parseContext, results);
}
+ } catch (Throwable t) {
+ // results never reach the caller; nothing else would delete the
pages
+ try {
+ results.close();
+ } catch (IOException e) {
+ t.addSuppressed(e);
+ }
+ throw t;
} finally {
if (mustClose) {
pdDocument.close();
diff --git
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/poppler/PopplerRenderer.java
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/poppler/PopplerRenderer.java
index 9e6ac64ac7..714cb00a6c 100644
---
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/poppler/PopplerRenderer.java
+++
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/poppler/PopplerRenderer.java
@@ -114,8 +114,18 @@ public class PopplerRenderer implements Renderer {
TemporaryResources tmp = new TemporaryResources();
PageBasedRenderResults results = new PageBasedRenderResults(tmp);
Path path = tis.getPath();
- for (RenderRequest request : requests) {
- renderRequest(path, metadata, parseContext, request, results, tmp);
+ try {
+ for (RenderRequest request : requests) {
+ renderRequest(path, metadata, parseContext, request, results,
tmp);
+ }
+ } catch (Throwable t) {
+ // results never reach the caller; nothing else would delete the
pages
+ try {
+ results.close();
+ } catch (IOException e) {
+ t.addSuppressed(e);
+ }
+ throw t;
}
return results;
}
diff --git
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRendererTest.java
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRendererTest.java
index 0bcf1dc396..a2de6211f9 100644
---
a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRendererTest.java
+++
b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRendererTest.java
@@ -18,11 +18,20 @@ package org.apache.tika.renderer.pdf.pdfbox;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
+import java.io.IOException;
import java.io.InputStream;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.HashSet;
+import java.util.Set;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.parallel.Isolated;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
@@ -32,6 +41,7 @@ import org.apache.tika.renderer.PageBasedRenderResults;
import org.apache.tika.renderer.PageRangeRequest;
import org.apache.tika.renderer.RenderResult;
+@Isolated // testFailedRenderLeavesNoTempFiles snapshots java.io.tmpdir
public class PDFBoxRendererTest {
private long renderedPngBytes(ParseContext context) throws Exception {
@@ -72,4 +82,30 @@ public class PDFBoxRendererTest {
assertTrue(uncompressed > compressed * 4,
"quality 1.0 should be far larger: " + uncompressed + " vs " +
compressed);
}
+
+ /** An out-of-range page throws past the per-page IOException catch after
RENDER_ALL wrote pages. */
+ @Test
+ public void testFailedRenderLeavesNoTempFiles() throws Exception {
+ PDFBoxRenderer renderer = new PDFBoxRenderer();
+ Set<String> before = tikaTempEntries();
+ try (InputStream is =
getClass().getResourceAsStream("/test-documents/testPDF.pdf");
+ TikaInputStream tis = TikaInputStream.get(is)) {
+ assertThrows(RuntimeException.class, () -> renderer.render(tis,
new Metadata(),
+ new ParseContext(), PageRangeRequest.RENDER_ALL, new
PageRangeRequest(9999, 9999)));
+ }
+ Set<String> leaked = tikaTempEntries();
+ leaked.removeAll(before);
+ assertTrue(leaked.isEmpty(), "temp files left behind: " + leaked);
+ }
+
+ private static Set<String> tikaTempEntries() throws IOException {
+ Set<String> names = new HashSet<>();
+ Path tmpDir = Paths.get(System.getProperty("java.io.tmpdir"));
+ try (DirectoryStream<Path> entries = Files.newDirectoryStream(tmpDir,
"tika-pdfbox-rendering-*")) {
+ for (Path p : entries) {
+ names.add(p.getFileName().toString());
+ }
+ }
+ return names;
+ }
}
diff --git
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PerClientServerManager.java
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PerClientServerManager.java
index 103cb397a3..a30d3e24a0 100644
---
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PerClientServerManager.java
+++
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PerClientServerManager.java
@@ -415,7 +415,6 @@ public class PerClientServerManager implements
ServerManager {
} else {
LOG.error("clientId={}: Process exited with code {}
before connecting to socket",
clientId, exitValue);
- ServerProcessIO.surfaceCrashDiagnostics(LOG,
"clientId=" + clientId, tmpDir);
}
// Always treat pre-connect death as retryable.
// The only non-retryable paths are:
@@ -430,7 +429,6 @@ public class PerClientServerManager implements
ServerManager {
long elapsed = System.currentTimeMillis() - startTime;
if (elapsed > SOCKET_CONNECT_TIMEOUT_MS) {
LOG.error("clientId={}: Timed out waiting for server to
connect after {}ms", clientId, elapsed);
- ServerProcessIO.surfaceCrashDiagnostics(LOG, "clientId=" +
clientId, tmpDir);
throw new ServerInitializationException(
"Server did not connect within " +
SOCKET_CONNECT_TIMEOUT_MS + "ms");
}
@@ -439,6 +437,7 @@ public class PerClientServerManager implements
ServerManager {
}
}
+ @SuppressWarnings("deprecation")
private synchronized void startServer() throws IOException,
InterruptedException, TimeoutException, ServerInitializationException {
if (closed) {
throw new IllegalStateException("PerClientServerManager is
closed");
@@ -457,7 +456,7 @@ public class PerClientServerManager implements
ServerManager {
LOG.trace("clientId={}: starting server on port={}", clientId, port);
- tmpDir = pipesConfig.createTempDirectory("pipes-server-" + clientId +
"-");
+ tmpDir = pipesConfig.createTempDirectory(PipesServer.TEMP_DIR_PREFIX +
clientId + "-");
ProcessBuilder pb = new ProcessBuilder(getCommandline(tmpDir));
// Tell the child our PID so it can watch ProcessHandle.onExit() and
// self-terminate promptly if we die. Without this, an orphan child
@@ -536,6 +535,8 @@ public class PerClientServerManager implements
ServerManager {
destroyProcess();
if (tmpDir != null) {
+ // never delete a crash log unread
+ ServerProcessIO.surfaceCrashDiagnostics(LOG, "clientId=" +
clientId, tmpDir);
deleteDir(tmpDir);
tmpDir = null;
}
diff --git
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PipesConfig.java
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PipesConfig.java
index ce73c844f4..c253a167e7 100644
---
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PipesConfig.java
+++
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PipesConfig.java
@@ -28,6 +28,8 @@ import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.util.StdConverter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import org.apache.tika.config.ExceptionReporting;
import org.apache.tika.config.TimeoutLimits;
@@ -44,6 +46,8 @@ import org.apache.tika.utils.StringUtils;
@JsonDeserialize(converter = PipesConfig.PostDeserializationCheck.class)
public class PipesConfig {
+ private static final Logger LOG =
LoggerFactory.getLogger(PipesConfig.class);
+
/** Runs {@link #checkPayloadLimits()} on every Jackson deserialization
path. */
public static class PostDeserializationCheck extends
StdConverter<PipesConfig, PipesConfig> {
@Override
@@ -166,11 +170,8 @@ public class PipesConfig {
private ArrayList<String> forkedJvmArgs = new ArrayList<>();
private String javaPath = "java";
- /**
- * Optional directory for temporary files during pipes-based parsing.
- * If not set, the system default temp directory will be used.
- * Consider using a RAM-backed filesystem (e.g., /dev/shm) for better
performance.
- */
+ /** @deprecated since 4.1, removal planned for 5.0; see {@link
#setTempDirectory(String)} */
+ @Deprecated
private String tempDirectory = null;
/**
@@ -528,11 +529,8 @@ public class PipesConfig {
this.configStoreParams = configStoreParams;
}
- /**
- * Gets the directory for temporary files during pipes-based parsing.
- *
- * @return the temp directory path, or null to use system default
- */
+ /** @deprecated since 4.1, removal planned for 5.0; see {@link
#setTempDirectory(String)} */
+ @Deprecated
public String getTempDirectory() {
return tempDirectory;
}
@@ -541,7 +539,10 @@ public class PipesConfig {
* Creates a temp directory under {@link #getTempDirectory()}, or under
the system default
* when unset. Callers must not use {@code Files.createTempDirectory}
directly or the
* configured directory is silently ignored.
+ *
+ * @deprecated since 4.1, removal planned for 5.0; use {@code
Files.createTempDirectory(prefix)}
*/
+ @Deprecated
public Path createTempDirectory(String prefix) throws IOException {
if (StringUtils.isBlank(tempDirectory)) {
return Files.createTempDirectory(prefix);
@@ -552,13 +553,25 @@ public class PipesConfig {
}
/**
- * Sets the directory for temporary files during pipes-based parsing.
- * If not set, the system default temp directory will be used.
- * Consider using a RAM-backed filesystem (e.g., /dev/shm or /tmpfs) for
better performance.
+ * Directory the forks create their private temp dirs under. It only ever
covered the
+ * forks: this JVM's own spooling and every library calling {@code
File.createTempFile}
+ * use {@code java.io.tmpdir}, fixed at JVM start. Set {@code
-Djava.io.tmpdir} on the
+ * parent JVM instead; forks inherit subdirectories under it.
+ * <p>
+ * <b>DO NOT USE tmpfs</b> ({@code /dev/shm}) here or for {@code
java.io.tmpdir}: spool
+ * size is bounded by input, not config, and a fork dir orphaned by a
killed parent pins
+ * RAM until deleted.
*
- * @param tempDirectory the temp directory path, or null to use system
default
+ * @deprecated since 4.1, removal planned for 5.0; set {@code
-Djava.io.tmpdir} instead
*/
+ @Deprecated
public void setTempDirectory(String tempDirectory) {
+ if (!StringUtils.isBlank(tempDirectory)) {
+ LOG.warn("pipes.tempDirectory={} is deprecated (removal in 5.0)
and only redirects " +
+ "the forks; this JVM's dependencies (POI, PDFBox, ...)
write to " +
+ "java.io.tmpdir={}, fixed at JVM start. Set
-Djava.io.tmpdir on the parent JVM.",
+ tempDirectory, System.getProperty("java.io.tmpdir"));
+ }
this.tempDirectory = tempDirectory;
}
diff --git
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/SharedServerManager.java
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/SharedServerManager.java
index a4a0478ae9..c0a8088d2c 100644
---
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/SharedServerManager.java
+++
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/SharedServerManager.java
@@ -295,6 +295,7 @@ public class SharedServerManager implements ServerManager {
}
}
+ @SuppressWarnings("deprecation")
private void startServer() throws IOException, InterruptedException,
TimeoutException, ServerInitializationException {
// Clean up any previous server
if (process != null || tmpDir != null) {
@@ -333,7 +334,7 @@ public class SharedServerManager implements ServerManager {
" in-flight parses as collateral damage.\n");
LOG.info("Starting shared server with {} connections", numConnections);
- tmpDir = pipesConfig.createTempDirectory("pipes-shared-server-");
+ tmpDir =
pipesConfig.createTempDirectory(PipesServer.SHARED_TEMP_DIR_PREFIX);
ProcessBuilder pb = new ProcessBuilder(getCommandline());
// Pass port and auth token via environment variables so they are not
// visible in /proc/<pid>/cmdline. The token is only readable via
@@ -397,7 +398,6 @@ public class SharedServerManager implements ServerManager {
if (!process.isAlive()) {
int exitValue = process.exitValue();
LOG.error("Shared server process exited with code {}
before becoming ready", exitValue);
- ServerProcessIO.surfaceCrashDiagnostics(LOG,
"shared-server", tmpDir);
throw new ServerInitializationException(
"Shared server failed to start (exit code " +
exitValue + "). Check JVM arguments and classpath.");
}
@@ -406,7 +406,6 @@ public class SharedServerManager implements ServerManager {
long elapsed = System.currentTimeMillis() - startTime;
if (elapsed > STARTUP_TIMEOUT_MILLIS) {
LOG.error("Timed out waiting for shared server to start
after {}ms", elapsed);
- ServerProcessIO.surfaceCrashDiagnostics(LOG,
"shared-server", tmpDir);
destroyProcessUnsafe();
throw new ServerInitializationException(
"Shared server did not start within " +
STARTUP_TIMEOUT_MILLIS + "ms");
@@ -452,6 +451,8 @@ public class SharedServerManager implements ServerManager {
destroyProcessUnsafe();
if (tmpDir != null) {
+ // never delete a crash log unread
+ ServerProcessIO.surfaceCrashDiagnostics(LOG, "shared-server",
tmpDir);
deleteDir(tmpDir);
tmpDir = null;
}
diff --git
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/server/PipesServer.java
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/server/PipesServer.java
index dd545e38bb..a8a0a2d277 100644
---
a/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/server/PipesServer.java
+++
b/tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/server/PipesServer.java
@@ -41,6 +41,7 @@ import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
+import org.apache.commons.io.FileUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.xml.sax.SAXException;
@@ -158,6 +159,10 @@ public class PipesServer implements AutoCloseable {
* process handle and exit promptly if the parent dies. */
public static final String PARENT_PID_ENV = "TIKA_PIPES_PARENT_PID";
+ /** Prefixes of the temp dirs the parent creates for forks; a fork deletes
only a dir so named. */
+ public static final String TEMP_DIR_PREFIX = "pipes-server-";
+ public static final String SHARED_TEMP_DIR_PREFIX = "pipes-shared-server-";
+
/** Exit code used when the child self-terminates because its parent JVM
* disappeared. Distinct from UNSPECIFIED_CRASH (19) so log readers can
* tell the difference between "I crashed" and "my parent went away". */
@@ -737,17 +742,43 @@ public class PipesServer implements AutoCloseable {
if (parent.isEmpty()) {
LOG.error("parent pid {} not found at startup; exiting to avoid
orphan",
parentPid);
- System.exit(PARENT_GONE_EXIT_CODE);
+ exitParentGone();
return;
}
parent.get().onExit().thenRun(() -> {
LOG.error("parent pid {} exited; shutting down to avoid orphan",
parentPid);
- System.exit(PARENT_GONE_EXIT_CODE);
+ exitParentGone();
});
LOG.info("watching parent pid {} for exit", parentPid);
}
+ /** Only when the parent is gone: on the fork's own crash the dir must
survive for the parent to read. */
+ private static void exitParentGone() {
+ try {
+ deleteOwnTempDir(Paths.get(System.getProperty("java.io.tmpdir")));
+ } finally {
+ System.exit(PARENT_GONE_EXIT_CODE);
+ }
+ }
+
+ /** @return true if {@code dir} is parent-created (by name) and is now
deleted */
+ static boolean deleteOwnTempDir(Path dir) {
+ String name = dir.getFileName() == null ? "" :
dir.getFileName().toString();
+ if (!name.startsWith(TEMP_DIR_PREFIX) &&
!name.startsWith(SHARED_TEMP_DIR_PREFIX)) {
+ LOG.warn("java.io.tmpdir={} was not created by a parent manager;
leaving it", dir);
+ return false;
+ }
+ try {
+ FileUtils.deleteDirectory(dir.toFile());
+ LOG.info("deleted own temp dir {}", dir);
+ return true;
+ } catch (IOException e) {
+ LOG.warn("couldn't delete own temp dir {}: {}", dir, e.toString());
+ return false;
+ }
+ }
+
/** Below this, ordinary documents -- not just pathological ones -- start
OOMing. */
private static final long MIN_USABLE_HEAP_BYTES = 256L * 1024 * 1024;
diff --git
a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/server/PipesServerTempDirTest.java
b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/server/PipesServerTempDirTest.java
new file mode 100644
index 0000000000..39b7e0033e
--- /dev/null
+++
b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/server/PipesServerTempDirTest.java
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.pipes.core.server;
+
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+public class PipesServerTempDirTest {
+
+ /** Only a parent-named dir is deleted; the system temp dir a
hand-launched server sees is not. */
+ @Test
+ public void testDeletesOnlyParentCreatedDir(@TempDir Path tmp) throws
Exception {
+ for (String prefix : new String[]{PipesServer.TEMP_DIR_PREFIX,
+ PipesServer.SHARED_TEMP_DIR_PREFIX}) {
+ Path own = Files.createDirectories(tmp.resolve(prefix + "1-abc"));
+ Files.writeString(own.resolve("spooled.tmp"), "document bytes");
+ Files.createDirectories(own.resolve("nested"));
+ assertTrue(PipesServer.deleteOwnTempDir(own));
+ assertFalse(Files.exists(own));
+ }
+
+ Path notOwn = Files.createDirectories(tmp.resolve("something-else"));
+ Files.writeString(notOwn.resolve("keep.tmp"), "not ours");
+ assertFalse(PipesServer.deleteOwnTempDir(notOwn));
+ assertTrue(Files.exists(notOwn.resolve("keep.tmp")));
+ assertFalse(PipesServer.deleteOwnTempDir(tmp));
+ assertTrue(Files.exists(notOwn));
+ }
+}
diff --git
a/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaLoader.java
b/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaLoader.java
index e117b71553..e04120c6cd 100644
---
a/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaLoader.java
+++
b/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaLoader.java
@@ -19,7 +19,9 @@ package org.apache.tika.config.loader;
import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
+import java.nio.file.Files;
import java.nio.file.Path;
+import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
@@ -30,6 +32,8 @@ import com.fasterxml.jackson.core.StreamReadConstraints;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import org.apache.tika.detect.CompositeDetector;
import org.apache.tika.detect.CompositeEncodingDetector;
@@ -60,6 +64,7 @@ import org.apache.tika.serialization.JsonMetadataList;
import org.apache.tika.serialization.ParseContextUtils;
import org.apache.tika.serialization.config.GlobalSettings;
import org.apache.tika.serialization.serdes.ParseContextDeserializer;
+import org.apache.tika.utils.StringUtils;
/**
* Main entry point for loading Tika components from JSON configuration.
@@ -93,6 +98,10 @@ import
org.apache.tika.serialization.serdes.ParseContextDeserializer;
*/
public class TikaLoader {
+ private static final Logger LOG =
LoggerFactory.getLogger(TikaLoader.class);
+
+ private static volatile boolean tempDirectoryLogged = false;
+
// Static registration of component configurations
static {
registerComponentConfigs();
@@ -170,9 +179,30 @@ public class TikaLoader {
* @throws TikaConfigException if loading global settings fails
*/
private void init() throws TikaConfigException, IOException {
+ checkTempDirectory();
loadGlobalSettings();
}
+ /** Fail here, naming the dir, rather than on the first spool with a bare
NoSuchFileException. */
+ private static void checkTempDirectory() throws TikaConfigException {
+ String prop = System.getProperty("java.io.tmpdir");
+ if (StringUtils.isBlank(prop)) {
+ throw new TikaConfigException("java.io.tmpdir is not set");
+ }
+ Path dir = Paths.get(prop);
+ if (!Files.isDirectory(dir)) {
+ throw new TikaConfigException(
+ "java.io.tmpdir does not exist or is not a directory: " +
dir);
+ }
+ if (!Files.isWritable(dir)) {
+ throw new TikaConfigException("java.io.tmpdir is not writable: " +
dir);
+ }
+ if (!tempDirectoryLogged) {
+ LOG.info("temporary files go to java.io.tmpdir={}",
dir.toAbsolutePath());
+ tempDirectoryLogged = true;
+ }
+ }
+
/**
* Loads a Tika configuration from a file.
* Global settings are automatically loaded and applied during
initialization.
diff --git
a/tika-serialization/src/test/java/org/apache/tika/config/loader/TikaLoaderTest.java
b/tika-serialization/src/test/java/org/apache/tika/config/loader/TikaLoaderTest.java
index d294ac203a..ec7e3b1782 100644
---
a/tika-serialization/src/test/java/org/apache/tika/config/loader/TikaLoaderTest.java
+++
b/tika-serialization/src/test/java/org/apache/tika/config/loader/TikaLoaderTest.java
@@ -28,6 +28,7 @@ import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
import org.xml.sax.helpers.DefaultHandler;
import org.apache.tika.config.EmbeddedLimits;
@@ -56,6 +57,27 @@ public class TikaLoaderTest {
assertNotNull(parser, "Parser should not be null");
}
+ @Test
+ public void testUnusableTempDirectoryFailsAtLoad(@TempDir Path tmp) throws
Exception {
+ Path configPath =
Path.of(getClass().getResource("/configs/test-loader-config.json").toURI());
+ Path missing = tmp.resolve("missing");
+ Path file = Files.writeString(tmp.resolve("file"), "");
+ String original = System.getProperty("java.io.tmpdir");
+ try {
+ for (Path bad : new Path[]{missing, file}) {
+ System.setProperty("java.io.tmpdir", bad.toString());
+ TikaConfigException e = assertThrows(TikaConfigException.class,
+ () -> TikaLoader.load(configPath), bad.toString());
+ assertTrue(e.getMessage().contains("java.io.tmpdir"),
e.getMessage());
+ assertTrue(e.getMessage().contains(bad.toString()),
e.getMessage());
+ }
+ System.setProperty("java.io.tmpdir", tmp.toString());
+ assertNotNull(TikaLoader.load(configPath));
+ } finally {
+ System.setProperty("java.io.tmpdir", original);
+ }
+ }
+
@Test
public void testConfigurableParserConfiguration() throws Exception {
URL configUrl =
getClass().getResource("/configs/test-loader-config.json");
diff --git
a/tika-translate/src/main/java/org/apache/tika/language/translate/impl/MarianTranslator.java
b/tika-translate/src/main/java/org/apache/tika/language/translate/impl/MarianTranslator.java
index 9170e154f7..11b882c9ac 100644
---
a/tika-translate/src/main/java/org/apache/tika/language/translate/impl/MarianTranslator.java
+++
b/tika-translate/src/main/java/org/apache/tika/language/translate/impl/MarianTranslator.java
@@ -132,15 +132,14 @@ public class MarianTranslator extends AbstractTranslator {
StringBuilder translation = new StringBuilder();
File tmpFile = Files.createTempFile(INPUT_TMP_NAME, ".tmp").toFile();
- tmpFile.deleteOnExit();
- try (OutputStreamWriter out = new OutputStreamWriter(new
FileOutputStream(tmpFile),
- Charset.defaultCharset())) {
- out.append(text).append('\n').close();
- }
- File tmpTranslatedFile = Files.createTempFile(OUTPUT_TMP_NAME,
".tmp").toFile();
- tmpTranslatedFile.deleteOnExit();
-
+ File tmpTranslatedFile = null;
try {
+ try (OutputStreamWriter out = new OutputStreamWriter(new
FileOutputStream(tmpFile),
+ Charset.defaultCharset())) {
+ out.append(text).append('\n').close();
+ }
+ tmpTranslatedFile = Files.createTempFile(OUTPUT_TMP_NAME,
".tmp").toFile();
+
String preProcessScript =
config.getProperty("translator.marian.preprocess");
executeScript(preProcessScript, tmpFile);
@@ -161,15 +160,25 @@ public class MarianTranslator extends AbstractTranslator {
} catch (InterruptedException e) {
throw new TikaException("Failed perform translation", e);
- }
-
- if (!tmpFile.delete() || !tmpTranslatedFile.delete()) {
- throw new IOException("Failed to delete temporary files.");
+ } finally {
+ deleteQuietly(tmpFile);
+ deleteQuietly(tmpTranslatedFile);
}
return translation.toString();
}
+ private static void deleteQuietly(File file) {
+ if (file == null) {
+ return;
+ }
+ try {
+ Files.deleteIfExists(file.toPath());
+ } catch (IOException e) {
+ LOG.warn("Failed to delete temporary file {}", file, e);
+ }
+ }
+
/**
* Process the translation request using a local instance of Marian - i.e.
either </i>marian-decoder</i>
* or <i>marian</i> command line applications.
