[
https://issues.apache.org/jira/browse/TIKA-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18108202#comment-18108202
]
ASF GitHub Bot commented on TIKA-4835:
--------------------------------------
Copilot commented on code in PR #3067:
URL: https://github.com/apache/tika/pull/3067#discussion_r3860503252
##########
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/detect/microsoft/POIFSContainerDetector.java:
##########
@@ -583,16 +597,54 @@ private Set<String> getTopLevelNames(TikaInputStream
stream) throws IOException
}
}
+ /**
+ * Opens the OLE2 container from memory instead of spooling it to a temp
file. Returns
+ * null when the object is too large for the in-memory limit or POI cannot
load it from a
+ * stream (the caller then spools, as before).
+ */
+ private Set<String> getTopLevelNamesInMemory(TikaInputStream stream,
ParseContext context)
+ throws IOException {
+ CacheMemoryBudget budget = context == null ? null :
context.get(CacheMemoryBudget.class);
+ long limit = budget == null ? DEFAULT_IN_MEMORY_POIFS :
+ Math.min(MAX_IN_MEMORY_POIFS, budget.getMaxBytes());
+ // attach the budget first: without it the drain below caches only the
per-object
+ // default (1MB) in memory and spills the rest before we can even
check the size
+ stream.enableRewind(budget);
+ // the channel is served from memory while the content fits the
cache/budget
+ try (SeekableByteChannel channel = stream.getSeekableByteChannel()) {
+ long size = channel.size();
+ if (size > limit) {
+ return null;
+ }
+ if (budget != null) {
+ if (budget.tryReserve(size) == 0) {
+ return null;
+ }
+ stream.addCloseableResource(() -> budget.release(size));
+ }
+ POIFSFileSystem fs = new
POIFSFileSystem(Channels.newInputStream(channel));
+ stream.setOpenContainer(fs);
+ return getTopLevelNames(fs.getRoot());
Review Comment:
When POI's stream loader throws (IOException/RuntimeException), this method
returns null and falls back to spooling, but the CacheMemoryBudget reservation
added at `stream.addCloseableResource(() -> budget.release(size))` is still
held until the TikaInputStream is closed. That can artificially reduce the
shared budget and force unrelated caches to spill even though the in-memory
POIFS path was abandoned.
Consider only registering the release callback after successfully
constructing the POIFSFileSystem, and explicitly releasing the reservation on
failure before returning null.
##########
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFParser.java:
##########
@@ -315,10 +321,25 @@ private void scanXRefOffsets(PDFParserConfig localConfig,
List<StartXRefOffset> xRefOffsets = new ArrayList<>();
//TODO
> Diagnose performance issues in 4.0.0
> ------------------------------------
>
> Key: TIKA-4835
> URL: https://issues.apache.org/jira/browse/TIKA-4835
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Minor
>
> In both tika-server and tika-app, I'm seeing slower performance. This is
> somewhat explainable by the switch to pipes – we're trading robustness for
> speed.
> However, with some params on some docs on some vms, we're doing much better,
> and obv, tika-pipes in shared-server mode is much faster than the default
> isolated.
> I already opened a ticket to improve excess caching to disk, which wasn't
> great on a vm with a slowish drive. I also drafted this:
> https://tika.apache.org/docs/4.0.x/pipes/performance.html
> We should figure out if this is a change in how the parsers are working,
> something at the tika-level generally or something within pipes.
> There's more work to do here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
