[
https://issues.apache.org/jira/browse/TIKA-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490717#comment-15490717
]
Nick Burch commented on TIKA-2058:
----------------------------------
Looking at the patch, I'm not sure how it will help? When the
{{FileBackedDataSource}} is closed, it empties out {{buffersToClean}}, so no
references should be held no matter what the type.
>From a quick review, we could probably save a bit of time by not bothering to
>record read-only memory buffers in that list, as we don't need any special
>unmapping, but it shouldn't make any huge difference. If you're seeing these
>still being used after you're done with a file, then that implies we're not
>closing something somewhere, and would hit the same kind of problems on a
>read/write opened file. I don't want to just fix the symptom if an underlying
>problem remains!
> Memory Leak in Tika version 1.13 when parsing millions of files
> ---------------------------------------------------------------
>
> Key: TIKA-2058
> URL: https://issues.apache.org/jira/browse/TIKA-2058
> Project: Tika
> Issue Type: Bug
> Affects Versions: 1.13
> Reporter: Tim Barrett
> Attachments: poi-3.15-beta1-p1.jar, poi-3.15-beta1-p1.pom,
> prevents-OOM-when-writable-is-false.patch, screenshot-1.png,
> screenshot-2.png, screenshot-3.png
>
>
> We have an application using Tika which parses roughly 7,000,000 files of
> different types, many of the files are MSG files with attachments. This works
> correctly with Tika 1.9, and has been in production for over a year, with
> parsing runs taking place every few weeks. The same application runs into
> insufficient memory problems (java heap) when using Tika 1.13.
> I have used lsof and file leak detector to track down open files, however
> neither shows any open files when the application is running. I did find an
> issue with open files https://issues.apache.org/jira/browse/TIKA-2015,
> however there was a workaround for this and this is not the issue.
> I am sorry to have to report this with a level of vagueness, but with lsof
> turning nothing up I am a bit stuck as to how to investigate further. We are
> more than willing to help by testing on the basis of any ideas provided.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
