[
https://issues.apache.org/jira/browse/NIFI-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15442158#comment-15442158
]
ASF GitHub Bot commented on NIFI-2681:
--------------------------------------
GitHub user markap14 opened a pull request:
https://github.com/apache/nifi/pull/958
NIFI-2681: Refactored IndexManager into an interface and renamed the …
…existing implementation to CachingIndexManager. Implemented a new
SimpleIndexManager that performs no caching of IndexSearchers.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/markap14/nifi NIFI-2681
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/958.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #958
----
commit 58c715cefaeaa9ef2b7d89cd751456da73f42953
Author: Mark Payne <[email protected]>
Date: 2016-08-27T00:03:16Z
NIFI-2681: Refactored IndexManager into an interface and renamed the
existing implementation to CachingIndexManager. Implemented a new
SimpleIndexManager that performs no caching of IndexSearchers.
----
> Avoid caching Provenance Index Searchers
> ----------------------------------------
>
> Key: NIFI-2681
> URL: https://issues.apache.org/jira/browse/NIFI-2681
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Critical
> Fix For: 1.1.0
>
>
> In NIFI-2600 and NIFI-2452, we addressed two bugs where the Provenance
> Repository closes a cached IndexSearcher too soon. The IndexManager keeps the
> searchers cached in an effort to offer better performance when performing a
> Provenance Query. This was done because it was recommended in the Lucene
> documentation. However, we occasionally still see nodes crashing with
> segfaults due to the Lucene Searching. We should update the Persistent
> Provenance Repository to stop caching Index Searchers in order to trade a
> slight performance improvement for significantly better reliability.
> Playing around with the idea in order to test it out shows very favorable
> results. On a system where I could cause a seg fault almost every time that I
> ran a large provenance query, I updated the code to no longer cache the
> readers and saw perfect stability with no noticeable performance degradation.
> I will cleanup the code and submit a PR for these changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
