GitHub user YolandaMDavis opened a pull request:
https://github.com/apache/nifi/pull/840
NIFI-2524 - Fixes to improve handling of missing journal files
This fix focuses on preventing some of the side effects seen during
provenance journal rollover outlined in NIFI-2524 after an OutOfMemoryError
(which appeared to cause the removal of journal files from disk before they
were merged). The goals of this fix was to a) ensure that any information
captured is rolled over and b) to ensure that if no files are available (or if
an other error occurs during processing) that the system will a reasonable
effort to retry the issue until it's resolved (yet not retry infinitely). This
pr include:
- Removal of partial file check (based on missing first file)
- Addition of conditional check to merge if at least one journal files
available on disk. If all files are missing from disk that is considered an
error.
- Addition of retry logic to prevent endless thread execution when
encountering errors (such as in missing all journal files). Retry is attempted
5 times before the thread is cancelled.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/YolandaMDavis/nifi NIFI-2524
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/840.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #840
----
commit 3560bd7bb6a765a0d8a37257e00602a8a22348ed
Author: Yolanda M. Davis <[email protected]>
Date: 2016-08-11T09:42:19Z
NIFI-2524 - Fixes to improve handling of missing journal files during
rollover/merge execution.Includes:
Removed partial file check (based on missing first file)
Added condition to merge if at least one journal files available on disk.
If all files are missing from disk that is considered an error.
Added retry logic to prevent endless thread execution when encountering
errors (such as missing files).
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
