[
https://issues.apache.org/jira/browse/NIFI-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393650#comment-16393650
]
ASF GitHub Bot commented on NIFI-3039:
--------------------------------------
Github user mosermw commented on the issue:
https://github.com/apache/nifi/pull/1240
I tested this a lot, and based on the conversation in the JIRA ticket, I
adjusted the purge low water mark to 88%. This allows the prov_repo to purge
old events quicker after a large spike in events, whereas before it could take
a long time to bring the prov_repo back down to 90% usage. It also doesn't
delete too many events. I also adjusted the rollover high water mark to 99%,
which allows space for journals and indexes. This keeps prov_repo usage much
closer to the nifi.provenance.repository.max.storage.size.
+1 will merge.
> Provenance Repository - Fix PurgeOldEvent and Rollover Size Limits
> ------------------------------------------------------------------
>
> Key: NIFI-3039
> URL: https://issues.apache.org/jira/browse/NIFI-3039
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.0.0, 1.1.0, 0.8.0, 0.7.1
> Reporter: Joe Skora
> Assignee: Joe Skora
> Priority: Major
>
> Current {purgeOldEvents} logic triggers cleanup when 90% of space is used,
> but it only removes one file if usage is under 100%, causing thrashing around
> 100% usage. In testing, cleanup up to 70% after hitting 90% makes the system
> run more smoothly.
> Also, {rollover} will not trigger cleanup unless 110% of the allowed space is
> in use, changing this to 100% also make a difference in testing.
> Before these changes, a test system that generates huge amounts of provenance
> would become unstable and stop processing provenance until restarted. With
> these changes, the system consistently recovers even under heavy load.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
