[
https://issues.apache.org/jira/browse/NIFI-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18106564#comment-18106564
]
ASF subversion and git services commented on NIFI-16238:
--------------------------------------------------------
Commit db732669a521b3df05b4ee1f2ef1cb1d37c78846 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=db732669a52 ]
NIFI-16238: Checkpoint write-ahead logs when the journal reaches a co… (#11575)
Journals could grow without bound between interval-based checkpoints. A maximum
journal size now triggers a checkpoint so storage is reclaimed even when
updates keep arriving.
> Allow specifying a max journal size for flowfile repo and local state
> ---------------------------------------------------------------------
>
> Key: NIFI-16238
> URL: https://issues.apache.org/jira/browse/NIFI-16238
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Both the local state provider and the FlowFile Repository make use of the
> {{SequentialAccessWriteAheadLog}} and checkpoint periodically based on a time
> threshold. But under heavy load we can write significant amounts of data,
> resulting on {{IOException: No space left on device}} issues. We should allow
> configuring a max journal size and if configured checkpoint when the journal
> exceeds that threshold. This allows us to avoid overfilling the disk even
> under heavy load.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
