[
https://issues.apache.org/jira/browse/NIFI-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976088#comment-15976088
]
ASF subversion and git services commented on NIFI-3273:
-------------------------------------------------------
Commit 0207f21ce4bb68536843d65ecc9110ec834028cd in nifi's branch
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=0207f21 ]
NIFI-3273: Added nifi-toolkit-flowfile-repo that contains a simple Java class
that is capable of recovering a FlowFile Repository manually in the case of an
operating system crash that results in trailing 0's being dumped into the edit
logs. Also refactored flowfile repo into some independent modules so that it
additional capabilities can be added in the future to examine the flowfile repo
> MinimalLockingWriteAheadLog doesn't properly handle corrupted journals
> -----------------------------------------------------------------------
>
> Key: NIFI-3273
> URL: https://issues.apache.org/jira/browse/NIFI-3273
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Joseph Percivall
> Assignee: Joseph Witt
> Priority: Critical
> Fix For: 1.2.0
>
>
> When NiFi is running if the system dies abruptly (sudden power loss) without
> flushing writes then anything that was being written to disk can become
> corrupted. A ticket for the provenance repository is already created here[1].
> The content repo handles this automatically since the content claim won't be
> valid if it hasn't been written out yet. The database repo is just a cache
> and is rebuilt anyway. The logs are handled by logback. The flow.xml.gz can
> be rolled back to one the last archive (manually).
> This ticket is for the MinimalLockingWriteAheadLog which backs the FlowFile
> repo and local state. Originally brought up here[2] for MiNiFi, it will also
> affect NiFi.
> One possible solution is to restore transactions up until the corrupted id
> and then ignore the rest. This could cause state to become out of sync with
> the processed flowfiles (if FF repo is restored but local state cannot be
> fully restored) but given the rarity of the event I think it is an
> appropriate risk to accept.
> The workaround for the FF repo is to set
> "nifi.flowfile.repository.always.sync" but currently there is no way to set
> "alway sync" for the local state provider.
> [1] https://issues.apache.org/jira/browse/NIFI-2890
> [2]
> https://community.hortonworks.com/questions/75280/why-does-my-minifi-flow-fail-to-run-when-turning-o.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
