[
https://issues.apache.org/jira/browse/NIFI-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15156171#comment-15156171
]
Joseph Witt commented on NIFI-1527:
-----------------------------------
Re-opened to evaluate an observation. Might be nothing...but want to be sure.
[~markap14] Here is a testing scenario I ran.
Start a flow with GenerateFlowFile and UpdateAttribute
UpdAtt was disabled
caused say 1M flow files to backup
then stopped gneerating data just to keep things simple
so now 1M backed up
then stopped nifi and restarted
all good upon restart its all there again
Then
i made another copy of that flow on the flow
and let that one generate data and terminate data
what i wanted to observe is that the flow files generated were actually having
their backing content deleted
because the archive size was set at 10% threshold and i am on a 22% full disk
so was intentionally saying we don't have archive space left
anyway - the data on the second flow was being properly removed from the
content repo
However then I stopped that flow
and I ensureed all that was left in the content repo was the original stuff I
made
verified by timetstamp on the files in the content repo anyway
THEN
i let UpdateAttribute on the original flow run so it would clear out all the
flow files
in this case, the content repo was NOT getting cleared out
What I see in the content repo are files like this
./71/1456079610973-71
...nothing in ./71/archive/...
This does not seem right.
Thanks
Joe
> Resource Claim counts not incremented on restart for FlowFiles that are
> swapped out
> -----------------------------------------------------------------------------------
>
> Key: NIFI-1527
> URL: https://issues.apache.org/jira/browse/NIFI-1527
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Blocker
> Fix For: 0.6.0, 0.5.1
>
> Attachments:
> 0001-NIFI-1527-Ensure-that-we-increment-Claimant-Counts-f.patch,
> 0001-NIFI-1527-Ensure-that-we-increment-Claimant-Counts-f.patch
>
>
> When NiFi starts, it tallies the count of how many FlowFiles reference each
> Resource Claim and then removes or archives any file in the Content
> Repository for which there is no claim.
> However, the claim counts are not incremented for data that is swapped out.
> As a result, the content repository could delete or archive data on restart
> that it should not. This could potentially result in data loss, if NiFi is
> restarted while both of the following conditions are met:
> * All FlowFiles that reference a Resource Claim are swapped out
> * Content Repository's archive is disabled, or the archival threshold is
> already exceeded from data outside of the Content Repository
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
