[
https://issues.apache.org/jira/browse/NIFI-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420876#comment-17420876
]
Andrew Heys commented on NIFI-9056:
-----------------------------------
When restarting one of the nodes on our production cluster, I see about 40k
instances of this line in the logs:
"...FileSystemRepository Found unknown file <path to content repo file> (x
bytes) in File System Repository: archiving file"
We have been running some tests in a 3 node test cluster and so far have had no
luck re-creating the behavior to this degree. I have one node with 1.5x more
file space and the content storage utilization is keeping an even amount
percentage-wise across the nodes. In this case all the flowfiles are evenly
sized and distributed on the cluster. After restarting one of the test nodes, I
see the unknown file log only about 50 times.
> Content Repository Filling Up
> -----------------------------
>
> Key: NIFI-9056
> URL: https://issues.apache.org/jira/browse/NIFI-9056
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.13.2
> Reporter: Andrew Heys
> Priority: Major
>
> We have a clustered nifi setup that has recently been upgraded to 1.13.2 from
> 1.11.4. After upgrading, one of the issues we have run into is that the
> Content Repository will fill up to the
> nifi.content.repository.archive.backpressure.percentage mark and lock the
> processing & canvas. The only solution is to restart nifi at this point. We
> have the following properties set:
> nifi.content.repository.archive.backpressure.percentage=95%
> nifi.content.repository.archive.max.usage.percentage=25%
> nifi.content.repository.archive.max.retention.period=2 hours
> The max usage property seems to be completed ignored. Monitoring the nifi
> cluster disk % for content repository shows that it slowly fills up over time
> and never decreasing. If we pause the input to entire nifi flow and let all
> the processing clear out with 0 flowfiles remaining on the canvas for 15+
> minutes, the content repository disk usage does not decrease. Currently, our
> only solution is to restart nifi on a daily cron schedule. After restarting
> the nifi, it will clear out the 80+ GB of the content repository and usage
> falls down to 0%.
>
> There seems to be an issue removing the older content claims in 1.13.2.
> Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
