Joseph Gresock created NIFI-2934:
------------------------------------
Summary: Archiver still not respecting
nifi.content.repository.archive.max.usage.percentage
Key: NIFI-2934
URL: https://issues.apache.org/jira/browse/NIFI-2934
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 0.7.0, 0.7.1
Reporter: Joseph Gresock
Attachments: content_repository usage.png
This seems related to NIFI-1726: we've noticed that the content repository
takes up increasingly more space over time, even beyond the configured max
usage percentage (see images). After restarting the NiFi cluster we get an
immediate drop in disk usage with lots of log statements indicating that
expired content is being removed.
Not sure if this is related, but we also often get "Too many open files" during
this expiration process after NiFi restart, despite lsof indicating a count far
lower than our configured nofile and fs-max.
In the environment indicated by the pictures,
nifi.content.repository.archive.max.usage.percentage = 50%. Note that the flow
itself only has ~240GB queued across the entire cluster, but each
content_repository directory has over 360GB on each worker. Also note the disk
usage graph increasing above 50% on each worker, until we finally restart and
then the usage drops below 50%.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
