Hi Joe, There were around 50 or so “Queued” < 100MB.
Restarting the cluster nodes cleared up the used space. So apparently at some point the archiver stopped doing its thing. I looked for odd messages from FileSystemRepository but didn’t see anything. This problem looks similar to the one discussed in the the thread which can be accessed by the URL below. The difference seems to be is that in my case just restarting with the existing settings got things back on track. http://markmail.org/message/5kj3akgexosg4jav#query:+page:1+mid:ohxtkuayhqqhcmjc+state:results Thanks, Chris On 4/3/16, 5:23 PM, "Joe Witt" <[email protected]> wrote: >Chris, > >How many flow files do you have actively in the flow? The archive >goal says 50% of total space and the space appears to be 148GB so >archive should start removing old files around 70ish GB. But for data >actually sitting in the flow NiFi will retain that. > >Thanks >Joe > >On Sun, Apr 3, 2016 at 4:09 PM, McDermott, Chris Kevin (MSDU - >STaTS/StorefrontRemote) <[email protected]> wrote: >> I’m having a problem where the disk on which I am storing my NiFi data keeps >> filling up. The content repository is taking up the overwhelming majority >> of that space. It seems that the archiver is not working the way I would >> expect. >> >> $ du -sh /opt/mount2/nifi_data/* >> 712K /opt/mount2/nifi_data/conf >> 147G /opt/mount2/nifi_data/content_repository >> 656K /opt/mount2/nifi_data/database_repository >> 318M /opt/mount2/nifi_data/flowfile_repository >> 937M /opt/mount2/nifi_data/provenance_repository >> >> $ df -lh | grep mount2 >> /dev/mapper/disk2-mount2 >> 148G 148G 0 100% /opt/mount2 >> >> # Content Repository >> nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository >> nifi.content.claim.max.appendable.size=10 MB >> nifi.content.claim.max.flow.files=100 >> nifi.content.repository.directory.default=/opt/mount2/nifi_data/content_repository >> nifi.content.repository.archive.max.retention.period=12 hours >> nifi.content.repository.archive.max.usage.percentage=50% >> nifi.content.repository.archive.enabled=true >> nifi.content.repository.always.sync=false >> nifi.content.viewer.url=/nifi-content-viewer/ >> >> Can anyone point out what I might be missing? >> >> Thanks, >> Chris
