[
https://issues.apache.org/jira/browse/NIFI-905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720359#comment-14720359
]
Mark Payne commented on NIFI-905:
---------------------------------
So I finally found the issue, i believe. Easy fix once it was found. Looks like
there's a counter that keeps track of how many things are archived. That
counter was getting incremented based on the number of Content Claims but it
should have been incremented on number of ResourceClaims. As a result, it
thought there were a lot of files archived when there were none. So it waited
indefinitely for the archive to be cleaned up.
While I was testing the patch, I found a few erroneous log messages that I
addressed and also found a condition where if the storage requirements for
archive hadn't been met, we still aged off the oldest file from the archive
every second, even if it wasn't old enough to warrant that. So I addressed that
as well.
> Clean up not occurring when content repository reaches max usage percentage
> ---------------------------------------------------------------------------
>
> Key: NIFI-905
> URL: https://issues.apache.org/jira/browse/NIFI-905
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 0.3.0
> Reporter: Bryan Bende
> Assignee: Mark Payne
> Fix For: 0.3.0
>
> Attachments:
> 0001-NIFI-905-Ensure-that-when-archive-threshold-is-hit-a.patch, nifi.dump
>
>
> Created a 500MB partition and set the content repository to use that
> partition, then created a simple Flow with GenerateFlowFile ->
> UpdateAttribute, using 10kb FlowFiles.
> When the content repository reached approx 224MB it started logging:
> "Unable to write to container default to archive file size constraints;
> waiting for archive cleanup"
> It appears that the clean up was never occurring and a thread dump shows a
> blocked thread:
> {code}
> "FileSystemRepository Workers Thread-2" daemon prio=10 tid=0x00007f78c2660000
> nid=0x2ae7 waiting for monitor entry [0x00007f78a907d000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.archive(FileSystemRepository.java:1095)
> - waiting to lock <0x00000000e193ae00> (a
> java.util.concurrent.LinkedBlockingQueue)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.access$1200(FileSystemRepository.java:83)
> at
> org.apache.nifi.controller.repository.FileSystemRepository$ArchiveOrDestroyDestructableClaims.run(FileSystemRepository.java:1357)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
