[
https://issues.apache.org/jira/browse/NIFI-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061573#comment-16061573
]
Joseph Percivall commented on NIFI-3897:
----------------------------------------
I believe I hit this state. I debugged a bit with [~markap14] and I'll let him
comment on what he's going to do but we believe it to be related to un-caught
throwables (ie. out of memory errors) but not 100% sure it only arises under
those conditions. Below I'll outline a summary of what the problem was, what I
tried and what fixed it:
NiFi 1.3.0 standalone instance
Apparent problem:
* An RPG is routed to a funnel and FlowFiles are not being processed by the
funnel from that queue. The RPG continues to grab more FlowFiles
Other symptoms:
* The funnel will still process flowfiles coming from another connection
* There are 2 swap files on disk for that queue (logs for the creation of
them have aged off)
* There is a distinct cliff of flowfiles getting processed and then stopping
* There was a swap for that queue a half second prior to the cliff but is
swapped back in a couple seconds later
* An out of memory error log is present in the bootstrap logs for a couple
days prior to the cliff
Actions Tried:
* List queue says there are 0 flowfiles in the queue
* Moving the connection to another component. Then moving it back. Also
tried starting the other component. Neither worked.
* Setting the expiration had no effect
* Back-pressure works as expected.
* Sent in a large group of flowfiles (larger than the swap size). Once it
hit the next swap size it flipped and started processing those. It only
processed down to 2 swap files worth and then got into the same state.
Remedy:
* Restarted NiFi and it successfully processed them
> Queue with flow files thinks it is empty
> ----------------------------------------
>
> Key: NIFI-3897
> URL: https://issues.apache.org/jira/browse/NIFI-3897
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.2.0
> Reporter: Joseph Gresock
> Attachments: queue-has-no-flowfiles.png
>
>
> A particular queue is listed as having 80,000 flow files, but NiFi thinks
> there are none to list. I was originally inclined to believe that the queue
> is indeed empty, and that there is somehow a pointer to 80,000 flow files
> leftover on the queue.
> Unfortunately, the logs have rolled over so I can't see what error or warning
> messages may have happened at the time, and I can't reproduce it reliably.
> Update: I happened to reboot my cluster, and suddenly the 80,000 flow files
> went through the flow. This makes the problem considerably worse, since it
> appears the flow files were "invisible" until the reboot.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
