[
https://issues.apache.org/jira/browse/NIFI-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876581#comment-15876581
]
Joseph Witt commented on NIFI-3511:
-----------------------------------
[~markap14] any thoughts?
Ricky a couple thoughts:
1) The query will stop and return what it has after 1000 matching hits i
believe. That doesn't sound like your case.
2) When splitting a single object into hundreds of thousands it is probably
best to do so in a multi step split. I wonder if you're seeing any out of
memory logs during those times. You'd know if it was common as the split
itself would fail.
> Searching High Volume Provenance Seemingly Applies Arbitrary Range
> Restriction
> -------------------------------------------------------------------------------
>
> Key: NIFI-3511
> URL: https://issues.apache.org/jira/browse/NIFI-3511
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Ricky Saltzer
> Assignee: Ricky Saltzer
>
> This issue seems to happen in environments which have a large volume of
> FlowFiles, or in our case, a single FlowFile that is expanded into several
> hundred thousand FlowFiles (several hundred a day).
> There are times when we need to validate a certain FlowFile passed through
> our data flow. It appears that even though we search on a specific attribute
> that did in fact pass through the data flow within the time range specified,
> it some how misses being searched unless we make the time range smaller.
> Does query runner stops searching after hitting some sort of limit?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
