[
https://issues.apache.org/jira/browse/MINIFICPP-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16800847#comment-16800847
]
Mr TheSegfault edited comment on MINIFICPP-788 at 3/25/19 4:24 PM:
-------------------------------------------------------------------
Directory can use flow file attributes to provide a directory to list, so there
may be one to many directories to evaluate. If the listing is empty the listing
will be populated then the threads will work on that list in batches – Since
this is a bug, what bug do you purport it causes?
was (Author: phrocker):
Directory can use flow file attributes to evaluate the directory, so there may
be one to many directories to evaluate. If the listing is empty the listing
will be populated then the threads will work on that list in batches – Since
this is a bug, what bug do you purport it causes?
> GetFile multithreading: directory listing should be single-threaded
> -------------------------------------------------------------------
>
> Key: MINIFICPP-788
> URL: https://issues.apache.org/jira/browse/MINIFICPP-788
> Project: NiFi MiNiFi C++
> Issue Type: Bug
> Reporter: Arpad Boda
> Assignee: Arpad Boda
> Priority: Minor
>
> While working on MINIFICPP-786 I found that multithreading might be
> ineffective in some edgecase.
> While multiple consumers are fine (they create flow files from the listed
> files), producer should be single-threaded. Listing the same folder in
> parallel threads seems to make few sense.
> This can only occur in case listing takes longer than scheduling/configured
> time and it adds no files to the queue. This shouldn't occur frequently, but
> I can imagine edge-cases, like using getfile on a folder with a lot of files,
> while age/size/regex properties filter most of those.
> In my opnion simply skip listing in case listing is already being executed
> would be a slight improvement in such cases.
> Set prio to minor as it's really an edgecase.
> [~phrocker] what's your opinion?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
