[
https://issues.apache.org/jira/browse/MINIFICPP-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910228#comment-17910228
]
Marton Szasz commented on MINIFICPP-2507:
-----------------------------------------
[~shivaniadapa] I don't think this would address your performance problem.
Every time TailFile runs, it has to recursively list all files, and match them
against the regex, which takes some time, especially with a large amount of
files. Matching in the tail-base-directory doesn't seem feasible, as Ferenc
mentioned, and matching in File to Tail would list the same large number of
files.
Does NiFi perform any better in your use case? If so, we can try to match it
behavior. Or if you know an alternative solution with a different approach that
works better, we can explore imitating that too.
> TailFile "tail-base-directory" property should support Wildcards
> ----------------------------------------------------------------
>
> Key: MINIFICPP-2507
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2507
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
> Reporter: Shivani
> Priority: Major
>
> Currently, the _File to Tail_ property supports wildcards but is limited to a
> specific directory. Requesting an enhancement that allows specifying a regex
> pattern in the parent directory of the _tail-base-directory_ property.
> While it is possible to use multiple TailFile processors as a workaround to
> achieve this, it is often not feasible due to the potentially exhaustive and
> indeterministic list of files that need to be monitored.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
