[
https://issues.apache.org/jira/browse/FLUME-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15836743#comment-15836743
]
Stephen O'Donnell commented on FLUME-2958:
------------------------------------------
Is the idea here that for each file group, you can specify an ignore pattern eg:
{code}
a1.sources.r1.filegroups = f1 f2
a1.sources.r1.filegroups.f1 = /var/log/test1/.*
a1.source.r1.ignorePattern.f1 = .*backup
a1.sources.r1.filegroups.f2 = /var/log/test2/.*log.*
a1.sources.r1.ignorePattern.f2 = /var/log/test2/.*notAlog.*
{code}
If no ignore pattern is present, then the behaviour is as today, but if there
is an ignore pattern, then matched files will be further filtered down by the
ignorePattern, eg:
{code}
final PathMatcher matcher = FS.getPathMatcher("regex:" + regex);
final PathMatcher ignoreMatcher = FS.getPathMatcher("regex:" + ignoreRegex);
this.fileFilter = new DirectoryStream.Filter<Path>() {
@Override
public boolean accept(Path entry) throws IOException {
return matcher.matches(entry.getFileName()) &&
!Files.isDirectory(entry) && !ignoreMatcher.maches(entry.getFileName());
}
};
{code}
> Add ignorePattern for TaildirSource
> -----------------------------------
>
> Key: FLUME-2958
> URL: https://issues.apache.org/jira/browse/FLUME-2958
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.7.0
> Reporter: Hu Liu,
> Assignee: Hu Liu,
> Fix For: v1.8.0
>
>
> we have tried the TaildirSource and found that it's lack of ignorePattern
> specifying which files to ignore. I'm glad to work on it if anyone could
> assign it to me
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
