----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59743/#review176765 -----------------------------------------------------------
Thank you for this bugfix. I understand that this change can ease the blocking nature of one "over used" file in this taildir-source. This cannot solve the overload, that's why we should somehow give a feedback of this lack of resources. If the reshold is high enough, the log will not triggered too often I think. flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSource.java Line 273 (original), 275 (patched) <https://reviews.apache.org/r/59743/#comment250233> I would prefer to make it configurable. 1000 can be a default. flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSource.java Line 276 (original), 278 (patched) <https://reviews.apache.org/r/59743/#comment250235> If the readCount reaches numTreshold we could log a warning. Or it can trigger a metrics counter. Do you have an idea how to cover this case with a test? Thank you for this nice contribution. - Miklos Csanady On June 2, 2017, 10:09 a.m., hun shenshi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59743/ > ----------------------------------------------------------- > > (Updated June 2, 2017, 10:09 a.m.) > > > Review request for Flume. > > > Bugs: FLUME-3101 > https://issues.apache.org/jira/browse/FLUME-3101 > > > Repository: flume-git > > > Description > ------- > > Add a threshold in TaildirSource. Use the threshold to control the number of > reads when a file is written by high frequency. > > > Diffs > ----- > > > flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSource.java > a107a01 > > > Diff: https://reviews.apache.org/r/59743/diff/1/ > > > Testing > ------- > > mvn clean install -DskipTests -> built > > > Thanks, > > hun shenshi > >
