+1. Eager to see the code!
Thanks, Hari On Thu, Oct 9, 2014 at 8:13 PM, Roshan Naik <[email protected]> wrote: > Sounds interesting. Actually a source like this diminishes the need for > disk based buffering channels. And sounds like it does handle log file > rotation. Which is awesome. > A few questions: > - How does it know when to stop tailing the current file and switch to or > start tailing another file > - When there is a backlog of many files being built up... how does it > order the files for consumption > - Sounds like there is some C/C++ native code + JNI to work with inodes ? > what api are you using. > - does it auto delete the consumed files ? > FYI: > - For Windows Support: > http://www.codeproject.com/Articles/4692/How-to-get-a-notification-if-change-occurs-in-a-sp > -roshan > On Thu, Oct 9, 2014 at 6:58 PM, Otis Gospodnetic <[email protected] >> wrote: >> Same here! >> The only-on-UNIX part is not great, and I'm wondering if the code could be >> structured in a way that allow one (or Flume devs) to add something that >> also runs under WInblows, even if it cannot be as clever as the UNIX side. >> But even without this, I think what you describe is better than other >> tailing implementations because it handles rotations and reads from the >> last known checkpoint. >> >> Btw., because the position in the file is checkpointed periodically, does >> that mean that it is possible that, after a restart, some number of lines >> that have already been tailed, will be read again? >> >> Could you also compare your implementation with these two: >> 1) https://issues.apache.org/jira/browse/FLUME-2344 (and the comment from >> May 2 2014, which points to 2) below) >> 2) >> >> http://search-hadoop.com/m/DEeB4urEUJ&subj=Re+Flume+Jambalaya+A+Flume+Plugin+with+Multiple+Components >> >> Otis >> -- >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >> Solr & Elasticsearch Support * http://sematext.com/ >> >> >> On Thu, Oct 9, 2014 at 6:24 AM, Santiago Mola <[email protected]> wrote: >> >> > 2014-10-09 8:22 GMT+02:00 飯島賢志 <[email protected]>: >> > >> > > Can we contribute it to the Apache Flume community? >> > > If acceptable I will attach the patch of it in JIRA. >> > > >> > >> > That would be definitely useful for us. >> > >> > Best, >> > -- >> > >> > Santiago M. Mola >> > >> > >> > <http://www.stratio.com/> >> > Avenida de Europa, 26. Ática 5. 3ª Planta >> > 28224 Pozuelo de Alarcón, Madrid >> > Tel: +34 91 352 59 42 // *@stratiobd <https://twitter.com/StratioBD>* >> > >> > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
