Thank you for your replying and some questions. I will create a new ticket with this patch in JIRA later. I would like to discuss there.
Thanks, -Satoshi Iijima 2014-10-10 12:25 GMT+09:00 Hari Shreedharan <[email protected]> > wrote: > > +1. Eager to see the code! > > > Thanks, > Hari > > On Thu, Oct 9, 2014 at 8:13 PM, Roshan Naik <[email protected]> > wrote: > > > Sounds interesting. Actually a source like this diminishes the need for > > disk based buffering channels. And sounds like it does handle log file > > rotation. Which is awesome. > > A few questions: > > - How does it know when to stop tailing the current file and switch to > or > > start tailing another file > > - When there is a backlog of many files being built up... how does it > > order the files for consumption > > - Sounds like there is some C/C++ native code + JNI to work with inodes > ? > > what api are you using. > > - does it auto delete the consumed files ? > > FYI: > > - For Windows Support: > > > http://www.codeproject.com/Articles/4692/How-to-get-a-notification-if-change-occurs-in-a-sp > > -roshan > > On Thu, Oct 9, 2014 at 6:58 PM, Otis Gospodnetic < > [email protected] > >> wrote: > >> Same here! > >> The only-on-UNIX part is not great, and I'm wondering if the code could > be > >> structured in a way that allow one (or Flume devs) to add something that > >> also runs under WInblows, even if it cannot be as clever as the UNIX > side. > >> But even without this, I think what you describe is better than other > >> tailing implementations because it handles rotations and reads from the > >> last known checkpoint. > >> > >> Btw., because the position in the file is checkpointed periodically, > does > >> that mean that it is possible that, after a restart, some number of > lines > >> that have already been tailed, will be read again? > >> > >> Could you also compare your implementation with these two: > >> 1) https://issues.apache.org/jira/browse/FLUME-2344 (and the comment > from > >> May 2 2014, which points to 2) below) > >> 2) > >> > >> > http://search-hadoop.com/m/DEeB4urEUJ&subj=Re+Flume+Jambalaya+A+Flume+Plugin+with+Multiple+Components > >> > >> Otis > >> -- > >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management > >> Solr & Elasticsearch Support * http://sematext.com/ > >> > >> > >> On Thu, Oct 9, 2014 at 6:24 AM, Santiago Mola <[email protected]> > wrote: > >> > >> > 2014-10-09 8:22 GMT+02:00 飯島賢志 <[email protected]>: > >> > > >> > > Can we contribute it to the Apache Flume community? > >> > > If acceptable I will attach the patch of it in JIRA. > >> > > > >> > > >> > That would be definitely useful for us. > >> > > >> > Best, > >> > -- > >> > > >> > Santiago M. Mola > >> > > >> > > >> > <http://www.stratio.com/> > >> > Avenida de Europa, 26. Ática 5. 3ª Planta > >> > 28224 Pozuelo de Alarcón, Madrid > >> > Tel: +34 91 352 59 42 // *@stratiobd <https://twitter.com/StratioBD>* > >> > > >> > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > >
