Thanks a lot! How do people handle continuous feed into HDFS? Does everything go under one directory? Can it be split into multiple files?
On Sat, Mar 31, 2012 at 1:07 AM, Will McQueen <[email protected]> wrote: > Hi Mohit, > > The avro-client sends a file that you specify (run "flume help" to see see > the options) in avro format/protocol to the host and port you specify. For > example, you could setup a Flume NG avro source (specifying an avro source > in your config file with a logger sink), and then run your avro-client > against the flume node, looking at the logger output to confirm that the > events are coming through. > > Cheers, > Will > > > On Fri, Mar 30, 2012 at 4:27 PM, Mohit Anchlia <[email protected]>wrote: > >> I got the package compiled. Going through the getting started guide, >> what's the difference between flume node and flume avro-client? I see one >> line description but don't understand when to run one over other? >> >> >> On Thu, Mar 29, 2012 at 1:42 PM, Will McQueen <[email protected]> wrote: >> >>> Hi Mohit, >>> >>> You might want to start with the Flume NG Getting Started guide at: >>> https://cwiki.apache.org/FLUME/getting-started.html >>> >>> We also have an architecture doc at: >>> https://blogs.apache.org/flume/entry/flume_ng_architecture >>> >>> Cheers, >>> Will >>> >>> Cheers, >>> Will >>> >>> >>> On Thu, Mar 29, 2012 at 1:36 PM, Mohit Anchlia >>> <[email protected]>wrote: >>> >>>> I am new to Flume. I am looking at Flume NG and HDFS as sink for it. I >>>> am just trying to do basic testing to understand how it works. Some basic >>>> questions I have is that do I need to also install flume on the client >>>> machine, for eg web servers? >>>> >>>> Also, if for some reason server slows down or network connection breaks >>>> then does Flume client picks up the data from where it stopped writing? >>>> Wondering how that works >>>> >>> >>> >> >
