Also the link below is associated with Flume OG and not Flume NG http://archive.cloudera.com/cdh/3/flume/UserGuide/
The architecture and features have changed significantly since that version On 7 April 2013 17:54, Israel Ekpo <[email protected]> wrote: > Sandeep, > > So Flume currently has two tracks: > > Flume OG (not actively supported) > https://cwiki.apache.org/confluence/display/FLUME/Flume+OG+%28pre+1.0%29 > Flume NG (currently active) > https://cwiki.apache.org/confluence/display/FLUME/Flume+NG > > The latest stable version for Flume NG is 1.3.1 > > The NG stands for Next Generation and it is the current active development > track. > > The OG refers to the Original Generation of Flume. This includes releases > before the 1.0.0 release. > > New comers and existing users of the OG track are encouraged to migrate > over to the NG track. > > You can download Flume NG 1.3.1 here > > http://flume.apache.org/download.html > > Regarding "Getting Started", in the next couple of weeks, additional > information will be added to Wiki to make the on-boarding process easier > for new comers. > > In the time being, please bear with us. > > I would recommend you download and install the latest version of Java 1.6. > > Then download Flume and extract it to folder in your directory. > > Then you can use the following sources, channels and sinks to get started. > > This is the best way for you to learn and understand the various pieces. > > SOURCE: Spooling Directory Source > CHANNEL: File Channel (more reliable) or Memory Channel (faster) > SINK: File Roll Sink > > You can create a directory that you will be spooling and dump a couple of > log files in there. Make sure the files a new-line delimited. > > Each line will represent an event in the log files. > > Then configure the file channel and the file roll sink using guidelines > and examples available in the user guide. > > http://flume.apache.org/FlumeUserGuide.html > > That will give you a feel for how flume works. > > Once you have that set up then you can run the agent and see what happens. > > Once you start getting the hang of things you can try other sources and > sinks or maybe even create a few of your own custom sources, channels or > sinks. > > > > On 7 April 2013 17:10, Sandeep Baldawa <[email protected]> wrote: > >> >> Thanks for the detailed reply. >> >> Awesome questions and I should have added these details in my question, >> am learning flume more as a hobby, learning experience and a tech >> enthusiastic(heard pretty good things about flume). >> >> Thanks again for the instructions. Just one question about setting things >> up, are instructions at >> http://archive.cloudera.com/cdh/3/flume/UserGuide/ relevant with the >> latest build?, I liked the documentation in this link which has a quick >> start guide too. >> >> >> On Sun, Apr 7, 2013 at 1:28 PM, Israel Ekpo <[email protected]> wrote: >> >>> Sandeep, >>> >>> Excellent questions. >>> >>> You asked "what problem Flume is trying to solve?". >>> >>> It think the more appropriate question is what problem you are trying to >>> solve? >>> >>> This will go a long way in helping us understand which components of >>> Flume you may need and how you need to set it up. >>> >>> Are you using Flume as part of your job or personal hubby? Are you using >>> Flume for a course at school or part of an academic project? >>> >>> Going back to your original question, in the simplest terms, and for >>> most use cases, Flume is a system designed for collecting and transporting >>> large amounts of data and events from one or more sources and then >>> aggregating the collected data in a centralized data store or for onward >>> propagation to subsequent sources. >>> >>> You can use it for aggregating data from log files, network traffic, >>> click streams, twitter and any other source that can generate events. >>> >>> Spend more time to review the user guide and you will find a lot of >>> information and answers to prospective questions. >>> >>> http://flume.apache.org/FlumeUserGuide.html >>> >>> To install flume you will need to set up Java 1.6 and then make sure >>> that it is available in your PATH and then download the latest version of >>> Flume and decompress the tarball or zip file. >>> >>> You will need to set up the configuration file(s) for the agents based >>> on the sources, channels and sinks you choose to use. >>> >>> I would recommend that you go ahead and get started with setting it up >>> and let us know if you run into any issues. >>> >>> If you can share which use case and what problem you are trying to >>> solve, someone can point you in the right direction. >>> >>> >>> >>> On 7 April 2013 14:45, Sandeep Baldawa <[email protected]> wrote: >>> >>>> Hello, >>>> >>>> I am a new user for flume and was wondering what was the best way to >>>> learn about flume from a user's perspective. >>>> >>>> I am interested more in getting answers for the following questions >>>> - What problem is flume trying to solve. >>>> - How to install flume in the most simple way to understand the >>>> concepts. >>>> - Am trying to follow http://flume.apache.org/FlumeUserGuide.html# and >>>> am not sure if I can find a quick start guide here, can someone point me to >>>> the correct link if possible. >>>> >>>> Thanks, >>>> Sandeep >>>> >>> >>> >> >
