what happens when you tail the collector.log file? can you share your initial_adaptors and collectors file in conf ?
On Thu, Nov 10, 2011 at 8:13 AM, Mohammad Tariq <[email protected]> wrote: > Hello AD, > I am sorry for leaving the discussion in between, I was busy with > my exams.Now I am back and need your help.I started with what you have > suggested and read about the concepts (agents and collectors etc). Now > I am trying to collect the data from a file on the local file system > and trying to dump it into the HDFS and I am running the agent and > collector on the same machine.I was able to start the agent but when I > tried to start the collector I got the following line at the terminal > and the terminal got stuck there itself - > tariq@ubuntu:~/chukwa-0.4.0$ 2011-11-10 18:42:09.521::INFO: Logging > to STDERR via org.mortbay.log.StdErrLog > 2011-11-10 18:42:09.544::INFO: jetty-6.1.11 > > When I issued jps, I could see only the agent. > > Regards, > Mohammad Tariq > > > > On Sat, Nov 5, 2011 at 5:38 AM, Bill Graham <[email protected]> wrote: > > One difference between Chukwa and Flume is that in Chukwa you configure > each > > component individually to know how to handle different datatypes. This > makes > > it fairly straight-forward to set up, but can be painful to manage for > very > > large installs (so I've heard). Flume centralized the configs of "flows" > > which helps when managing larger installs, at the cost of additional > > complexity. > > > > On Fri, Nov 4, 2011 at 7:02 AM, AD <[email protected]> wrote: > >> > >> Are you running the agent and collector on the same machine? Off the > cuff > >> its something like > >> 1 - set your CHUKWA_HOME env variable > >> 2 - edit conf/chukwa-env.sh and make sure JAVA_HOME and all the vars > look > >> good > >> 3 - edit conf/initial_adaptors with the adaptors you need. note here > that > >> the dataType > >> 4 - edit conf/collectors and add the hostname/ip of the collectors > >> 5 - edit conf/agents and add ip/hostname of the agents > >> 6 - edit conf/chukwa-demux.conf and make sure you have a demux processor > >> for your dataType (name from initial_adaptors) > >> On agent node run bin/chukwa agent , tail ${CHUKWA_LOGS}/agent.log > >> On collector node run bin/chukwa collector , tail > >> ${CHUKWA_LOGS]/collector.log > >> On collector node run bin/chukwa Demux , tail ${CHUKWA_LOGS}/Demux.log > >> > >> > >> On Fri, Nov 4, 2011 at 9:48 AM, Mohammad Tariq <[email protected]> > wrote: > >>> > >>> Hello AD, > >>> Yes, I have read the wiki and right now I am trying to set up a > >>> Chukwa cluster using the wiki itself. Are there any other links you > >>> are aware of that can help me learn and use Chukwa in a better > >>> fashion?? I am trying to use Chukwa for data in "The Wellsite > >>> Information Transfer Standard Markup Language (WITSML)" format. > >>> > >>> Regards, > >>> Mohammad Tariq > >>> > >>> > >>> On Fri, Nov 4, 2011 at 6:49 PM, AD <[email protected]> wrote: > >>> > > >>> > I think it helps to read he wiki and understand the basic concepts, > >>> > like agents and collectors. What kind of data do you want to parse? > I have > >>> > used both Flume and Chukwa myself and can say for sure Chukwa is > miles > >>> > easier to work with and configure. Flume may have some more advanced > >>> > features, but so far i found Chukwa both easy to extend from a > development > >>> > standpoint, and much easier to configure. > >>> > Have you read the wiki pages ? > >>> > > >>> > On Fri, Nov 4, 2011 at 8:39 AM, Mohammad Tariq <[email protected]> > >>> > wrote: > >>> >> > >>> >> Hello list, > >>> >> I am totally new to Chukwa and I have to learn and start using it > as > >>> >> soon soon as possible as a part of my new project. Could any one > let me know > >>> >> how to begin with?? It would be a great favor to me. I know Flume > and have > >>> >> used it already.Also I would like to know how different is Chukwa > from > >>> >> Flume?? > >>> >> Many thanks in advance. > >>> >> > >>> >> Regards, > >>> >> Mohammad Tariq > >>> > > >> > > > > >
