This is dynamite I checked out the source from svn and I have undertaken initial analysis. Your hBase package looks like it inherits feom a nice interface face plus your documentation is very helpful. Thanks
On Sunday, February 15, 2015, Eric Yang <eric...@gmail.com> wrote: > Hi Lewis, > > There is an example in Chukwa to write errors out map reduce error to > InError file, and send out Nagios alert. > The example is in: > src/main/java/org/apache/hadoop/chukwa/extraction/demux/DemuxManager.java > > Nagios support is updated in CHUKWA-735 to support better encryption for > Nagios communication. > > Log4j Nagios appender maybe useful in CHUKWA-734. > > regards, > Eric > > On Sat, Feb 14, 2015 at 1:31 PM, Lewis John Mcgibbney < > lewis.mcgibb...@gmail.com > <javascript:_e(%7B%7D,'cvml','lewis.mcgibb...@gmail.com');>> wrote: > >> https://issues.apache.org/jira/browse/CHUKWA-734 >> >> On Sat, Feb 14, 2015 at 12:13 PM, Lewis John Mcgibbney < >> lewis.mcgibb...@gmail.com >> <javascript:_e(%7B%7D,'cvml','lewis.mcgibb...@gmail.com');>> wrote: >> >>> Hi Eric, >>> Thank you for the feedback. >>> This is more than helpful. >>> I am going to write a Gora module for Chuckwa. >>> I am going to progress on basis of implementing log monitor for Nutch. >>> Can Chuckwa currently write to file and email response? >>> Thanks >>> Lewis >>> >>> [0] http://gora.apache.org >>> >>> On Sat, Feb 14, 2015 at 9:30 AM, Eric Yang <eric...@gmail.com >>> <javascript:_e(%7B%7D,'cvml','eric...@gmail.com');>> wrote: >>> >>>> Hi Lewis, >>>> >>>> Parse error can be captured and store errors to another HDFS location. >>>> In Chukwa 0.4 and earlier, we have demux map reduce job which does the >>>> extraction and store structured data in HDFS, and errors are channel to >>>> another HDFS folder called InError, with the cause of the parsing error. >>>> This is still a batch oriented operation. In Chukwa 0.6, we can setup >>>> multiple pipeline writer. The pipeline writers can be configured to >>>> provide parsing and channel error to somewhere else, if data parse >>>> properly, then write it to HBase or HDFS. However, you will need to write >>>> the pipeline writer class to extend this functionality. We currently only >>>> have a couple pipeline writers, LocalWriter, HBaseWriter, and >>>> SeqFileWriter. SeqFileWriter needs to be the last one in the pipeline, if >>>> you choose to write data to HDFS. See this page for how to configure >>>> pipeline writer to achieve partially of what you are looking for: >>>> >>>> http://chukwa.apache.org/docs/r0.6.0/pipeline.html >>>> >>>> Hope this helps. >>>> >>>> regards, >>>> Eric >>>> >>>> On Thu, Feb 12, 2015 at 11:12 PM, Lewis John Mcgibbney < >>>> lewis.mcgibb...@gmail.com >>>> <javascript:_e(%7B%7D,'cvml','lewis.mcgibb...@gmail.com');>> wrote: >>>> >>>>> Hi Folks, >>>>> For some time I have been meaning to get in touch to get advice on >>>>> developing a tool for log analysis of Apache Nutch [0] logs. >>>>> What I am referring to particularly is monitoring of logs in a bid to >>>>> identify particular errors which we may anticipate. >>>>> Nutch jobs are batch oriented in architecture which are inherited from >>>>> Hadoop, we typically see errors in the parse phase of a crawl so it is >>>>> events like this that I would like to anticipate, monitor and report on, >>>>> possibly through email. >>>>> So I am therefore thinking about building a Chuckwa-powered tool for >>>>> Nutch which would become part of our codebase. >>>>> Is Chukwa the right tool for this? Any information about similar >>>>> efforts would be very much appreciated. >>>>> best >>>>> Lewis >>>>> >>>>> [0] http://nutch.apache.org >>>>> >>>>> -- >>>>> *Lewis* >>>>> >>>> >>>> >>> >>> >>> -- >>> *Lewis* >>> >> >> >> >> -- >> *Lewis* >> > > -- *Lewis*
