Thanks a lot for your responses. I think i can start with prducer-in-app approach.
I saw the logger option as well- Kafka appender. Will it be more faster and efficient? Anybody tried that? On Fri, Apr 6, 2012 at 12:58 AM, Felix GV <fe...@mate1inc.com> wrote: > As Hisham mentioned, what I've been working on is your option #2, and that > can be done by using the Kafka APIs... > > Currently, the easiest way to get this up and running quickly would > probably be your option #1, using the kafka-console-producer that was added > by the kind Kafka folks in KAFKA-130 :) > > (It's available in 0.7) > > -- > Felix > > > > On Wed, Apr 4, 2012 at 9:56 PM, Hisham Mardam-Bey <his...@mate1inc.com > >wrote: > > > On Wed, Apr 4, 2012 at 5:20 AM, navneet sharma > > <navneetsharma0...@gmail.com> wrote: > > > Hi, > > > > > > I am new to kafka. If i have an application which is generating logs > and > > i > > > need to read and do some CEP on these logs. > > > > > > For CEP, we are planning to use Esper and for reading weblogs, we are > > > examining Kafka. I have few questions: > > > 1) What will be a producer in this case. Will it be an external > > > entity(standalone program) which will do file I/o and send weblogs to > > kafka > > > broker? > > > 2) Or will it be part of application itself, so that it can read logs > in > > > parallel with the logging utility? > > > > > > Any pointers will be helpful... > > > > > > Thanks, > > > Navneet > > > > Navneet, > > > > We're currently doing that for our Tomcat servers. We've built an > > access log valve (inspired by the one bundled with Tomcat that writes > > log files to disk) that will format logs and push them into Kafka. We > > then pull the logs and push them into HBase for analysis. FelixGV (on > > this list) is implementing the code if you have questions for him. > > > > Hope this helps, > > > > hisham. > > > > -- > > Hisham Mardam Bey > > > > A: Because it messes up the order in which people normally read text. > > Q: Why is top-posting such a bad thing? > > A: Top-posting. > > Q: What is the most annoying thing in e-mail? > > > > -=[ Codito Ergo Sum ]=- > > >
