Re: Parsing httplog with java

[Damien Hardy]

Yes it could help us. I precised Java because there is no way in that
language to tag regex captures like you have done in it (according to my
deskmate).

I transmit it now to the dev :) hope that could help us.

many thanks,

-- 
Damien

2011/7/4 Holger Just 

> Hi Damien,
>
> On 2011-07-04 14:34, Damien Hardy wrote:
> > Does anyone have ever done the regex to parse the haproxy apachelog.
> > (we want to inject logs in hbase via flume :)
>
> although it's not directly targeted for Java, but written in Python, but
> I have already posted my approach of parsing the HAProxy HTTP logs to
> this list some time ago. See
> http://permalink.gmane.org/gmane.comp.web.haproxy/5320 for the
> boilerplate script including the regex.
>
> Hope that helps,
> Holger
>
>

Re: Parsing httplog with java

[Damien Hardy]

Not really, it's "common" log format not "combined", I need to capture
referrer and user agent that is added in other place in the raw of haproxy
log (with a specific format).
And spécific data of haproxy could be interresting too.

And by the way awstat cannot permit to store logs forever to requets new
stats like Hbase. The idea is to do something like
http://blog.mozilla.com/data/2011/03/22/how-glow-mozilla-org-gets-its-data/or
http://hadoopblog.blogspot.com/2011/07/realtime-hadoop-usage-at-facebook.html

Thanks you for helping.

-- 
Damien

2011/7/4 Chris Sarginson 

> Can you pull it from another app such as AWstats? It's just a standard log
> format?
>
> Chris
>
> Sent from my iPhone
>
> On 4 Jul 2011, at 13:34, Damien Hardy  wrote:
>
> > Hi there,
> >
> > Does anyone have ever done the regex to parse the haproxy apachelog.
> > (we want to inject logs in hbase via flume :)
> >
> > thank you.
> >
> > Cdt
> >
> > --
> > Damien
>
>

Re: Parsing httplog with java

[Holger Just]

Hi Damien,

On 2011-07-04 14:34, Damien Hardy wrote:
> Does anyone have ever done the regex to parse the haproxy apachelog.
> (we want to inject logs in hbase via flume :)

although it's not directly targeted for Java, but written in Python, but
I have already posted my approach of parsing the HAProxy HTTP logs to
this list some time ago. See
http://permalink.gmane.org/gmane.comp.web.haproxy/5320 for the
boilerplate script including the regex.

Hope that helps,
Holger

Re: Parsing httplog with java

[Chris Sarginson]

Can you pull it from another app such as AWstats? It's just a standard log 
format?

Chris

Sent from my iPhone

On 4 Jul 2011, at 13:34, Damien Hardy  wrote:

> Hi there,
> 
> Does anyone have ever done the regex to parse the haproxy apachelog.
> (we want to inject logs in hbase via flume :)
> 
> thank you.
> 
> Cdt
> 
> -- 
> Damien

Parsing httplog with java

[Damien Hardy]

Hi there,

Does anyone have ever done the regex to parse the haproxy apachelog.
(we want to inject logs in hbase via flume :)

thank you.

Cdt

-- 
Damien