Re: Parsing httplog with java
Not really, it's common log format not combined, I need to capture referrer and user agent that is added in other place in the raw of haproxy log (with a specific format). And spécific data of haproxy could be interresting too. And by the way awstat cannot permit to store logs forever to requets new stats like Hbase. The idea is to do something like http://blog.mozilla.com/data/2011/03/22/how-glow-mozilla-org-gets-its-data/or http://hadoopblog.blogspot.com/2011/07/realtime-hadoop-usage-at-facebook.html Thanks you for helping. -- Damien 2011/7/4 Chris Sarginson ch...@sargy.co.uk Can you pull it from another app such as AWstats? It's just a standard log format? Chris Sent from my iPhone On 4 Jul 2011, at 13:34, Damien Hardy damienhardy@gmail.com wrote: Hi there, Does anyone have ever done the regex to parse the haproxy apachelog. (we want to inject logs in hbase via flume :) thank you. Cdt -- Damien
Re: Parsing httplog with java
Yes it could help us. I precised Java because there is no way in that language to tag regex captures like you have done in it (according to my deskmate). I transmit it now to the dev :) hope that could help us. many thanks, -- Damien 2011/7/4 Holger Just hapr...@meine-er.de Hi Damien, On 2011-07-04 14:34, Damien Hardy wrote: Does anyone have ever done the regex to parse the haproxy apachelog. (we want to inject logs in hbase via flume :) although it's not directly targeted for Java, but written in Python, but I have already posted my approach of parsing the HAProxy HTTP logs to this list some time ago. See http://permalink.gmane.org/gmane.comp.web.haproxy/5320 for the boilerplate script including the regex. Hope that helps, Holger
log X-Forwarded-For in haproxy log
I just finished setting up apache+mod_security in front of haproxy: user-- apache+modsec -- haproxy -- webservers -- fastcgi The reasoning being that if apache was behind haproxy, then the backend (nginx+php) servers wouldn't show on the haproxy admin interface (the apaches would). I'm not 100% sure if this is the best way to go about it, but for the time being that's the approach. Feel free to suggest/discuss alternatives. Because the site is live, I'm doing this in phases. For now the firewall on the load balancers redirects incoming connections from certain IPs to the new apache+modsec setup, while everything else is business as usual. The few connections that go through the test setup get logged by haproxy as coming from 127.0.0.1. This is because the firewall redirects to 127.0.0.1: (apache) which then ProxyPass'es to haproxy (127.0.0.1:); therefore haproxy sees an incoming connection from 127.0.0.1. Apache properly sets the X-Forwarded-For header. Question: Can I somehow tell haproxy to log that instead? If it is possible, are there security implications ?
[no subject]
My name is Mrs. Elizabeth KrUger;I am a dying woman who have being diagnosed with cancer,I have decided to donate what i have to you for charitable Work.i was diagnosed with cancer immediately after the death of my husband 2 years ago, who has left me everything he worked for and because the doctors told me i will not live longer than some weeks because of my health, i decided to WILL/donate the the Funds to you for the good work of humanity, and also to help the motherless and less privilege and also for the assistance of the widows. I know i don't know you but i have been directed to do this. Thanks and God bless.I will appreciate your utmost Quickiest reply. i God bless you, Mrs. Elizabeth KrUger
Re: log X-Forwarded-For in haproxy log
On Tue, 05 Jul 2011 16:17:24 +0100, Hugo Silva wrote:
I just finished setting up apache+mod_security in front of haproxy:
user-- apache+modsec -- haproxy -- webservers -- fastcgi
The reasoning being that if apache was behind haproxy, then the
backend
(nginx+php) servers wouldn't show on the haproxy admin interface (the
apaches would).
I'm not 100% sure if this is the best way to go about it, but for the
time being that's the approach. Feel free to suggest/discuss
alternatives.
Because the site is live, I'm doing this in phases. For now the
firewall
on the load balancers redirects incoming connections from certain IPs
to
the new apache+modsec setup, while everything else is business as
usual.
The few connections that go through the test setup get logged by
haproxy
as coming from 127.0.0.1. This is because the firewall redirects to
127.0.0.1: (apache) which then ProxyPass'es to haproxy
(127.0.0.1:); therefore haproxy sees an incoming connection from
127.0.0.1.
Apache properly sets the X-Forwarded-For header.
Question: Can I somehow tell haproxy to log that instead?
If it is possible, are there security implications ?
x-forwarded-for is a http header. like any other http header, you can
ask haproxy to log it by using
frontend XYZ
[...]
option httplog
capture request header X-Forwarded-For len 50
it will appear in the logs in field #14, enclosed between {}
characters.
http://code.google.com/p/haproxy-docs/wiki/HTTPLogFormat
Julien
n{开-发-票}u
BauerBarnett 您好： 我.公.司.现.有.剩.余.发.票.可.以.代.开， 有.需.要.请.来.电.详.谈。 电 话：135-689-13788 Q Q ：162-899-1615 联 系：刘 海 海.联.财.税 GilmoreButler
RE: Unsubscribe
Yea, let me know if that works for you. I am still trying to figure out how to get off this list. From: Kristoffer Sheather - Cloud Central [mailto:kristoffer.sheat...@cloudcentral.com.au] Sent: Tuesday, July 05, 2011 4:59 PM To: haproxy@formilux.org Subject: Unsubscribe Unsubscribe
