Test, please ignore

2012-08-09 Thread Willy Tarreau
This is a test to check if list is back online.
Please ignore.

Willy




log format different and CAPTURE_LEN settings

2012-08-09 Thread Aleksandar Lazic

Hi,

I wanted to add the uniq-id logging to the http-log format, I just 
copied the format

string from src/log.c but I got different log entries. (see below)

I also needed to capture more then 63 bytes so I have build HAProxy 
like this


make TARGET=linux26 USE_LINUX_SPLICE=1 USE_STATIC_PCRE=1 # and added 
SMALL_OPTS = -DCAPTURE_LEN=256


After rebuild I still get the warning that the I can only capture 63 
bytes.

I have change include/common/defaults.h to

###
// reserved buffer space for header capture
#ifndef CAPTURE_LEN
#define CAPTURE_LEN 64
#endif
###

and rebuild it.

default format %Ci:%Cp  [%t]  %f %b/%s   %Tq/%Tw/%Tc/%Tr/%Tt  %st   %B 
%cc %cs %tsc

   %ac/%fc/%bc/%sc/%rc   %sq/%bq %hr %hs %{+Q}r

my log-format  %Ci:%Cp\ [%t]\ %f\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %st\ %B\ 
%cc\ %cs\ %tsc\

   %ac/%fc/%bc/%sc/%rc\ %ID\ %sq/%bq\ %hr\ %hs\ %{+Q}r

Output of default with original CAPTURE_LEN

Aug  8 15:54:00 localhost.localdomain haproxy[7295]: 
188.165.15.140:49534 [08/Aug/2012:15:53:58.819]
fe_panomax delivery/nginx 0/0/0/1563/1563 200 214 - -  2/2/0/0/0 32 
0/0 {Mozilla/4.0 (compatible;)}

POST REQUEST HTTP/1.1

Output of my log-format with new CAPTURE_LEN
Aug  8 16:03:55 localhost.localdomain haproxy[9810]: 
188.165.15.140:42660 [08/Aug/2012:16:03:54.455]
fe_panomax delivery/- 0/0/0/1344/+1344 200 +128 - -  5/5/1/0/0 143 
0/0 POST REQUEST HTTP/1.1


now original with new CAPTURE_LEN
Aug  8 16:11:42 localhost.localdomain haproxy[15396]: 
188.165.15.140:56583 [08/Aug/2012:16:11:42.199]
fe_panomax delivery/- 0/0/0/716/+716 200 +128 - -  4/4/1/0/0 0/0 
POST REQUEST HTTP/1.1


now original with original CAPTURE_LEN = currently running
Aug  8 16:24:52 localhost.localdomain haproxy[22350]: 
188.165.15.140:49382 [08/Aug/2012:16:24:50.269]
fe_panomax delivery/- 0/0/0/2690/+2690 200 +128 - -  5/5/1/0/0 0/0 
POST REQUEST HTTP/1.1


Currently running

haproxy-1.5-dev11# ./haproxy -vv
HA-Proxy version 1.5-dev11 2012/06/04
Copyright 2000-2012 Willy Tarreau w...@1wt.eu

Build options :
  TARGET  = linux26
  CPU = generic
  CC  = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing
  OPTIONS = USE_LINUX_SPLICE=1 USE_STATIC_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 
200


Encrypted password support via crypt(3): yes

Available polling systems :
 sepoll : pref=400,  test result OK
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 4 (4 usable), will use sepoll.

my config:

grep -v '^(#|$)' /home/al/download/haproxy/panomax.cfg
global
daemon
pidfile /var/run/haproxy.pid
maxconn 6  # warning: this has to be 3 times the expected 
value!

log 127.0.0.1 local0
ulimit-n 120022

defaults
modehttp
balance roundrobin
option  dontlognull
option  httplog
option http-server-close
option redispatch
option forwardfor
option http-no-delay
option httpchk GET /favicon.ico HTTP/1.0
unique-id-format %{+X}o\ %Ci:%Cp_%Ts_%rt
unique-id-header X-Unique-ID
balance source
retries 1
maxconn 2000
contimeout  5000
clitimeout  5
srvtimeout  5

frontend fe_panomax
bind IP:80
option forwardfor   # add 'X-Forwarded-For: IP'
log global

capture request header User-Agent len 128
capture cookie PHPSESSID len 128

rspdel ^X-Powered-By:.*

acl stat_request url_beg /haproxy_stats
use_backend stats_backend if stat_request

acl fpm_request url_beg /fpm_status
use_backend default if fpm_request

acl host_delivery   hdr_beg(host) -i delivery
use_backend delivery if host_delivery

acl host_admin   hdr_beg(host) -i admin
use_backend admin if host_admin

acl host_static   hdr_beg(host) -i static
use_backend static if host_static

acl host_panodata   hdr_beg(host) -i panodata
use_backend panodata if host_panodata

# send everything to next stage
default_backend default

backend default
log 127.0.0.1 local1
option  httplog
option httpchk GET /favicon.ico HTTP/1.0
server  nginx IP:81 maxconn 500 check inter 5s fall 3

backend panodata
#127.0.0.3:8002
log global
option  httplog
option  httpchk GET /crossdomain.xml HTTP/1.0
server  nginx IP:80 check inter 5s fall 3

backend delivery
#127.0.0.3:8002
log global
option  httplog
option  httpchk GET /crossdomain.xml HTTP/1.0
server  nginx IP:81 check inter 5s fall 3

backend static
#127.0.0.3:8002
log global
option  httplog
option  httpchk GET /crossdomain.xml HTTP/1.0
server  nginx IP:80 check inter 5s fall 3

backend admin
#127.0.0.3:8002
log global
option  httplog
option  httpchk GET /img/css-nav.gif HTTP/1.0
server  nginx IP:81 check inter 5s fall 3

backend stats_backend
mode http
balance
timeout connect 4000
timeout server 3
stats uri 

Re: log format different and CAPTURE_LEN settings

2012-08-09 Thread William Lallemand
On Thu, Aug 09, 2012 at 03:16:07PM +0200, Aleksandar Lazic wrote:
 Hi,


Hello,

 [...]
 
 As you can see I have not 'option logasap' but get the '+'-sign?!
  
 Please can anybody help me to find the error, thanks.
 
 Best regards
 Aleks
 

It looks like a bug with the option unique-id-format.

Can you try this patch ?


-- 
William Lallemand
From 7d40e9f6d3f8f1c5ce09e264226a1e5e369d70a0 Mon Sep 17 00:00:00 2001
From: William Lallemand wlallem...@exceliance.fr
Date: Thu, 9 Aug 2012 16:41:35 +0200
Subject: [PATCH] BUG/MINOR: to_log erased with unique-id-format

curproxy-to_log was reset to LW_INIT when using unique-id-format,
so logs looked like option logasap
---
 src/log.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/log.c b/src/log.c
index 2a3cd16..b1f532a 100644
--- a/src/log.c
+++ b/src/log.c
@@ -309,7 +309,7 @@ void parse_logformat_string(char *str, struct proxy *curproxy, struct list *list
 	struct logformat_node *tmplf, *back;
 	int options = 0;
 
-	curproxy-to_log = LW_INIT;
+	curproxy-to_log |= LW_INIT;
 
 	/* flush the list first. */
 	list_for_each_entry_safe(tmplf, back, list_format, list) {
-- 
1.7.9.5



Re: HAProxy stops working all of a sudden

2012-08-09 Thread Rahul Nair
Group,

Any clues on this issue..?

Thanks
Rahul N.

On Thursday, August 9, 2012, Rahul Nair rahul.n...@finicity.com wrote:
 Hello All,
 Please help me on this issue.
 Thanks,
 Rahul N.

 On Thu, Aug 9, 2012 at 12:13 AM, Rahul Nair rahul.n...@finicity.com
wrote:

 Guys,
 I am in process of implementing HAProxy with TPROXY in our setup for
mode tcp.
 All of a sudden the website stops working and gives out error in
browser: Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error.
 When I remove/comment  source 0.0.0.0 usesrc clientip the website
starts working fine.
 And later on when I again enable source 0.0.0.0 usesrc clientip it
starts working fine, It seems that the issue is intermittent.
 Please help me understand what exactly the problem could be.
 Hardware configuration of HAProxy server:
 RAM:256MB
 Processor:Single core
 Thanks,
 Rahul N.




 --
 -Rahul N.
 IT Department
 In2M Technologies Pvt Ltd. (Finicity)
 Website: www.finicity.com/india


-- 
Sent from Gmail Mobile


Re: log format different and CAPTURE_LEN settings

2012-08-09 Thread Aleksandar Lazic

Hi William,

On 09-08-2012 16:52, William Lallemand wrote:

On Thu, Aug 09, 2012 at 03:16:07PM +0200, Aleksandar Lazic wrote:

Hi,



Hello,


[...]

As you can see I have not 'option logasap' but get the '+'-sign?!

Please can anybody help me to find the error, thanks.

Best regards
Aleks



It looks like a bug with the option unique-id-format.

Can you try this patch ?


after the first minute the log is now as defined ;-)

Thanks.

Cheers
Aleks



Re: log format different and CAPTURE_LEN settings

2012-08-09 Thread Willy Tarreau
On Thu, Aug 09, 2012 at 06:57:16PM +0200, Aleksandar Lazic wrote:
 Hi William,
 
 On 09-08-2012 16:52, William Lallemand wrote:
 On Thu, Aug 09, 2012 at 03:16:07PM +0200, Aleksandar Lazic wrote:
 Hi,
 
 
 Hello,
 
 [...]
 
 As you can see I have not 'option logasap' but get the '+'-sign?!
 
 Please can anybody help me to find the error, thanks.
 
 Best regards
 Aleks
 
 
 It looks like a bug with the option unique-id-format.
 
 Can you try this patch ?
 
 after the first minute the log is now as defined ;-)

Thanks guys, patch applied.

Willy




Re: HAProxy stops working all of a sudden

2012-08-09 Thread Willy Tarreau
Hello Rahul,

On Thu, Aug 9, 2012 at 12:13 AM, Rahul Nair rahul.n...@finicity.com wrote:
 Guys,
 I am in process of implementing HAProxy with TPROXY in our setup for mode 
 tcp.
 All of a sudden the website stops working and gives out error in browser: 
 Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error.
 When I remove/comment  source 0.0.0.0 usesrc clientip the website starts 
 working fine.
 And later on when I again enable source 0.0.0.0 usesrc clientip it starts 
 working fine, It seems that the issue is intermittent.
 Please help me understand what exactly the problem could be.
 Hardware configuration of HAProxy server:
 RAM:256MB
 Processor:Single core
 Thanks,
 Rahul N.

From your description, it could be an issue with some connection tracking
somewhere caused by excess of source addr:ports. But it could be many things.
What's your haproxy version and kernel version ? Are you sure all your
servers route back through your haproxy box ? Did you test only from one
source machine or did you have many clients ?

Willy




Re: HAProxy stops working all of a sudden

2012-08-09 Thread Rahul Nair
Willy,

From your description, it could be an issue with some connection
tracking somewhere caused by excess of source addr:ports.

Ohh ok..
Also I just found that as per the documentation in this link , it says that
it can cause problems when IP connection tracking is enabled on the
machine, because a same connection may be seen twice with different states.
Does this mean that I need to disable the  nf_conntrack module by adding
net.netfilter.nf_conntrack_acct = 0  to /etc/sysctl.conf ?

Bu default this module seems to be enabled.
 cat /proc/sys/net/netfilter/nf_conntrack_acct
1

Following are the answers to your questions:

What's your haproxy version and kernel version ?

   - HA-Proxy version: 1.4.8 2010/06/16
   - Kernel Version: 2.6.32-24-server
   - OS: Ubuntu 10.04


Are you sure all your servers route back through your haproxy box ?

   - Yes the default gateway of all the real servers is HAProxy server.
   - On real servers I have multiple IPs of two different networks
  - One which we use for communication between HAproxy server and Real
  servers.
  - And One which is used by the real servers to communicate with our
  internal application servers

Did you test only from one source machine or did you have many clients ?

   - This issue occurs intermittently from one or two different source IPs
   - At the same time when I check the functionality from another source
   IP, it works fine.

Thanks
Rahul N.

On Thu, Aug 9, 2012 at 10:56 PM, Willy Tarreau w...@1wt.eu wrote:

 Hello Rahul,

 On Thu, Aug 9, 2012 at 12:13 AM, Rahul Nair rahul.n...@finicity.com
 wrote:
  Guys,
  I am in process of implementing HAProxy with TPROXY in our setup for
 mode tcp.
  All of a sudden the website stops working and gives out error in
 browser: Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error.
  When I remove/comment  source 0.0.0.0 usesrc clientip the website
 starts working fine.
  And later on when I again enable source 0.0.0.0 usesrc clientip it
 starts working fine, It seems that the issue is intermittent.
  Please help me understand what exactly the problem could be.
  Hardware configuration of HAProxy server:
  RAM:256MB
  Processor:Single core
  Thanks,
  Rahul N.

 From your description, it could be an issue with some connection tracking
 somewhere caused by excess of source addr:ports. But it could be many
 things.
 What's your haproxy version and kernel version ? Are you sure all your
 servers route back through your haproxy box ? Did you test only from one
 source machine or did you have many clients ?

 Willy




-- 
-Rahul N.
IT Department
In2M Technologies Pvt Ltd. (Finicity)
Website: www.finicity.com/india


Re: log format different and CAPTURE_LEN settings

2012-08-09 Thread Aleksandar Lazic

Hi will,

On 09-08-2012 19:21, Willy Tarreau wrote:

On Thu, Aug 09, 2012 at 06:57:16PM +0200, Aleksandar Lazic wrote:


[snip]


after the first minute the log is now as defined ;-)


Thanks guys, patch applied.


how about the CAPTURE_LEN setting.

###
After rebuild I still get the warning that the I can only capture 63 
bytes.

I have change include/common/defaults.h to

###
// reserved buffer space for header capture
#ifndef CAPTURE_LEN
#define CAPTURE_LEN 64
#endif
###

and rebuild it.
###

Best regards
Aleks



Re: log format different and CAPTURE_LEN settings

2012-08-09 Thread Willy Tarreau
Hi Aleks,

On Fri, Aug 10, 2012 at 12:56:18AM +0200, Aleksandar Lazic wrote:
 Hi will,
 
 On 09-08-2012 19:21, Willy Tarreau wrote:
 On Thu, Aug 09, 2012 at 06:57:16PM +0200, Aleksandar Lazic wrote:
 
 [snip]
 
 after the first minute the log is now as defined ;-)
 
 Thanks guys, patch applied.
 
 how about the CAPTURE_LEN setting.
 
 ###
 After rebuild I still get the warning that the I can only capture 63 
 bytes.
 I have change include/common/defaults.h to
 
 ###
 // reserved buffer space for header capture
 #ifndef CAPTURE_LEN
 #define CAPTURE_LEN 64
 #endif
 ###
 
 and rebuild it.
 ###

It's unclear to me what is causing you an issue with this one, because
it's only used for cookie captures now (you know, the request and response
cookie specified in capture cookie). Also, you shouldn't set it too
high because this memory is allocated for each session.

Regards,
Willy




Re: git clone haproxy repo is not working? server returns 404 not found

2012-08-09 Thread Willy Tarreau
Hi Han,

On Fri, Aug 10, 2012 at 10:51:47AM +0800, Han He wrote:
 Hi,
 
 I have problems to clone the haproxy repo, does anyone have the same problem?
 
 No response after the following commands:
 # git clone http://git.1wt.eu/git/haproxy.git
 
 I did a packets capture, and found git server return 404 not found for
 some requests.

This is something I've already encountered when I was using alternate
files. I have already noticed this issue with objects which were in a
pack and which were still requested as an object of their own by a
specific client. The issue disappeared after a git-gc. So I've done it
now, please let me know if it's better.

Regards,
Willy




Re: HAProxy stops working all of a sudden

2012-08-09 Thread Rahul Nair
Willy,

I have  upgraded the Linux kernel to and haproxy to 1.4.18 and kernel
to 2.6.38-15-server
Will monitor it for few days and will let you know the updates.

-Rahul N.


On Fri, Aug 10, 2012 at 2:04 AM, Willy Tarreau w...@1wt.eu wrote:

 On Thu, Aug 09, 2012 at 11:54:08PM +0530, Rahul Nair wrote:
  Willy,
 
  From your description, it could be an issue with some connection
  tracking somewhere caused by excess of source addr:ports.
 
  Ohh ok..
  Also I just found that as per the documentation in this link , it says
 that
  it can cause problems when IP connection tracking is enabled on the
  machine, because a same connection may be seen twice with different
 states.
  Does this mean that I need to disable the  nf_conntrack module by adding
  net.netfilter.nf_conntrack_acct = 0  to /etc/sysctl.conf ?

 You can't disable nf_conntrack using a sysctl. You need to unload the
 module itself. It's not nf_conntrack_acct but nf_conntrack.

  Bu default this module seems to be enabled.
   cat /proc/sys/net/netfilter/nf_conntrack_acct
  1
 
  Following are the answers to your questions:
 
  What's your haproxy version and kernel version ?
 
 - HA-Proxy version: 1.4.8 2010/06/16

 Be careful, this is quite outdated ! 2 years of fixes have been merged
 since :
  $ git log --pretty=oneline v1.4.8..|grep -c BUG
  72

 = Your version has 72 bugs that have already been fixed now.
I don't remember of any affecting transparent proxying though, but
when you fix the issue you'd be advised to update it.

 - Kernel Version: 2.6.32-24-server
 - OS: Ubuntu 10.04

 You should also check that your kernel is up to date, as what you're
 observing might as well simply be a kernel bug.

  Are you sure all your servers route back through your haproxy box ?
 
 - Yes the default gateway of all the real servers is HAProxy server.
 - On real servers I have multiple IPs of two different networks
- One which we use for communication between HAproxy server and
 Real
servers.
- And One which is used by the real servers to communicate with our
internal application servers

 OK.

  Did you test only from one source machine or did you have many clients ?
 
 - This issue occurs intermittently from one or two different source
 IPs
 - At the same time when I check the functionality from another source
 IP, it works fine.

 Fine, then it really makes me think about a conntrack issue. Also, you
 should ensure that your client never directly talks to the server without
 passing via haproxy (which I can imagine you do during your tests when
 observing the issue). It only makes the problem worse with conntrack.

 Regards,
 Willy




-- 
-Rahul N.
IT Department
In2M Technologies Pvt Ltd. (Finicity)
Website: www.finicity.com/india


Re: HAProxy stops working all of a sudden

2012-08-09 Thread Rahul Nair
Willy,

The issue still persists.
Not sure what am I missing.

-Rahul N.

On Friday, August 10, 2012, Rahul Nair rahul.n...@finicity.com wrote:
 Willy,
 I have  upgraded the Linux kernel to and haproxy to 1.4.18 and kernel
to 2.6.38-15-server
 Will monitor it for few days and will let you know the updates.
 -Rahul N.

 On Fri, Aug 10, 2012 at 2:04 AM, Willy Tarreau w...@1wt.eu wrote:

 On Thu, Aug 09, 2012 at 11:54:08PM +0530, Rahul Nair wrote:
  Willy,
 
  From your description, it could be an issue with some connection
  tracking somewhere caused by excess of source addr:ports.
 
  Ohh ok..
  Also I just found that as per the documentation in this link , it says
that
  it can cause problems when IP connection tracking is enabled on the
  machine, because a same connection may be seen twice with different
states.
  Does this mean that I need to disable the  nf_conntrack module by
adding
  net.netfilter.nf_conntrack_acct = 0  to /etc/sysctl.conf ?

 You can't disable nf_conntrack using a sysctl. You need to unload the
 module itself. It's not nf_conntrack_acct but nf_conntrack.

  Bu default this module seems to be enabled.
   cat /proc/sys/net/netfilter/nf_conntrack_acct
  1
 
  Following are the answers to your questions:
 
  What's your haproxy version and kernel version ?
 
 - HA-Proxy version: 1.4.8 2010/06/16

 Be careful, this is quite outdated ! 2 years of fixes have been merged
 since :
  $ git log --pretty=oneline v1.4.8..|grep -c BUG
  72

 = Your version has 72 bugs that have already been fixed now.
I don't remember of any affecting transparent proxying though, but
when you fix the issue you'd be advised to update it.

 - Kernel Version: 2.6.32-24-server
 - OS: Ubuntu 10.04

 You should also check that your kernel is up to date, as what you're
 observing might as well simply be a kernel bug.

  Are you sure all your servers route back through your haproxy box ?
 
 - Yes the default gateway of all the real servers is HAProxy server.
 - On real servers I have multiple IPs of two different networks
- One which we use for communication between HAproxy server and
Real
servers.
- And One which is used by the real servers to communicate with
our
internal application servers

 OK.

  Did you test only from one source machine or did you have many clients
?
 
 - This issue occurs intermittently from one or two different source
IPs
 - At the same time when I check the functionality from another
source
 IP, it works fine.

 Fine, then it really makes me think about a conntrack issue. Also, you
 should ensure that your client never directly talks to the server without
 passing via haproxy (which I can imagine you do during your tests when
 observing the issue). It only makes the problem worse with conntrack.

 Regards,
 Willy




 --
 -Rahul N.
 IT Department
 In2M Technologies Pvt Ltd. (Finicity)
 Website: www.finicity.com/india


-- 
-Rahul N.
IT Department
In2M Technologies Pvt Ltd. (Finicity)
Website: www.finicity.com/india