Re: Set balance cookie domain dynamically

2011-09-08 Thread Willy Tarreau
Hi,

On Thu, Sep 01, 2011 at 01:40:22PM +1000, Leigh Dyer wrote:
 We allow our users to create their own custom domains, so it's several 
 thousand domains at the moment. We don't have to worry about subdomains 
 on those custom domains, though, so I could probably use one backend for 
 our primary domain plus its subdomains, and a second backend that leaves 
 the domain on the cookie blank. Would that work?

It's not even certain :-/

I had to run a number of tests about one year ago on cookie domains to
see how browsers used to process them. There are about as many behaviours
as there are browsers. Some accept to learn a cookie for a domain which is
different from the one they visit, some don't. Some are very restrictive
and strictly apply the rule from RFC2965 (subdomain matching exactly the
full hostname minus the first component), some also accept a full host name
in the cookie, some even accept an IP address.

Given all the security impacts there are in injecting cookies in browsers,
we can only expect a progressive hardening in browser rules, even if this
means breaking some existing apps.

I invite you to read RFC6265 which replaces 2965 and puts the real issues
on the table as they currently are.

What you can do however at the haproxy level is to use the cookie prefix
mode. The idea is that you'll use the application cookie as a carrier for
the stickiness. This means you don't need to worry about those rules in
your haproxy configuration, you'll automatically adapt to any tricks that
your customers might invent for their apps. If they're able to pass a session
cookie between two domains, then haproxy will find it.

Regards,
Willy




Re: httpclose/forceclose and TCP states

2011-09-08 Thread Willy Tarreau
On Thu, Sep 01, 2011 at 09:04:41PM -0400, Chris Burroughs wrote:
 I'm trying to figure out what exactly the httpclose/forceclose is doing
 when it forces the closing of the outgoing server channel as soon as
 the server begins to reply and only if the request buffer is empty.  Is
 it sending a RST?

No, otherwise it would abort the server's response. It's doing a
shutdown(SHUT_WR), which results in a FIN on the wire. Anyway, as of 1.4
we don't do that anymore since we're aware of the end of response. So we
close the connection as soon as we get the whole response from the server
and here yes, we close with an RST (at least we try to do so, since we
don't control this).

 I've looked at the source code and I think that's what's going on, but
 it has been a while since I've read C networking code.

It depends what version you're reading :-)

Regards,
Willy




Re: httpclose/forceclose and TCP states

2011-09-08 Thread Willy Tarreau
On Tue, Sep 06, 2011 at 07:01:44PM -0400, Chris Burroughs wrote:
 On 09/01/2011 09:04 PM, Chris Burroughs wrote:
  I've looked at the source code and I think that's what's going on, but
  it has been a while since I've read C networking code.
 
 If someone is in a particularly explanatory mood, I'm also trying to
 figure out how haproxy handles the SO_LINGER blocking/throws-away-data
 trap.  Apache httpd for example does this:
 https://github.com/apache/httpd/blob/trunk/server/connection.c#L43

Those are complex issues and we had to perform some changes in the past.
To make it short, by default the system handles orphans, which are
connections that have been closed but still have unacked data. This is
very common with protocols working in question/response/close mode, as
the server closes after sending the response.

An issue was introduced with keep-alive support in HTTP : the client may
send a new request after the first one. As long as the client waits for
the whole server response, it doesn't cause any issue. But if the client
talks before the end of response, we risk causing the server to emit an
RST and destroy part of the in-flight response. This situation happens
with pipelining, because the client is pushing new requests before the
server responds. In practice, browsers generally don't pipeline after the
first request, so they can detect a server that would systematically close.
But this can still happen if the server is wishing to close several objects
later. What haproxy is doing is to read everything it can on the request
while sending a response, so that we limit the risk of having unacked data
in the kernel buffers in the event of a close. We had to do this recently
because a browser was systematically sending a CRLF approximately one second
after each post, and this CRLF was not consumed.

Since you have no way to be notified when the client has ACKed all the data,
the only remaining solution to this mess is to drain everything from the
client when you want to close. But this is a real mess when you're sending
a 302 or 403 on a POST request ! You have to read all the data you're not
interested in, causing them to pass over the network and taking a lot of
client time, just because you can't be notified that your FIN was read.

Under linux, we're also able to issue a getsockopt() at the TCP level to
check if our data were completely ACKed. But still, this requires active
polling, because you're not notified for that. So if the client receives
your data and disconnects from the net without closing the other side,
you're never notified.

Ideally we should adapt systems so that they can inform apps when it's
possible to close, because the systems themselves do know it. For instance,
we could have poll() return POLLOUT after a shutdown(SHUT_WR) to indicate
that it's now safe to close.

But without this, were doing as most other products : cover the common
cases in a reasonable way, not the perfect way.

Regards,
Willy




[PATCH v2] *_dom matching header functions now also split on :

2011-09-08 Thread Finn Arne Gangstad
*_dom is mostly used for matching Host headers, and host headers may
include port numbers. To avoid having to create multiple rules with
and without :port-number in hdr_dom rules, change the *_dom
matching functions to also handle : as a delimiter.

Typically there are rules like this in haproxy.cfg:

  acl is_foo  hdr_dom(host) www.foo.com

Most clients send Host: www.foo.com in their HTTP header, but some
send Host: www.foo.com:80 (which is allowed), and the above
rule will now work for those clients as well.
---

There are two versions here, one straightforward version which is more
or less identical to the existing code in performance, and one
optimized version which is significantly faster (at least on a 2 year
old core2 and a i7-2600, all I have to test on).



 src/acl.c |   55 ---
 1 files changed, 40 insertions(+), 15 deletions(-)

diff --git a/src/acl.c b/src/acl.c
index 9d9a746..cb49b43 100644
--- a/src/acl.c
+++ b/src/acl.c
@@ -546,12 +546,38 @@ int acl_match_sub(struct acl_test *test, struct 
acl_pattern *pattern)
return ACL_PAT_FAIL;
 }
 
+#if 0
+/* Straightforward implementation first - resaonably quick, but
+   can do better. */
+typedef const char *delimiter_mask_t;
+#define DELIMITER_MASK(a,b,c,d) a b c d
+#define IS_DELIMITER(c) (delimiter[0] == (c) || delimiter[1] == (c) || 
delimiter[2] == (c) || delimiter[3] == (c))
+
+#else
+
+/* Fast version. Background: Fast way to find a zero byte in a word
+ * http://graphics.stanford.edu/~seander/bithacks.html#ZeroInWord
+ * hasZeroByte = (v - 0x01010101UL)  ~v  0x80808080UL;
+ *
+ * To look for 4 different byte values, xor the word with those bytes and
+ * then check for zero bytes:
+ *
+ * v = (((unsigned char)c * 0x1010101U) ^ delimiter)
+ * where delimiter is the 4 byte values to look for (as an uint)
+ * and c is the character that is being tested
+ */
+typedef unsigned int delimiter_mask_t;
+#define DELIMITER_MASK(a,b,c,d) ((unsigned char)(*a) | (unsigned char)(*b)  
8 | (unsigned char)(*c)  16 | (unsigned char)(*d)  24)
+#define IS_DELIMITER(c) (unsigned char)c * 0x1010101U) ^ delimiter) - 
0x01010101)  ~(((unsigned char)c * 0x1010101U) ^ delimiter)  0x80808080U)
+
+#endif
+
 /* This one is used by other real functions. It checks that the pattern is
  * included inside the tested string, but enclosed between the specified
- * delimitor, or a '/' or a '?' or at the beginning or end of the string.
- * The delimitor is stripped at the beginning or end of the pattern.
+ * delimiters or at the beginning or end of the string.
+ * Delimiters are stripped at the beginning and end of the pattern.
  */
-static int match_word(struct acl_test *test, struct acl_pattern *pattern, char 
delim)
+static int match_word(struct acl_test *test, struct acl_pattern *pattern, 
delimiter_mask_t delimiter)
 {
int may_match, icase;
char *c, *end;
@@ -560,13 +586,12 @@ static int match_word(struct acl_test *test, struct 
acl_pattern *pattern, char d
 
pl = pattern-len;
ps = pattern-ptr.str;
-   while (pl  0  (*ps == delim || *ps == '/' || *ps == '?')) {
+   while (pl  0  IS_DELIMITER(*ps)) {
pl--;
ps++;
}
 
-   while (pl  0 
-  (ps[pl - 1] == delim || ps[pl - 1] == '/' || ps[pl - 1] == '?'))
+   while (pl  0  IS_DELIMITER(*ps))
pl--;
 
if (pl  test-len)
@@ -576,7 +601,7 @@ static int match_word(struct acl_test *test, struct 
acl_pattern *pattern, char d
icase = pattern-flags  ACL_PAT_F_IGNORE_CASE;
end = test-ptr + test-len - pl;
for (c = test-ptr; c = end; c++) {
-   if (*c == '/' || *c == delim || *c == '?') {
+   if (IS_DELIMITER(*c)) {
may_match = 1;
continue;
}
@@ -587,12 +612,12 @@ static int match_word(struct acl_test *test, struct 
acl_pattern *pattern, char d
if (icase) {
if ((tolower(*c) == tolower(*ps)) 
(strncasecmp(ps, c, pl) == 0) 
-   (c == end || c[pl] == '/' || c[pl] == delim || 
c[pl] == '?'))
+   (c == end || IS_DELIMITER(c[pl])))
return ACL_PAT_PASS;
} else {
if ((*c == *ps) 
(strncmp(ps, c, pl) == 0) 
-   (c == end || c[pl] == '/' || c[pl] == delim || 
c[pl] == '?'))
+   (c == end || IS_DELIMITER(c[pl])))
return ACL_PAT_PASS;
}
may_match = 0;
@@ -601,21 +626,21 @@ static int match_word(struct acl_test *test, struct 
acl_pattern *pattern, char d
 }
 
 /* Checks that the pattern is included inside the tested string, but enclosed
- * between slashes or at the beginning or end of the string. Slashes at the
- * beginning or end of 

Error 504

2011-09-08 Thread Christophe Rahier
Hi,

I've a question about this error :
504 Gateway Time-out
The server didn't respond in time.

What could I check in my config ? I created 2 LB with a virtual IP and all 
request are coming from the firewall to this IP.

I think it's possible, if needed, I can copy my configuration file.

Thanks for your help, I'm lost.

Regards, Christophe


Re: Error 504

2011-09-08 Thread Baptiste
Hello,

you server might be very slow or your server timeout in your conf
might be too low.

If you can copy/paste your conf and tell us which version you're using
and the underlying OS.

cheers


On Thu, Sep 8, 2011 at 1:35 PM, Christophe Rahier
christo...@qualifio.com wrote:
 Hi,
 I've a question about this error :

 504 Gateway Time-out

 The server didn't respond in time.

 What could I check in my config ? I created 2 LB with a virtual IP and all
 request are coming from the firewall to this IP.
 I think it's possible, if needed, I can copy my configuration file.
 Thanks for your help, I'm lost.
 Regards, Christophe



Re: [PATCH] Read acl included files relative to the configuration file

2011-09-08 Thread Brane F. Gračnar
On Thursday 08 of September 2011 11:21:34 Finn Arne Gangstad wrote:
 On Thu, Sep 08, 2011 at 07:43:46AM +0200, Willy Tarreau wrote:
  Hi again,
  
  This morning I had an better idea : pass the config directory parameter
  on the command line and have haproxy chdir() to it. That way, everything
  specified after it is relative to this dir, and you don't need a full
  path
  
  for config files. Eg :
  haproxy -C /etc/haproxy -f haproxy.cfg
  
  I think it's easier to explain and to understand than previous proposal,
  and it can completely solve your multi-machine issue (well doing cd
  before starting haproxy also does, but I agree it can be less convenient,
  especially when copy-pasting a command line from ps).

What about settings this varible by itself, for example, setting it by 
dirname(config_file) inside haproxy?

Brane



Re: Error 504

2011-09-08 Thread Christophe Rahier
Hi,

Here's my config. Webservers are IIS.

global
log 192.168.0.2 local0
log 127.0.0.1 local1 notice
maxconn 10240
defaults
logglobal
option dontlognull
retries2
clitimeout  5
srvtimeout  5
contimeout  5
timeout server 60s

listen WebPlayer-Farm 192.168.0.2:80
mode http
option httplog
balance source
#balance leastconn
option forwardfor
stats enable
option http-server-close
server Player1 192.168.0.10:80 check
server Player2 192.168.0.11:80 check
server Player3 192.168.0.12:80 check
server Player4 192.168.0.13:80 check

listen WebPlayer-Farm-SSL 192.168.0.2:443
mode tcp
option ssl-hello-chk
balance source
server Player1 192.168.0.10:443 check
server Player2 192.168.0.11:443 check
server Player3 192.168.0.12:443 check
server Player4 192.168.0.13:443 check

listen  Manager-Farm192.168.0.2:81
mode http
option httplog
balance source
option forwardfor
stats enable
option http-server-close
server  Manager1 192.168.0.60:80 check
server  Manager2 192.168.0.61:80 check

listen Manager-Farm-SSL 192.168.0.2:444
mode tcp
option ssl-hello-chk
balance source
server Manager1 192.168.0.60:443 check
server Manager2 192.168.0.61:443 check

listen  info 192.168.0.2:90
mode http
balance source
stats uri /



Thanks for your help!

Christophe




Le 08/09/11 14:16, « Baptiste » bed...@gmail.com a écrit :

Hello,

you server might be very slow or your server timeout in your conf
might be too low.

If you can copy/paste your conf and tell us which version you're using
and the underlying OS.

cheers


On Thu, Sep 8, 2011 at 1:35 PM, Christophe Rahier
christo...@qualifio.com wrote:
 Hi,
 I've a question about this error :

 504 Gateway Time-out

 The server didn't respond in time.

 What could I check in my config ? I created 2 LB with a virtual IP and
all
 request are coming from the firewall to this IP.
 I think it's possible, if needed, I can copy my configuration file.
 Thanks for your help, I'm lost.
 Regards, Christophe







(in)sanity check on hdr_cnt

2011-09-08 Thread Hank A. Paulson
does hdr_cnt not work or am I just completely unable to get an example that 
works? I can't imagine it doesn't work but I have tried _many_ - some examples 
and nothing seems to work (maybe it is 40+ hrs):


acl hdrcnttest  hdr_cnt gt 0
reqadd x-has-host:\ YES if hdrcnttest

acl hdrcnttest  hdr_cnt(host) gt 0
reqadd x-has-host:\ YES if hdrcnttest

acl hdrcnttest  hdr_cnt(Host) gt 0
reqadd x-has-host:\ YES if hdrcnttest

acl hdrcnttest  hdr_cnt(Host) 1
reqadd x-has-host:\ YES if hdrcnttest

reqadd x-has-host:\ YES if { hdr_cnt(Host) gt 0 }

reqadd x-has-host:\ YES if { hdr_cnt(Host:) gt 0 }

Nothing seems to work, I tried 1.4.15, 1.4.17 and I recompiled 1.4.17 without 
any options at all for make except linux26



Other acl criteria seem to work as normal, just hdr_cnt...

Thanks.



Deadwood: The Complete Series on DVD or Blu-ray

2011-09-08 Thread store-news

 
  
   


















Please click here if the e-mail below is not displayed correctly.




Follow us:













 
 
 
 
  
  

Free Two-Day Shipping with Amazon Prime


  

  
  
  Your Amazon.com
  Today's Deals
  See All Departments
  
  
  
  

 
 
 
 
 
 
 
 
 














 
  
   

 
  
   

 
  
   

 
  
  
 

   
  
 
 
  
   

 
  
   

 
  
   

 
  
  
 

   
   

 
  

   

 
  Deadwood: The Complete Series on DVD or Blu-ray

   

   
   
   
   
   

 
   
   





   

 This critically acclaimed HBO Western can be yours in its entirety: All 36 episodes on 19 discs. Choose from either the DVD or Blu-ray version.



   
   
   

   
   
   

  
 

   
  
 

   
  
 

   
  
 
 
  
   

 
  
  
 

   
  
 

   
  
 

   
  
 



   
  
  
   












 
 
 
 
  
  
  
  Explore Other Gold Box Deals
  
  
  
   
 
   
 




 
  
   

 
  
   

   
  
 

   
  
 

 
  
   

 
 

   
  
 

 
  
   

 
  
   

   
  
 

   
  
 

 
  
   

 
 

   
  
 

 
  
   

 
  
   

   
  
 

   
  
 

 
  
   

 
 

   
  
 

 
  
   

 
  
   

   
  
 

   
  
 




 
  
   

 
 

   
  
 
 
  
   

 
 

   
  
 
 
  
   

 
 

   
  
 
 
  
   

 
 

   
  
 
 
  
   

 
 

   
  
 
 
  
   

 
 

   
  
 
 
  
   

 
 

   
  
 






 
  
   

 
  
  
   
Buy A Game, Get One 50% Off
   
  






 


 
  
 

   
  
 

 
  
   

 

   
  
 

 
  
   

 
  
  
   
Panasonic All-in-One Laser Machine
   
  








 299.95

 225.95

  
  
   (25 off)
  
  







 


 
  
 

   
  
 

 
  
   

 

   
  
 

 
  
   

 
  
  
   
10k Yellow Gold Created Ruby and Diamond-Accent Flower Pendant, 18
   
  








 169.00

 49.99

  
  
   (70 off)
  
  







 


 
  
 

   
  
 

 
  
   

 

   
  
 

 
  
   

 
  
  
   
18k Gold Plated Sterling Silver Multi 0 Bracelet
   
  








 119.99

 34.99

  
  
   (71 off)
  
  







 


 
  
 

   
  
 









 
  
   

 
  
   

   
  
 

   
  
 

 
  
   

 
 

   
  
 

 
  
   

 
  
   

   
  
 

   
  
 

 
  
   

 
 

   
  
 

 
  
   

 
  
   

   
  
 

   
  
 

 
  
   

 
 

 

Re: Error 504

2011-09-08 Thread Baptiste
I can't see anything weird here.
are the backend status OK on the haproxy http stat page?

cheers

On Thu, Sep 8, 2011 at 2:28 PM, Christophe Rahier
christo...@qualifio.com wrote:
 Hi,

 Here's my config. Webservers are IIS.

 global
 log 192.168.0.2 local0
 log 127.0.0.1 local1 notice
 maxconn     10240
 defaults
 log    global
 option dontlognull
 retries    2
 clitimeout  5
 srvtimeout  5
 contimeout  5
 timeout server 60s

 listen WebPlayer-Farm 192.168.0.2:80
 mode http
 option httplog
 balance source
 #balance leastconn
 option forwardfor
 stats enable
 option http-server-close
 server Player1 192.168.0.10:80 check
 server Player2 192.168.0.11:80 check
 server Player3 192.168.0.12:80 check
 server Player4 192.168.0.13:80 check

 listen WebPlayer-Farm-SSL 192.168.0.2:443
 mode tcp
 option ssl-hello-chk
 balance source
 server Player1 192.168.0.10:443 check
 server Player2 192.168.0.11:443 check
 server Player3 192.168.0.12:443 check
 server Player4 192.168.0.13:443 check

 listen  Manager-Farm    192.168.0.2:81
 mode http
 option httplog
 balance source
 option forwardfor
 stats enable
 option http-server-close
 server  Manager1 192.168.0.60:80 check
 server  Manager2 192.168.0.61:80 check

 listen Manager-Farm-SSL 192.168.0.2:444
 mode tcp
 option ssl-hello-chk
 balance source
 server Manager1 192.168.0.60:443 check
 server Manager2 192.168.0.61:443 check

 listen  info 192.168.0.2:90
 mode http
 balance source
 stats uri /



 Thanks for your help!

 Christophe




 Le 08/09/11 14:16, « Baptiste » bed...@gmail.com a écrit :

Hello,

you server might be very slow or your server timeout in your conf
might be too low.

If you can copy/paste your conf and tell us which version you're using
and the underlying OS.

cheers


On Thu, Sep 8, 2011 at 1:35 PM, Christophe Rahier
christo...@qualifio.com wrote:
 Hi,
 I've a question about this error :

 504 Gateway Time-out

 The server didn't respond in time.

 What could I check in my config ? I created 2 LB with a virtual IP and
all
 request are coming from the firewall to this IP.
 I think it's possible, if needed, I can copy my configuration file.
 Thanks for your help, I'm lost.
 Regards, Christophe








Re: Error 504

2011-09-08 Thread Christophe Rahier
Yes ...

Is it possible to improve my config?



Le 08/09/11 15:50, « Baptiste » bed...@gmail.com a écrit :

I can't see anything weird here.
are the backend status OK on the haproxy http stat page?

cheers

On Thu, Sep 8, 2011 at 2:28 PM, Christophe Rahier
christo...@qualifio.com wrote:
 Hi,

 Here's my config. Webservers are IIS.

 global
 log 192.168.0.2 local0
 log 127.0.0.1 local1 notice
 maxconn 10240
 defaults
 logglobal
 option dontlognull
 retries2
 clitimeout  5
 srvtimeout  5
 contimeout  5
 timeout server 60s

 listen WebPlayer-Farm 192.168.0.2:80
 mode http
 option httplog
 balance source
 #balance leastconn
 option forwardfor
 stats enable
 option http-server-close
 server Player1 192.168.0.10:80 check
 server Player2 192.168.0.11:80 check
 server Player3 192.168.0.12:80 check
 server Player4 192.168.0.13:80 check

 listen WebPlayer-Farm-SSL 192.168.0.2:443
 mode tcp
 option ssl-hello-chk
 balance source
 server Player1 192.168.0.10:443 check
 server Player2 192.168.0.11:443 check
 server Player3 192.168.0.12:443 check
 server Player4 192.168.0.13:443 check

 listen  Manager-Farm192.168.0.2:81
 mode http
 option httplog
 balance source
 option forwardfor
 stats enable
 option http-server-close
 server  Manager1 192.168.0.60:80 check
 server  Manager2 192.168.0.61:80 check

 listen Manager-Farm-SSL 192.168.0.2:444
 mode tcp
 option ssl-hello-chk
 balance source
 server Manager1 192.168.0.60:443 check
 server Manager2 192.168.0.61:443 check

 listen  info 192.168.0.2:90
 mode http
 balance source
 stats uri /



 Thanks for your help!

 Christophe




 Le 08/09/11 14:16, « Baptiste » bed...@gmail.com a écrit :

Hello,

you server might be very slow or your server timeout in your conf
might be too low.

If you can copy/paste your conf and tell us which version you're using
and the underlying OS.

cheers


On Thu, Sep 8, 2011 at 1:35 PM, Christophe Rahier
christo...@qualifio.com wrote:
 Hi,
 I've a question about this error :

 504 Gateway Time-out

 The server didn't respond in time.

 What could I check in my config ? I created 2 LB with a virtual IP and
all
 request are coming from the firewall to this IP.
 I think it's possible, if needed, I can copy my configuration file.
 Thanks for your help, I'm lost.
 Regards, Christophe












Re: Increase log size in config.

2011-09-08 Thread Damien Hardy
Hi Graeme,

You are rigth so maybe it could be great that haproxy could log the full
entry by other way than syslog
Using the Unix socket allready existing could be tailed by any external
program I suppose, and shouldn't be limited by a 1024 string maybe.

Regards,

-- 
Damien

2011/9/8 Graeme Donaldson gra...@donaldson.za.net

 Hi Damien

 I may be wrong, but as far as I can gather the 1024-byte limitation is
 imposed by the syslog protocol RFC (http://www.faqs.org/rfcs/rfc3164.html)
 and not HAproxy itself.

 Regards,
 Graeme.

 On 8 September 2011 16:09, Damien Hardy damienhardy@gmail.com wrote:

 Hello there.

 We are working on our realtime statistics of consultation platform based
 on haproxy logs.
 Our probleme is that logs can capture HTTP headers but are limited to a
 1024 length string witch is very short if Host: Referer: and User-Agent: are
 captured as we are doing.

 Is it planned to set the max log size by config parameter.

 Do you now if there is any side effects increasing this value and rebuild
 a package ?
 The syslog server is a cloudera flume node located on the same server with
 haproxy (flume is not able to read in a Unix socket as far as I know).

 Thank you.

 Best regards,

 --
 Damien






Re: Increase log size in config.

2011-09-08 Thread Damien Hardy
By the way, haproxy could be configured with default max size at 1024 and if
we want to increase it by config it may be available (as long as we are not
using a real syslog server).

My concern is about network device with UDP frame bigger than 1024 if
anybody is aware of potential  probleme that could occure

2011/9/8 Graeme Donaldson gra...@donaldson.za.net

 Hi Damien

 I may be wrong, but as far as I can gather the 1024-byte limitation is
 imposed by the syslog protocol RFC (http://www.faqs.org/rfcs/rfc3164.html)
 and not HAproxy itself.

 Regards,
 Graeme.

 On 8 September 2011 16:09, Damien Hardy damienhardy@gmail.com wrote:

 Hello there.

 We are working on our realtime statistics of consultation platform based
 on haproxy logs.
 Our probleme is that logs can capture HTTP headers but are limited to a
 1024 length string witch is very short if Host: Referer: and User-Agent: are
 captured as we are doing.

 Is it planned to set the max log size by config parameter.

 Do you now if there is any side effects increasing this value and rebuild
 a package ?
 The syslog server is a cloudera flume node located on the same server with
 haproxy (flume is not able to read in a Unix socket as far as I know).

 Thank you.

 Best regards,

 --
 Damien






Re: New benchmark of HAProxy at 10 Gbps using Myricom's 10GbE NICs possible?

2011-09-08 Thread Aleksandar Lazic


On Don 08.09.2011 07:51, Willy Tarreau wrote:

Hi Aleks,

On Thu, Sep 08, 2011 at 12:52:20AM +0200, Aleksandar Lazic wrote:

Hi Willy,

I have take a look about the last test on

http://haproxy.1wt.eu/10g.html

and thought it would be nice to see a test with the brand new 1.5
version.

What do you think ;-)?


I agree in principle, but you know, running benchmarks takes a *lot* of
time. The main issue I'm having right now is that I don't have a 10Gb
switch, so I'm running with direct-attached machines, with only one
client, one proxy and one server. In such a setup, it's common to see
all 3 machines at 100% CPU during some tests, and it's hard to tell the
part caused by client, server or proxy. The worst case I had was to be
limited to about 15 Gbps full duplex and not being able to determine
which machine or NIC was causing this, as most PCI-e chipsets I've
encountered in the past were not able to reach the line rate.

I really need to build a lab with at least 5 machines (2 clients, 2
servers, one proxy and a switch). This is the only way to measure the
impact on each component.

Anyway, what I'm expecting from new 10G runs with 1.5 is to have the
same data rate over the wire (since we're doing nothing for this, we
just use splice() to ask the kernel to do the work for us), but see an
increase in the amount of peak connections per second. This is
especially important for systems dealing with DDoSes using
stick-tables, as they need to get rid of a connection as soon as
possible.


Well you are right.

I'a just a benchmark junky and haven't such a equipment her so I thought
you or someone on the list maybe have the possibility.

Well it was just for my curiosity.

BR
Aleks



Re: (in)sanity check on hdr_cnt

2011-09-08 Thread Hank A. Paulson
Whether I have the rules in the backend or the front does not seem to make a 
difference - I tried some rules in front and back and neither worked.


Maybe I am missing something obvious.
Thanks.

Example with config:

[haproxy]# wget -S -O - http://10.1.1.251:82/blank.gif
--2011-09-08 19:00:59--  http://10.1.1.251:82/blank.gif
Connecting to 10.1.1.251:82... connected.

HTTP request sent, awaiting response... T 10.1.1.251:12427 - 10.1.1.251:82 [AP]
  GET /blank.gif HTTP/1.0..User-Agent: Wget/1.12 (linux-gnu)..Accept: 
*/*..Host: 10.1.1.251:82..Connection: Keep-Alive 



  HTTP/1.0 200 OK
  Server: thttpd
  Content-Type: image/gif
  Date: Fri, 09 Sep 2011 02:00:59 GMT
  Last-Modified: Wed, 07 Sep 2011 17:17:06 GMT
  Accept-Ranges: bytes
  Content-Length: 43
  X-nohdrsub: 1  the only rsp hdr added is the negation of a hdr* acl
  Connection: keep-alive
Length: 43 [image/gif]
Saving to: “STDOUT”

2011-09-08 19:00:59 (8.57 MB/s) - written to stdout [43/43]

config file:

defaults
#option splice-auto
option tcp-smart-connect
option http-server-close
timeout queue 27s
timeout http-request 5s
timeout client 33s
timeout connect 8s
timeout server 33s
timeout http-keep-alive 77s
timeout tarpit 190s

global
node   hdr_cnt
description hdr_cnt
loglocalhost local1
#   loglocalhost local1 err
maxconn32768
uid99
gid99
chroot /var/empty
pidfile/var/run/haproxy.pid
stats socket /tmp/hap.sock
daemon
quiet
spread-checks 6

frontend hdr_cnt
bind 10.0.1.251:82
bind 10.0.1.252:82
bind 10.0.1.253:82
mode http
log  global
option   httplog
option http-server-close
option   log-separate-errors
maxconn  32768

capture request  header Host   len 32
capture request  header User-Agent len 256
capture request  header Content-Length len 10
capture request  header Refererlen 384
capture request  header Vialen 64
capture request  header Cookie len 128

capture response header Content-Length len 10

default_backend www

backend www
modehttp
balance roundrobin
server www1 127.0.0.1:81 maxconn 10
option http-server-close
acl hashosthdr_via_hdrcntge1 hdr_cnt(Host) ge 1
acl hashosthdr_via_hdrcntlt9 hdr_cnt(Host) lt 9
acl hashosthdr_via_hdrsub  hdr_sub(host) -i 10.1

acl hasuahdr_via_hdrcntge1 hdr_cnt(User-Agent) ge 1
acl hasuahdr_via_hdrcnt1 hdr_cnt(User-Agent) 1

rspadd X-gothdrcntge1:\ 1 if hashosthdr_via_hdrcntge1
rspadd X-gothdrcntlt9:\ 1 if hashosthdr_via_hdrcntlt9

rspadd X-gothdrsub:\ 1 if hashosthdr_via_hdrsub
rspadd X-nohdrsub:\ 1 if !hashosthdr_via_hdrsub

rspadd X-gotuahdrcntge1:\ 1 if hasuahdr_via_hdrcntge1
rspadd X-gotuahdrcnt1:\ 1 if hasuahdr_via_hdrcnt1



On 9/8/11 6:49 AM, Baptiste wrote:

hi,

where are you doing your ACLs?
Frontend or backend?

cheers

On Thu, Sep 8, 2011 at 3:06 PM, Hank A. Paulson
h...@spamproof.nospammail.net  wrote:

does hdr_cnt not work or am I just completely unable to get an example that
works? I can't imagine it doesn't work but I have tried _many_ - some
examples and nothing seems to work (maybe it is 40+ hrs):

acl hdrcnttest  hdr_cnt gt 0
reqadd x-has-host:\ YES if hdrcnttest

acl hdrcnttest  hdr_cnt(host) gt 0
reqadd x-has-host:\ YES if hdrcnttest

acl hdrcnttest  hdr_cnt(Host) gt 0
reqadd x-has-host:\ YES if hdrcnttest

acl hdrcnttest  hdr_cnt(Host) 1
reqadd x-has-host:\ YES if hdrcnttest

reqadd x-has-host:\ YES if { hdr_cnt(Host) gt 0 }

reqadd x-has-host:\ YES if { hdr_cnt(Host:) gt 0 }

Nothing seems to work, I tried 1.4.15, 1.4.17 and I recompiled 1.4.17
without any options at all for make except linux26


Other acl criteria seem to work as normal, just hdr_cnt...

Thanks.