Re: haproxy duplicate http_request_counter values (BUG)
On Tue, Aug 20, 2013 at 04:14:05PM -0400, Patrick Hemmer wrote:
I see 2 ways of handling this.
1) Move the code that populates the session unique_id member to
http_process_req_common (or to http_wait_for_request where it's
allocated). This will let requests terminated by an `errorfile`
directive log out a request ID.
2) Initialize the unique_id member upon allocation.
I've attached a patch which does option 2, but I'm not sure if option 1
would be preferable so that even `errorfile` requests will get a request ID.
-Patrick
Hello Patrick,
Thanks for reporting the bug, I implemented something more relevant, the
unique-id is now generated when a request failed.
--
William Lallemand
From 6c2adb543c54df657e37836fc484a7f4e97ef7e1 Mon Sep 17 00:00:00 2001
From: William Lallemand wlallem...@exceliance.fr
Date: Wed, 28 Aug 2013 15:44:19 +0200
Subject: [PATCH] BUG/MEDIUM: unique_id: junk in log on empty unique_id
When a request fail, the unique_id was allocated but not generated.
The string was not initialized and junk was printed in the log with %ID.
This patch changes the behavior of the unique_id. The unique_id is now
generated when a request failed.
This bug was reported by Patrick Hemmer.
---
src/log.c| 10 +-
src/proto_http.c | 9 +
2 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/src/log.c b/src/log.c
index 369dc34..f1fe40c 100644
--- a/src/log.c
+++ b/src/log.c
@@ -1488,8 +1488,10 @@ int build_logline(struct session *s, char *dst, size_t maxsize, struct list *lis
break;
case LOG_FMT_UNIQUEID: // %ID
+ret = NULL;
src = s-unique_id;
-ret = lf_text(tmplog, src, maxsize - (tmplog - dst), tmp);
+if (src)
+ ret = lf_text(tmplog, src, maxsize - (tmplog - dst), tmp);
if (ret == NULL)
goto out;
tmplog = ret;
@@ -1541,6 +1543,12 @@ void sess_log(struct session *s)
level = LOG_ERR;
}
+ /* if unique-id was not generated */
+ if (!s-unique_id !LIST_ISEMPTY(s-fe-format_unique_id)) {
+ if ((s-unique_id = pool_alloc2(pool2_uniqueid)) != NULL)
+ build_logline(s, s-unique_id, UNIQUEID_LEN, s-fe-format_unique_id);
+ }
+
tmplog = update_log_hdr();
size = tmplog - logline;
size += build_logline(s, tmplog, sizeof(logline) - size, s-fe-logformat);
diff --git a/src/proto_http.c b/src/proto_http.c
index 8d6eaf5..6ab2676 100644
--- a/src/proto_http.c
+++ b/src/proto_http.c
@@ -2635,9 +2635,6 @@ int http_wait_for_request(struct session *s, struct channel *req, int an_bit)
}
}
- if (!LIST_ISEMPTY(s-fe-format_unique_id))
- s-unique_id = pool_alloc2(pool2_uniqueid);
-
/* 4. We may have to convert HTTP/0.9 requests to HTTP/1.0 */
if (unlikely(msg-sl.rq.v_l == 0) !http_upgrade_v09_to_v10(txn))
goto return_bad_req;
@@ -3950,8 +3947,12 @@ int http_process_request(struct session *s, struct channel *req, int an_bit)
/* add unique-id if header-unique-id is specified */
- if (!LIST_ISEMPTY(s-fe-format_unique_id))
+ if (!LIST_ISEMPTY(s-fe-format_unique_id)) {
+ if ((s-unique_id = pool_alloc2(pool2_uniqueid)) == NULL)
+ goto return_bad_req;
+ s-unique_id[0] = '\0';
build_logline(s, s-unique_id, UNIQUEID_LEN, s-fe-format_unique_id);
+ }
if (s-fe-header_unique_id s-unique_id) {
chunk_printf(trash, %s: %s, s-fe-header_unique_id, s-unique_id);
--
1.8.1.5
Load Balance individual requests
Hi, We plan to use HAProxy to load-balance an MS Lync 2010 Infrastructure. We also plan to use it as reverse proxy? Is HAproxy able to load balance individual requests within a TCP session ? This a requirements for effcient load-balance. Is somebody already use HAProxy with Lync 2010 ? Thanks a lot --- Kevin C
RE: Load Balance individual requests
Hi Kevin, Is HAproxy able to load balance individual requests within a TCP session? If you're talking about HTTP, then yes, thats possibile. Individual requests may be served by different backends via content switching (if you want). It we talk about a proprietary application protocol based on TCP, its obvious all haproxy can do is load-balance the layer 4 payload as-is. Not sure if Lync uses HTTP. Regards, Lukas
Re: Load Balance individual requests
Hi Lukas, Yes, link use SIP and HTTPS. The documentation says tha HAproxy must load balance individual requests within a TCP session and make persistence with a cookie. Regards, Kevin C Le 28/08/2013 19:06, Lukas Tribus a écrit : Hi Kevin, Is HAproxy able to load balance individual requests within a TCP session? If you're talking about HTTP, then yes, thats possibile. Individual requests may be served by different backends via content switching (if you want). It we talk about a proprietary application protocol based on TCP, its obvious all haproxy can do is load-balance the layer 4 payload as-is. Not sure if Lync uses HTTP. Regards, Lukas
RE: Load Balance individual requests
Yes, link use SIP and HTTPS. HAProxy can't load balance UDP based SIP. The documentation says tha HAproxy must load balance individual requests within a TCP session and make persistence with a cookie. This implies however that HAProxy has to offload SSL. So the certificate needs to be installed on the HAProxy box. Lukas
RE: Load Balance individual requests
We are just using to load balance a Front-End pool in Lync 2010. Each local Lync branch servers have one failover pool assigned, which is the load balanced pool at datacenter (2 servers at datacenter). And these two front end servers is load-balanced using haproxy ( ports 80, 443, 8080, 4443 ). ~~~some info: The following settings should be configured on your hardware load balancer to properly load balance requests for Lync Web Services: •For internal Web Services virtual IPs (VIPs), set source_addr persistence (internal port 80, 443) on the hardware load balancer. For Lync Server 2010, source_addr persistence indicates that multiple connections coming from a single IP address are always sent to one server to maintain session state. •For external Web Services virtual IPs (VIPs), set cookie-based persistence on a per port basis for external ports 4443, 8080 on the hardware load balancer. For Lync Server 2010, cookie-based persistence indicates that multiple connections from a single client are always sent to one server to maintain session state. To configure cookie-based persistence, the load balancer must decrypt and re-encrypt SSL traffic. Therefore, any certificate assigned to the external web service FQDN must also be assigned the 4443 VIP of the hard load balancer. ◦Cookies must not be set to HTTP only. ◦Cookies must not be configured with an expiration time. ◦Cookies must be configured to filter on ‘MS WSMAN’. ◦Cookies must be set in every HTTP response for which the incoming HTTP request did not have a cookie, regardless of whether a previous HTTP response on that same TCP connection had already obtained a cookie. If the Load Balancer optimizes cookie insert to only occur once per TCP connection, that optimization MUST NOT be used. •If a reverse proxy is used, set the Forward host header to True in the reverse proxy publishing rule for port 4443. This will ensure that the original URL is forwarded to the target web server. ~~~ : http://blogs.technet.com/b/nexthop/archive/2011/11/03/hardware-load-balancer-requirements-for-lync-server-2010.aspx -Original Message- From: Lukas Tribus [mailto:luky...@hotmail.com] Sent: Wednesday, August 28, 2013 1:41 PM To: Kevin C Cc: haproxy@formilux.org Subject: RE: Load Balance individual requests Yes, link use SIP and HTTPS. HAProxy can't load balance UDP based SIP. The documentation says tha HAproxy must load balance individual requests within a TCP session and make persistence with a cookie. This implies however that HAProxy has to offload SSL. So the certificate needs to be installed on the HAProxy box. Lukas
Re: Load Balance individual requests
Hi did you know this one http://www.exceliance.fr/sites/default/files/biblio/appnotes_0061_lync_2010_deployment_guide_en.pdf cheers Thomas Heil Am 28.08.2013 um 21:29 schrieb Ozgur Tas o...@morrisonhershfield.com: We are just using to load balance a Front-End pool in Lync 2010. Each local Lync branch servers have one failover pool assigned, which is the load balanced pool at datacenter (2 servers at datacenter). And these two front end servers is load-balanced using haproxy ( ports 80, 443, 8080, 4443 ). ~~~some info: The following settings should be configured on your hardware load balancer to properly load balance requests for Lync Web Services: •For internal Web Services virtual IPs (VIPs), set source_addr persistence (internal port 80, 443) on the hardware load balancer. For Lync Server 2010, source_addr persistence indicates that multiple connections coming from a single IP address are always sent to one server to maintain session state. •For external Web Services virtual IPs (VIPs), set cookie-based persistence on a per port basis for external ports 4443, 8080 on the hardware load balancer. For Lync Server 2010, cookie-based persistence indicates that multiple connections from a single client are always sent to one server to maintain session state. To configure cookie-based persistence, the load balancer must decrypt and re-encrypt SSL traffic. Therefore, any certificate assigned to the external web service FQDN must also be assigned the 4443 VIP of the hard load balancer. ◦Cookies must not be set to HTTP only. ◦Cookies must not be configured with an expiration time. ◦Cookies must be configured to filter on ‘MS WSMAN’. ◦Cookies must be set in every HTTP response for which the incoming HTTP request did not have a cookie, regardless of whether a previous HTTP response on that same TCP connection had already obtained a cookie. If the Load Balancer optimizes cookie insert to only occur once per TCP connection, that optimization MUST NOT be used. •If a reverse proxy is used, set the Forward host header to True in the reverse proxy publishing rule for port 4443. This will ensure that the original URL is forwarded to the target web server. ~~~ : http://blogs.technet.com/b/nexthop/archive/2011/11/03/hardware-load-balancer-requirements-for-lync-server-2010.aspx -Original Message- From: Lukas Tribus [mailto:luky...@hotmail.com] Sent: Wednesday, August 28, 2013 1:41 PM To: Kevin C Cc: haproxy@formilux.org Subject: RE: Load Balance individual requests Yes, link use SIP and HTTPS. HAProxy can't load balance UDP based SIP. The documentation says tha HAproxy must load balance individual requests within a TCP session and make persistence with a cookie. This implies however that HAProxy has to offload SSL. So the certificate needs to be installed on the HAProxy box. Lukas
Re: 1.5-dev19 crashes
Hi Everyone, apologies for not replying sooner, I had my filtering a bit too overbearing and didn't notice the replies. In answer to your question, no actually I had never noticed any crashes of haproxy before build19. Unfortunately I just experienced a crash using the git checkout, it was the first since posting here. So, if it's useful at all, the debug log I collected using the previous command is available here: https://silverbucket.net/haproxy_debug3.log.gz I've just recompiled haproxy using the following command: # make DEBUG=-DDEBUG_FULL CFLAGS=-g -O0 TARGET=linux2628 CPU=native USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 Adjusted ulimit, but cannot set `suid_dumpable` # echo 2 /proc/sys/fs/suid_dumpable bash: echo: write error: Operation not permitted Nor can I set the core dump pattern: # echo /tmp/core-%e-%s-%u-%g-%p-%t /proc/sys/kernel/core_pattern bash: echo: write error: Operation not permitted (running as root) Anyway, I'm running the haproxy that's been compiled with the debug flags now... Cheers Nick On Wed, Aug 28, 2013 at 1:17 AM, amyas am...@yasmolive.com wrote: Willy Tarreau w at 1wt.eu writes: +-+ | | | HAPROXY MUST NEVER EVER CRASH | | | +-+ I know you have said that in the past, but I don't know if people are used to packages with that level of commitment. Might it be worth adding that to the README file in the how to report bugs section as you mention on the homepage? Also, is it close to time for a new dev release? There is more than a page or so of improvements since the last one. Thanks as always.
Re: 1.5-dev19 crashes
Nick Jennings nick@... writes: Nor can I set the core dump pattern:# echo /tmp/core-%e-%s-%u-%g-%p-%t /proc/sys/kernel/core_pattern bash: echo: write error: Operation not permitted (running as root) What distro/kernel? Did you check that you can see the value: cat /proc/sys/kernel/core_pattern abrt and systemd can hijack core saving systemd saves it in a journal and abrt in /var/cache/core or /var/spool/core There are other gotchas for cores depending on your env.
Re: 1.5-dev19 crashes
On Thu, Aug 29, 2013 at 2:41 AM, Amyas am...@yasmolive.com wrote: Nick Jennings nick@... writes: Nor can I set the core dump pattern:# echo /tmp/core-%e-%s-%u-%g-%p-%t /proc/sys/kernel/core_pattern bash: echo: write error: Operation not permitted (running as root) What distro/kernel? Did you check that you can see the value: cat /proc/sys/kernel/core_pattern I'm running CentOS 6 with a 2.6.18 kernel, aside from a few additional packages via the EPEL, there are no significant modifications. # uname -a Linux 2.6.18-308.8.2.el5.028stab101.1 #1 SMP Sun Jun 24 20:25:35 MSD 2012 x86_64 x86_64 x86_64 GNU/Linux And yes, both /proc/sys/kernel/core_pattern and /proc/sys/fs/suid_dumpable exist. abrt and systemd can hijack core saving systemd saves it in a journal and abrt in /var/cache/core or /var/spool/core There are other gotchas for cores depending on your env.
Re: 1.5-dev19 crashes
Nick Jennings nick@... writes: I'm running CentOS 6 with a 2.6.18 kernel, aside from a few additional packages via the EPEL, there are no significant modifications. # uname -aLinux 2.6.18-308.8.2.el5.028stab101.1 #1 SMP That might be one problem, you have TARGET=linux2628 but are using an older kernel than 2.6.28, it should be And on a classic Linux with SSL and ZLIB support (eg: Red Hat 5.x) : $ make TARGET=linux26 CPU=native USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 As far as the core values problem with /proc - could it be selinux ?
