Re: HAProxy and TIME_WAIT
On Wed, Nov 30, 2011 at 06:10:29PM +0200, Daniel Rankov wrote: > Hi, Thank you, these explonations are really helpfull. > Now may be because of a bug or something but "option nolinger" is not > working for backend. it works great for frontend. And I've tested putting > this option all over the config file... That's is what had confused me. Indeed you're right, I can reproduce this behaviour too. It happened when we introduced the systematic shutdown() before the close() to avoid resetting too many connections. Please apply the attached patch which fixes it. Thanks for your report, Willy >From b7e257fe61890e4edc839d76dc0223a8d5bdb0f2 Mon Sep 17 00:00:00 2001 From: Willy Tarreau Date: Wed, 30 Nov 2011 18:02:24 +0100 Subject: BUG: tcp: option nolinger does not work on backends Daniel Rankov reported that "option nolinger" is inefficient on backends. The reason is that it is set on the file descriptor only, which does not prevent haproxy from performing a clean shutdown() before closing. We must set the flag on the stream_interface instead if we want an RST to be emitted upon active close. --- src/proto_tcp.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/proto_tcp.c b/src/proto_tcp.c index 37d9054..5ccfb81 100644 --- a/src/proto_tcp.c +++ b/src/proto_tcp.c @@ -239,7 +239,7 @@ int tcpv4_connect_server(struct stream_interface *si, setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, (char *) &one, sizeof(one)); if (be->options & PR_O_TCP_NOLING) - setsockopt(fd, SOL_SOCKET, SO_LINGER, (struct linger *) &nolinger, sizeof(struct linger)); + si->flags |= SI_FL_NOLINGER; /* allow specific binding : * - server-specific at first -- 1.7.2.3
reqrep not working
Hi, IM trying to reqrep this URL http://www.domain.net/proxy.php?application=en.server&route=/friends/listFriends?userId=xxx to http://en.server.domain.net/friends/listHotFriends?userId=xxx this is the line Im using in haproxy.cfg reqrep ([^\ ]*)\ /proxy.php?application=(.*)[^&]route=(.*) http://\2.domain.net/\3 I'll appretiate a hand with this thx in advance Owen --- posted at http://www.serverphorums.com http://www.serverphorums.com/read.php?10,411976,411976#msg-411976
Re: HAProxy and TIME_WAIT
Hi, Thank you, these explonations are really helpfull. Now may be because of a bug or something but "option nolinger" is not working for backend. it works great for frontend. And I've tested putting this option all over the config file... That's is what had confused me. OS is centos6, HA-Proxy version 1.4.18. here is the simplified config file, I quess there is nothing wrong with it, but still: global maxconn 32000 daemon log 127.0.0.1 local1 info defaults log global option tcplog maxconn 32000 frontend https-in bind 192.168.2.38:443 default_backend servers-https option nolinger backend servers-https mode tcp balance source option redispatch option nolinger server jetty-1 127.0.0.1:8443 With this configuration haproxy closes the connection to client with RST, but with backend it does not. By putting "option nolinger" in defaults section it works the same way. Am I wrong or is it a bug ? Thank you 2011/11/30 Willy Tarreau > On Wed, Nov 30, 2011 at 03:56:14PM +0200, Daniel Rankov wrote: > > Ok, now I'm kind of stuck here. > > Let me share you my observations on my really simple evirionment: > > for client I use wget on server with ip 192.168.2.30 > > haproxy is set on server with ip 192.168.2.38 > > and haproxy and web serer comunicate on 127.0.0.1. haproxy is in tcpmode. > > this is the monitored tcpdump for connection client to haproxy /just the > > closing connection part/ : > > > > 14:56:40.448210 IP 192.168.2.30.55867 > 192.168.2.38.443: . ack 7983 win > > 204 > > 14:56:40.448849 IP 192.168.2.30.55867 > 192.168.2.38.443: F 618:618(0) > ack > > 7983 win 204 > > 14:56:40.449513 IP 192.168.2.38.443 > 192.168.2.30.55867: F 7983:7983(0) > > ack 619 win 62 > > 14:56:40.449656 IP 192.168.2.30.55867 > 192.168.2.38.443: . ack 7984 win > > 204 > > > > and this is tcpdump for 127.0.0.1 /just the closing part again/ : > > > > 14:56:40.447887 IP 127.0.0.1.59302 > 127.0.0.1.8443: . ack 7983 win 386 > > > > 14:56:40.448914 IP 127.0.0.1.59302 > 127.0.0.1.8443: F 618:618(0) ack > 7983 > > win 386 > > 14:56:40.449236 IP 127.0.0.1.8443 > 127.0.0.1.59302: F 7983:7983(0) ack > 619 > > win 273 > > 14:56:40.449272 IP 127.0.0.1.59302 > 127.0.0.1.8443: . ack 7984 win 386 > > > > > > So that showes me that the connections from haproxy to webserver are > closed > > with FIN/FIN-ACK/ACK. > > here is netstat -anpo | grep TIME: > > tcp0 0 127.0.0.1:59302 127.0.0.1:8443 > > TIME_WAIT - timewait (58.73/0/0) > > > > is that the expected bahaviour ? > > Yes, if you're in TCP mode (I though you were using HTTP mode), it's > perfectly > expected because in TCP mode there is no way to know if some important data > were sent and not received by the other side, so you cannot use an RST to > force > a close. > > Also, in TCP mode, haproxy just relays on the other side what it sees. So > as > you can see, wget closes the connection to haproxy, then haproxy does the > same > with the server. > > If you want to force an RST, you can use "option nolinger" in the backend. > But > then again, this is really not recommended since it can lead to incomplete > data > being received by the server. In the case of HTTPS, it should not be an > issue > due to the SSL closing handshake, but this is something to keep in mind. > > Regards, > Willy > >
Bâches imprimées, Rollup, stands: moins chers, plus vite
Vos bâches imprimées sur mesure à partir de 9,92 €/m² (HT) Recevez vos bâches en 2 à 5 jours ouvrables Impression HD Encres résistantes aux UV 4 Supports d'impression au choix 8 Finitions (œillets, renforts du bord, etc.) Upload immédiat de vos fichiers ou transfert par FTP Rollup 85 x 200 ECO44 eur HT 85 x 20049 eur HT 100 x 20069 eur HT 120 x 200 89 eur HT Délais : Départ usine: J+2 Livraison Express: J+3 Livraison Eco: J+5 Caractéristiques : Impression HD Résistant aux UV Imprimé et monté en Belgique Pop Up Stand parapluie Pop Up 3 x 3 (2,6 x 2,2m) 690 eur HT Pop Up 4 x 3 (3,3 x 2,2m) 790 eur HT Délais : Départ usine: J+4 Livraison Express: J+5 Livraison Eco: J+7 Caractéristiques : Impression HD Résistant aux UV Imprimé et monté en Belgique VEDI express © 2002-2012 Rollup, Pop Up, bâches, stand parapluie aux meilleurs prix et dans les meilleurs délais. Powered by VEDI Si vous souhaitez ne plus recevoir nos messages Fichiers acceptés Taille des images : Pour Pop Up 3x3m : - Image 337,5x222cm : fichier 337,5x222cm Pour Pop Up 4x3m : - Image 405x222cm : fichier 405x222cm Résolutions conseillées: PDF: 80 dpi avec typo vectorisée TIFF ou JPEG: 120 dpi Format des fichiers acceptés : PDF HD, EPS (avec polices et images incorporées et vectorisées), TIFF, JPEG Respect des couleurs : De préférence, fournir les fichiers en CMYK (noir quadri: C70 M70 Y70 K100). Les fichiers RGB seront convertis. Garanties de qualité Qualité d'impression : Impression HD : vos images sont toujours imprimées en haute définition. Couleurs vivantes et respectées : La gestion des couleurs et la garantie de rendu sont assuréespar notre équipe d'infographistes et d'imprimeurs professionnels expérimentés. Encres UV résistantes : Les encres utilisées pour l'impression de votre image sont toujours des encres résistantes aux rayons ultra violets. Ces encres de qualité garantissent une bonne longévité aux couleurs de votre visuel, pour une image toujours vive. Supports de qualité : Vos images sont imprimées sur support satiné, pour un meilleur rendu des couleurs et des contrastes. D'une épaisseur de 170 µm ou supérieure, nos supports d'impression vous garantissent une image qui ne gondole pas et offrent une protection contre les griffes et les poussières. Résistants aux déchirures et à l'eau, ils peuvent se nettoyer. Images anti-feu : Vos images peuvent être imprimées sur support ignifugé B2 / M1 (en option), idéal et même parfois obligatoire pour les foires et salons. Qualité : Structure solide : éléments construits en acier peint de section 20cm x 20 cm. Simple à monter : assemblage sans outil, montage en 30 minutes à deux personnes. Idéal pour la participation à vos salons grand public ou professionnels. La structure Wave permet de construire des stands d’une multitude de dimensions. Les images murales sont fournies roulées avec une barre de suspension intégré (Veclip). Elles se suspendent au montant horizontal supérieur du stand et se déroulent de haut en bas entre les piliers. Leur largeur est de maximum 2 mètres afin de faciliter le stockage et le transport. Il existe 4 largeurs (50, 100, 150 et 200 cm) et 2 hauteurs (220 cm pour les stands de 2m50 et 290cm pour les stand de 3m20). Mises dos à dos, elles permettent l’affichage recto-verso. Vous choisissez vous-même les différentes largeurs de sorte de combler complètement un côté du stand ou de laisser un passage. La largeur disponible pour l’image est égale à la largeur du stand moins 50 cm. Par exemple: si la largeur du côté du stand à fermer est de 5 mètres, la largeur disponible pour l’image est de 450cm. Pour combler cette largeur, vous pouvez commander 2 images de 200cm et 1 image de 50cm. Si la hauteur de votre stand est de 320 cm, vous choisirez une hauteur d’image de 290 cm. Production et livraison : Livraison express : votre stand livré, prêt à monter en 5 jours ouvrables. Conditions générales de ventes Généralités Seules les présentes conditions de vente sont d’application pour la vente. Ces conditions générales s'appliquent à l’exclusion de toutes autres conditions, même celles reprises sur des bons de commande émanant de l’acheteur. Le vendeur se réserve le droit de modifier à tout moment ces conditions générales. Néanmoins, les commandes en cours restent soumises aux conditions applicables au moment où la vente a été conclue. Les photos, illustrations, couleurs et textes associés aux produits repris sur ce site n’ont pas de caractère contractuel et de ce fait ne sauraient engager la responsabilité du vendeur. Toute commande donnera lieu à une facturation 30 jours après son acceptation dans le cas où la commande ne peut être exécutée en raison d'un retard imputable au client. En cas d'annulation de sa commande, le client sera redevable d'un montant correspondant aux frais déjà exposé, augmenté d'une indemnité de 15 % du montant commandé. La fa
Re: HAProxy and TIME_WAIT
On Wed, Nov 30, 2011 at 03:56:14PM +0200, Daniel Rankov wrote: > Ok, now I'm kind of stuck here. > Let me share you my observations on my really simple evirionment: > for client I use wget on server with ip 192.168.2.30 > haproxy is set on server with ip 192.168.2.38 > and haproxy and web serer comunicate on 127.0.0.1. haproxy is in tcpmode. > this is the monitored tcpdump for connection client to haproxy /just the > closing connection part/ : > > 14:56:40.448210 IP 192.168.2.30.55867 > 192.168.2.38.443: . ack 7983 win > 204 > 14:56:40.448849 IP 192.168.2.30.55867 > 192.168.2.38.443: F 618:618(0) ack > 7983 win 204 > 14:56:40.449513 IP 192.168.2.38.443 > 192.168.2.30.55867: F 7983:7983(0) > ack 619 win 62 > 14:56:40.449656 IP 192.168.2.30.55867 > 192.168.2.38.443: . ack 7984 win > 204 > > and this is tcpdump for 127.0.0.1 /just the closing part again/ : > > 14:56:40.447887 IP 127.0.0.1.59302 > 127.0.0.1.8443: . ack 7983 win 386 > > 14:56:40.448914 IP 127.0.0.1.59302 > 127.0.0.1.8443: F 618:618(0) ack 7983 > win 386 > 14:56:40.449236 IP 127.0.0.1.8443 > 127.0.0.1.59302: F 7983:7983(0) ack 619 > win 273 > 14:56:40.449272 IP 127.0.0.1.59302 > 127.0.0.1.8443: . ack 7984 win 386 > > > So that showes me that the connections from haproxy to webserver are closed > with FIN/FIN-ACK/ACK. > here is netstat -anpo | grep TIME: > tcp0 0 127.0.0.1:59302 127.0.0.1:8443 > TIME_WAIT - timewait (58.73/0/0) > > is that the expected bahaviour ? Yes, if you're in TCP mode (I though you were using HTTP mode), it's perfectly expected because in TCP mode there is no way to know if some important data were sent and not received by the other side, so you cannot use an RST to force a close. Also, in TCP mode, haproxy just relays on the other side what it sees. So as you can see, wget closes the connection to haproxy, then haproxy does the same with the server. If you want to force an RST, you can use "option nolinger" in the backend. But then again, this is really not recommended since it can lead to incomplete data being received by the server. In the case of HTTPS, it should not be an issue due to the SSL closing handshake, but this is something to keep in mind. Regards, Willy
Re: HAProxy and TIME_WAIT
Ok, now I'm kind of stuck here.
Let me share you my observations on my really simple evirionment:
for client I use wget on server with ip 192.168.2.30
haproxy is set on server with ip 192.168.2.38
and haproxy and web serer comunicate on 127.0.0.1. haproxy is in tcpmode.
this is the monitored tcpdump for connection client to haproxy /just the
closing connection part/ :
14:56:40.448210 IP 192.168.2.30.55867 > 192.168.2.38.443: . ack 7983 win
204
14:56:40.448849 IP 192.168.2.30.55867 > 192.168.2.38.443: F 618:618(0) ack
7983 win 204
14:56:40.449513 IP 192.168.2.38.443 > 192.168.2.30.55867: F 7983:7983(0)
ack 619 win 62
14:56:40.449656 IP 192.168.2.30.55867 > 192.168.2.38.443: . ack 7984 win
204
and this is tcpdump for 127.0.0.1 /just the closing part again/ :
14:56:40.447887 IP 127.0.0.1.59302 > 127.0.0.1.8443: . ack 7983 win 386
14:56:40.448914 IP 127.0.0.1.59302 > 127.0.0.1.8443: F 618:618(0) ack 7983
win 386
14:56:40.449236 IP 127.0.0.1.8443 > 127.0.0.1.59302: F 7983:7983(0) ack 619
win 273
14:56:40.449272 IP 127.0.0.1.59302 > 127.0.0.1.8443: . ack 7984 win 386
So that showes me that the connections from haproxy to webserver are closed
with FIN/FIN-ACK/ACK.
here is netstat -anpo | grep TIME:
tcp0 0 127.0.0.1:59302 127.0.0.1:8443
TIME_WAIT - timewait (58.73/0/0)
is that the expected bahaviour ?
All the best !
2011/11/29 Willy Tarreau
> Hi Daniel,
>
> On Tue, Nov 29, 2011 at 06:10:46PM +0200, Daniel Rankov wrote:
> > For sure TIME_WAIT connections are not an issue when thay keep
> information
> > about sockets to clients, but when TIME_WAIT connections keep sockets
> bussy
> > for your host where HAProxy is deployed to the backend the limit can be
> > reached - it's defined by ip_local_port_range.
> > Here is what I mean:
> > Client -(1)-> HAProxy -(2)-> Webserver
> > / it doesn't metter if the web server and haproxy are on the same
> server./
> > I) client connects to haproxy
> > socket is tooken - clientIP:random_port:haproxy_ip:haproxy_port
> >
> > II) haproxy connects to webserver
> > socket is tooken haproxy_local_ip:random_port:backend_ip:backend_port
> >
> > III) client closes a conneciton to haproxy (1) in normal way -
> > FIN/FIN-ACK/ACK
> > this way we have one connections that goes from ESTABLISHED to TIME_WAIT
> > state. we don't really care about this TIME_WAIT connection beacause the
> > socket that is tooken is between a client and haproxy
> > - clientIP:random_port:haproxy_ip:haproxy_port
> >
> > IV) haproxy closes the connection to backend (2) with FIN/FIN-ACK/ACK
> > Now this ESTABLISHED connection goes to TIME_WAIT state. and the socket
> > that is tooken is between the haporxy and backend server.
>
> I agree on this point and this is why it does not happen :-)
>
> Haproxy uses an RST to close the connection to the backend server precisely
> because of this, otherwise it would not work at all. You can strace it, you
> will notice that it does a setsockopt(SO_LINGER, {0}) before closing. In
> fact, you cannot even configure it not to do this because it would cause
> too
> much harm.
>
> (...)
> > I believe that the common architecture is that backend servers are
> > phisically close to haproxy and are on high speed networks where no
> packet
> > lost is expected. So we don't really need TIME_WAIT state here. It's not
> > needed on localhost for sure.
>
> When haproxy closes the connection to a server, we never need the TIME_WAIT
> anyway, because if it closes, it means it has nothing left to say to the
> server and is not interested in getting its response. So even if some data
> were lost, it would not be an issue.
>
> For instance, one situation where you can observe this close is when you
> enable forceclose or http-server-close. You'll see that as soon as the
> last byte of payload is received, haproxy sends an RST to the server to
> release the connection so that another pending request may immediately
> reuse it.
>
> Even if the RST was lost, a packet from the server would reach the haproxy
> machine and match no known connection, causing an RST in return.
>
> That said, I completely agree with all your analysis and that's what has
> caused me a lot of gray hair when implementing the client-side keep-alive,
> precisely because I needed a way to make haproxy close the server
> connection
> without being affected by the TIME_WAIT on this side.
>
> Best regards,
> Willy
>
>
Re: HTTP Log format unconstant
Hello Cyril,
Hmm, I read the sources again searching on clf format and seeing the
timezone is forced to "+" in this particular format bring me some
doubts ...
I suppose that "get_gmtime(s->logs.accept_date.tv_sec, &tm);" (GMT ?) is
the same every where on the planet :)
So no need to get the Timezone anymore :)
Second problem solved today ... You rocks !
Thank you,
--
Damien
2011/11/30 Cyril Bonté
> Hi Damien,
>
> Le mercredi 30 novembre 2011 10:50:04, Damien Hardy a écrit :
> > Hello,
> >
> > An other point that could be enhanced in the log format is the date
> format:
> > we miss the timezone in IT
>
> Did you give a try to "option httplog clf" ?
>
> >
> > Best regards,
> >
> > > Hello haproxy-list,
> > >
> > > We are working on near real time web statistics architecture based on
> the
> > > haproxy access logs.
> > > (a first shoot of a flume plugin is available here :
> > > https://github.com/figarocms/flume-haproxy-extractor it is now
> designed
> > > to feet or needs especially but we are working on making it available
> for
> > > more general pupose)
> > >
> > > But HTTP logs are quite difficult to parse especially if there is
> captured
> > > headers:
> > > * Request and response headers could be captured enclosed both by {},
> but
> > > if you capture only one of them in a frontend there is no way to now
> if it
> > > is the request or the response header. because if the capture is not
> set
> > > the field is not set either in teh log.
> > > My suggestion is maybe to add a letter before "{" to precise if it's
> query
> > > ( ex : "Q{" ) or response ( "R{" ) set of captured headers. or always
> make
> > > them appear.
> > >
> > > * Olso the header captured is not precised in it, only values appear
> so
> > > you have olso to know the order of capture in the configuration to
> parse it
> > > well.
> > > Maybe the whole line of the header could be set in the field.
> > >
> > > * The headers are separated by a "|" but the eventuelly "|" allready
> > > present in the headers are not escaped, that could occure some
> probleme of
> > > parsing I suppose.
> > > No really answer to address this probleme but maybe enclosing headers
> by "
> > > (and escaped this char it like many other String method could do)
> could be
> > > a good envolvment.
> > >
> > > And As I precised it before syslog default size of 1024 is very short
> for
> > > today statistical need. I had to increase the constant to 4096
> (capture of
> > > full UA, and Referer are costly).
> > >
> > > Best regards,
> > >
> > > --
> > > Damien
>
> --
> Cyril Bonté
>
Re: HTTP Log format unconstant
[...] > >> * The headers are separated by a "|" but the eventuelly "|" allready >> present in the headers are not escaped, that could occure some probleme of >> parsing I suppose. >> No really answer to address this probleme but maybe enclosing headers by >> " (and escaped this char it like many other String method could do) could >> be a good envolvment. >> > My mistake ! I read the source with more attention and see that special characters are escaped for the capture header fields in "void init_proto_http()" This problem is [solved] \o/ that was fast ! Best regards, -- Damien
Re: Re: hashing + roundrobin algorithm
Hi,
Ricardo, from Tuenti, posted a very nice presentation on slideshare:
http://www.slideshare.net/ricbartm/load-balancing-at-tuenti
They explain how they did configure HAProxy to do what you're trying
to achieve ;)
cheers
2011/11/30 wsq003 :
>
> My modification is based on version 1.4.16.
>
> ===in struct server add
> following===
> char vgroup_name[100];
> struct proxy *vgroup; //if not NULL, means this is a Virtual GROUP
>
> ===in function process_chk() add
> at line 1198===
> if (s->vgroup) {
> if ((s->vgroup->lbprm.tot_weight > 0) && !(s->state & SRV_RUNNING)) {
> s->health = s->rise;
> set_server_check_status(s, HCHK_STATUS_L4OK, "vgroup ok");
> set_server_up(s);
> } else if (!(s->vgroup->lbprm.tot_weight > 0) && (s->state & SRV_RUNNING)) {
> s->health = s->rise;
> set_server_check_status(s, HCHK_STATUS_HANA, "vgroup has no available server");
> set_server_down(s);
> }
>
> if (s->state & SRV_RUNNING) {
> s->health = s->rise + s->fall - 1;
> set_server_check_status(s, HCHK_STATUS_L4OK, "vgroup ok");
> }
>
> while (tick_is_expired(t->expire, now_ms))
> t->expire = tick_add(t->expire, MS_TO_TICKS(s->inter));
> return t;
> }
>
> ===in function assign_server()
> add at line 622===
> if (s->srv->vgroup) {
> struct proxy *old = s->be;
> s->be = s->srv->vgroup;
> int ret = assign_server(s);
> s->be = old;
> return ret;
> }
>
> ===in function
> cfg_parse_listen() add at line
> 3949===
> else if (!defsrv && !strcmp(args[cur_arg], "vgroup")) {
> if (!args[cur_arg + 1]) {
> Alert("parsing [%s:%d] : '%s' : missing virtual_group name.\n",
> file, linenum, newsrv->id);
> err_code |= ERR_ALERT | ERR_FATAL;
> goto out;
> }
> if (newsrv->addr.sin_addr.s_addr) {
> //for easy indicate
> Alert("parsing [%s:%d] : '%s' : virtual_group requires the server address as 0.0.0.0\n",
> file, linenum, newsrv->id);
> err_code |= ERR_ALERT | ERR_FATAL;
> goto out;
> }
> newsrv->check_port = 1;
> strlcpy2(newsrv->vgroup_name, args[cur_arg + 1], sizeof(newsrv->vgroup_name));
> cur_arg += 2;
> }
>
> ===in function
> check_config_validdity() add at line
> 5680
> /*
> * set vgroup if necessary
> */
> newsrv = curproxy->srv;
> while (newsrv != NULL) {
> if (newsrv->vgroup_name[0] != '\0') {
> struct proxy *px = findproxy(newsrv->vgroup_name, PR_CAP_BE);
> if (px == NULL) {
> Alert("[%s][%s] : vgroup '%s' not exist.\n", curproxy->id, newsrv->id, newsrv->vgroup_name);
> err_code |= ERR_ALERT | ERR_FATAL;
> break;
> }
> newsrv->vgroup = px;
> }
> newsrv = newsrv->next;
> }
>
> ==
>
> and some minor changes in function stats_dump_proxy() that not important.
>
> ==
>
> sample config file looks like:
>
> backend internallighttpd
> option httpchk /monitor/ok.htm
> server wsqa 0.0.0.0 vgroup subproxy1 weight 32 check inter 4000 rise 3 fall 3
> server wsqb 0.0.0.0 vgroup subproxy2 weight 32 check inter 4000 rise 3 fall 3
> balance uri
> hash-type consistent
> option redispatch
> retries 3
>
> backend subproxy1
> option httpchk /monitor/ok.htm
> server wsq01 1.1.1.1:8001 weight 32 check inter 4000 rise 3 fall 3
> server wsq02 1.1.1.2:8001 weight 32 check inter 4000 rise 3 fall 3
> balance roundrobin
> option redispatch
> retries 3
>
> backend subproxy2
> option httpchk /monitor/ok.htm
> server wsq03 1.1.1.1:8002 weight 32 check inter 4000 rise 3 fall 3
> server wsq04 1.1.1.2:8002 weight 32 check inter 4000 rise 3 fall 3
> balance roundrobin
> option redispatch
> retries 3
>
> ==
>
> Sorry I can't provide a clean patch, because vgroup is just one of several
> changes.
> I did not consider the rewrite rules at that time. Maybe we can add
> a function call before calling assigen_server()?
>
>
> From: Willy Tarreau
> Date: 2011-11-30 01:47
> To: wsq003
> CC: Rerngvit Yanggratoke; haproxy; Baptiste
> Subject: Re: Re: hashing + roundrobin algorithm
> On Tue, Nov 29, 2011 at 02:56:49PM +0800, wsq003 wrote:
>>
>> Backend proxies may be multiple layers, then every layer can have its own LB param.
>> Logically this is a tree-like structure, every real server is a leaf. Every none-leaf node is a backend proxy and may have LB param.
>
> I clearly understand what it looks like from the outside. It's still not very
> clear how you *concretely* implemented it. Maybe you basically did what I've
> bee
Re: HTTP Log format unconstant
Hi Damien,
Le mercredi 30 novembre 2011 10:50:04, Damien Hardy a écrit :
> Hello,
>
> An other point that could be enhanced in the log format is the date format:
> we miss the timezone in IT
Did you give a try to "option httplog clf" ?
>
> Best regards,
>
> > Hello haproxy-list,
> >
> > We are working on near real time web statistics architecture based on the
> > haproxy access logs.
> > (a first shoot of a flume plugin is available here :
> > https://github.com/figarocms/flume-haproxy-extractor it is now designed
> > to feet or needs especially but we are working on making it available for
> > more general pupose)
> >
> > But HTTP logs are quite difficult to parse especially if there is captured
> > headers:
> > * Request and response headers could be captured enclosed both by {}, but
> > if you capture only one of them in a frontend there is no way to now if it
> > is the request or the response header. because if the capture is not set
> > the field is not set either in teh log.
> > My suggestion is maybe to add a letter before "{" to precise if it's query
> > ( ex : "Q{" ) or response ( "R{" ) set of captured headers. or always make
> > them appear.
> >
> > * Olso the header captured is not precised in it, only values appear so
> > you have olso to know the order of capture in the configuration to parse it
> > well.
> > Maybe the whole line of the header could be set in the field.
> >
> > * The headers are separated by a "|" but the eventuelly "|" allready
> > present in the headers are not escaped, that could occure some probleme of
> > parsing I suppose.
> > No really answer to address this probleme but maybe enclosing headers by "
> > (and escaped this char it like many other String method could do) could be
> > a good envolvment.
> >
> > And As I precised it before syslog default size of 1024 is very short for
> > today statistical need. I had to increase the constant to 4096 (capture of
> > full UA, and Referer are costly).
> >
> > Best regards,
> >
> > --
> > Damien
--
Cyril Bonté
Re: HTTP Log format unconstant
Hello,
An other point that could be enhanced in the log format is the date format:
we miss the timezone in IT
Best regards,
--
Damien
2011/11/29 Damien Hardy
> Hello haproxy-list,
>
> We are working on near real time web statistics architecture based on the
> haproxy access logs.
> (a first shoot of a flume plugin is available here :
> https://github.com/figarocms/flume-haproxy-extractor it is now designed
> to feet or needs especially but we are working on making it available for
> more general pupose)
>
> But HTTP logs are quite difficult to parse especially if there is captured
> headers:
> * Request and response headers could be captured enclosed both by {}, but
> if you capture only one of them in a frontend there is no way to now if it
> is the request or the response header. because if the capture is not set
> the field is not set either in teh log.
> My suggestion is maybe to add a letter before "{" to precise if it's query
> ( ex : "Q{" ) or response ( "R{" ) set of captured headers. or always make
> them appear.
>
> * Olso the header captured is not precised in it, only values appear so
> you have olso to know the order of capture in the configuration to parse it
> well.
> Maybe the whole line of the header could be set in the field.
>
> * The headers are separated by a "|" but the eventuelly "|" allready
> present in the headers are not escaped, that could occure some probleme of
> parsing I suppose.
> No really answer to address this probleme but maybe enclosing headers by "
> (and escaped this char it like many other String method could do) could be
> a good envolvment.
>
> And As I precised it before syslog default size of 1024 is very short for
> today statistical need. I had to increase the constant to 4096 (capture of
> full UA, and Referer are costly).
>
> Best regards,
>
> --
> Damien
>
>
>
>
>
>
>
>
>
>
