Working example of url32+src
Hey Guys,
Im trying to tarpit based on Unique IP and specific URL. I started with the
following:
listen web
...
# Track IP over 60sec, if http_req rate greater than 20 AND
page.html, send
# to new backend with tarpit only.
stick-table type ip size 1m expire 60s store gpc0,http_req_rate(60s)
tcp-request connection track-sc1 src
tcp-request connection reject if { src_get_gpc0 gt 0 }
acl ratelimiteIP src_http_req_rate ge 20
acl showPage path_end page.html
use_backend web-ratelimit if ratelimiteIP showPage
backend web-ratelimit
mode http
fullconn 500
timeout tarpit 5s
reqitarpit .
The above example works to a degree, but not what I was hoping for. I am
only sending to a new backend to easily see results in the stats web page.
The above example tracks all IP requests, and if the url matches page.html
it blocks it (Example: 100x req to index.html and 1 req to page.html would
trigger) . I am hoping to track ONLY ip addresses going to a specific URL,
not all in general.
I then moved onto the following example:
listen web
...
acl showPage path_end page.html
acl ratelimitIP sc1_get_gpc0 ge 0
stick-table type binary len 20 size 500 store gpc0
tcp-request content track-sc1 url32+src if showPage
use_backend web-ratelimit if ratelimitIP
backend web-ratelimit
mode http
fullconn 500
timeout tarpit 5s
reqitarpit .
But this doesnt seem to track them correctly.
Anyone have any pointers or a working config on url32+src? would be greatly
appreciated?
Thanks
Andrew
Re: Working example of url32+src
On Wed, Jun 18, 2014 at 8:09 AM, Andrew Kroenert and...@thek.ro wrote:
Hey Guys,
Im trying to tarpit based on Unique IP and specific URL. I started with the
following:
listen web
...
# Track IP over 60sec, if http_req rate greater than 20 AND
page.html, send
# to new backend with tarpit only.
stick-table type ip size 1m expire 60s store gpc0,http_req_rate(60s)
tcp-request connection track-sc1 src
tcp-request connection reject if { src_get_gpc0 gt 0 }
acl ratelimiteIP src_http_req_rate ge 20
acl showPage path_end page.html
use_backend web-ratelimit if ratelimiteIP showPage
backend web-ratelimit
mode http
fullconn 500
timeout tarpit 5s
reqitarpit .
The above example works to a degree, but not what I was hoping for. I am
only sending to a new backend to easily see results in the stats web page.
The above example tracks all IP requests, and if the url matches page.html
it blocks it (Example: 100x req to index.html and 1 req to page.html would
trigger) . I am hoping to track ONLY ip addresses going to a specific URL,
not all in general.
I then moved onto the following example:
listen web
...
acl showPage path_end page.html
acl ratelimitIP sc1_get_gpc0 ge 0
stick-table type binary len 20 size 500 store gpc0
tcp-request content track-sc1 url32+src if showPage
use_backend web-ratelimit if ratelimitIP
backend web-ratelimit
mode http
fullconn 500
timeout tarpit 5s
reqitarpit .
But this doesnt seem to track them correctly.
Anyone have any pointers or a working config on url32+src? would be greatly
appreciated?
Thanks
Andrew
Hi Andrew,
You pick up your example from the blog post related to brute force protection.
In such case you just want to protect a particular URL from being
hitting too much.
URL:
http://blog.haproxy.com/2013/04/26/wordpress-cms-brute-force-protection-with-haproxy/
Now, if you explain us your needs, we may be able to help you.
Baptiste
how to reproduce the performance test on http://haproxy.1wt.eu/10g.html
Hi, All We use haproxy as the load balance. When the HTTP request reach 20K requests per second, in statistics report, we observed too many 4xx request in frontend, nearly about 25% In page http://haproxy.1wt.eu/10g.html, the max http requests per second is 40K. Where can I find the script that called request generator? I want to use this script to test the haproxy run on our server, to see if we have met a beckneck. B.R. Jie Jin
RE: how to reproduce the performance test on http://haproxy.1wt.eu/10g.html
We use haproxy as the load balance. When the HTTP request reach 20K requests per second, in statistics report, we observed too many 4xx request in frontend, nearly about 25% In page http://haproxy.1wt.eu/10g.html, the max http requests per second is 40K. Where can I find the script that called request generator? I want to use this script to test the haproxy run on our server, to see if we have met a beckneck. Client: http://1wt.eu/tools/inject/ Server: http://1wt.eu/tools/httpterm/ Regards, Lukas
Re: how to reproduce the performance test on http://haproxy.1wt.eu/10g.html
On Wed, Jun 18, 2014 at 11:33 AM, Lukas Tribus luky...@hotmail.com wrote: We use haproxy as the load balance. When the HTTP request reach 20K requests per second, in statistics report, we observed too many 4xx request in frontend, nearly about 25% In page http://haproxy.1wt.eu/10g.html, the max http requests per second is 40K. Where can I find the script that called request generator? I want to use this script to test the haproxy run on our server, to see if we have met a beckneck. Client: http://1wt.eu/tools/inject/ Server: http://1wt.eu/tools/httpterm/ Regards, Lukas Jie, This article on Willy's website is outdated, we can do much more now Baptiste
Re: how to reproduce the performance test on http://haproxy.1wt.eu/10g.html
Thanks Lukas : I am looking at the tool. Hope I can understand how to run it (: Baptiste . which version of haproxy you used in the performance test? and how many you did in your test? We use haproxy 1.4. We got about 25% 4xx HTTP status in frontent. We are trying to find out why. It seems that the client did not send their requests in time, so a 4xx http status was returned to client. Do you have any idea? 谢谢 金杰 (Jie Jin) On Wed, Jun 18, 2014 at 5:36 PM, Baptiste bed...@gmail.com wrote: On Wed, Jun 18, 2014 at 11:33 AM, Lukas Tribus luky...@hotmail.com wrote: We use haproxy as the load balance. When the HTTP request reach 20K requests per second, in statistics report, we observed too many 4xx request in frontend, nearly about 25% In page http://haproxy.1wt.eu/10g.html, the max http requests per second is 40K. Where can I find the script that called request generator? I want to use this script to test the haproxy run on our server, to see if we have met a beckneck. Client: http://1wt.eu/tools/inject/ Server: http://1wt.eu/tools/httpterm/ Regards, Lukas Jie, This article on Willy's website is outdated, we can do much more now Baptiste
HAProxy 1.5 release?
Hey Willy! I'm involved in a group that is building a highly-scalable open source virtual appliance-based load balancer for use with cloud operating systems like OpenStack. We are planning on making haproxy the core component of the solution we're building. At my company we've actually been using haproxy 1.5 for a couple years now in production to great effect, and absolutely love it. But I'm having trouble getting the rest of the members of my team to go along with the idea of using 1.5 in our solution simply because of its official status as a development branch. There are just so many useful new features in 1.5 that I'd really rather not have to go back to 1.4 in our solution... So! My question is: What can we do to help y'all bring the 1.5 branch far enough along such that y'all are comfortable releasing it as the official stable branch of haproxy? (Note we do have people in our group with connections in some of the major linux distros who can help to fast-track its adoption into official releases of said distros.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807
Re: HAProxy 1.5 release?
Haproxy 1.6 is very close to release. See http://marc.info/?l=haproxym=140129354705695 and http://marc.info/?l=haproxym=140085816115800 -Patrick *From: *Stephen Balukoff sbaluk...@bluebox.net *Sent: * 2014-06-18 08:40:55 EDT *To: *haproxy@formilux.org *Subject: *HAProxy 1.5 release? Hey Willy! I'm involved in a group that is building a highly-scalable open source virtual appliance-based load balancer for use with cloud operating systems like OpenStack. We are planning on making haproxy the core component of the solution we're building. At my company we've actually been using haproxy 1.5 for a couple years now in production to great effect, and absolutely love it. But I'm having trouble getting the rest of the members of my team to go along with the idea of using 1.5 in our solution simply because of its official status as a development branch. There are just so many useful new features in 1.5 that I'd really rather not have to go back to 1.4 in our solution... So! My question is: What can we do to help y'all bring the 1.5 branch far enough along such that y'all are comfortable releasing it as the official stable branch of haproxy? (Note we do have people in our group with connections in some of the major linux distros who can help to fast-track its adoption into official releases of said distros.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807
Re: HAProxy 1.5 release?
Err, pardon the typo, 1.5 :-) -Patrick *From: *Patrick Hemmer hapr...@stormcloud9.net *Sent: * 2014-06-18 08:49:27 EDT *To: *Stephen Balukoff sbaluk...@bluebox.net, haproxy@formilux.org *Subject: *Re: HAProxy 1.5 release? Haproxy 1.6 is very close to release. See http://marc.info/?l=haproxym=140129354705695 and http://marc.info/?l=haproxym=140085816115800 -Patrick *From: *Stephen Balukoff sbaluk...@bluebox.net *Sent: * 2014-06-18 08:40:55 EDT *To: *haproxy@formilux.org *Subject: *HAProxy 1.5 release? Hey Willy! I'm involved in a group that is building a highly-scalable open source virtual appliance-based load balancer for use with cloud operating systems like OpenStack. We are planning on making haproxy the core component of the solution we're building. At my company we've actually been using haproxy 1.5 for a couple years now in production to great effect, and absolutely love it. But I'm having trouble getting the rest of the members of my team to go along with the idea of using 1.5 in our solution simply because of its official status as a development branch. There are just so many useful new features in 1.5 that I'd really rather not have to go back to 1.4 in our solution... So! My question is: What can we do to help y'all bring the 1.5 branch far enough along such that y'all are comfortable releasing it as the official stable branch of haproxy? (Note we do have people in our group with connections in some of the major linux distros who can help to fast-track its adoption into official releases of said distros.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807
VPrivé : CATMANDOO PRECEPT POWER BILT Golf - Promo : PANAGICA, Casques LIMAR
Offres exclusives sur les produits du site Allsportshop.fr Version en ligne| AjouterAllsportshop à votre carnet d'adresses VENTES PRIVÉES GOURDE SIGG TEXTILE CYCLE HIGH TECH FITNESS OUTDOOR GLISSE URBAINE VENTES PRIVÉES CATMANDOO: ALLSPORTSHOP.fr vous propose la ligne de vêtements golf de la marque CATMANDOO. PRECEPT : Jusqu'à -45% pour les balles de golf spéciales Femme PRECEPT Lady IQ. POWER BILT : Les sacs de golf POWER BILT pour droitiers sont disponibles en Série complète avec 10 clubs ou en Demi-série avec 6 clubs. KOTHAI : Les sacs NBA et aux couleurs d'évènements sportifs sont à -50%. Jusqu'au Mardi 24 Juin. 12 modèles disponibles CATMANDOO FEMME Fournisseur officiel des équipes nationales de Finlande JO et Golf Shorts, Pantalons, Robes Polos et Polos Jusqu'à -60% ACCÉDER À LA VENTE 7 modèles disponibles CATMANDOO HOMME Fournisseur officiel des équipes nationales de Finlande JO et Golf Bermudas, Pantalons et Polos Jusqu'à -60% ACCÉDER À LA VENTE Balles de Golf Lady IQ PRECEPT 2 coloris disponibles : Rose et blanc Jusqu'à -45% ACCÉDER À LA VENTE Série et Demi-série POWER BILT Série Complète / Demi-Série droitier - Homme et Femme Jusqu'à -32% ACCÉDER À LA VENTE 24 modèles disponibles KOTHAI Sacs bandoulières, sacoches, pochettes d'ordinateurs portables et Portefeuilles -50% ACCÉDER À LA VENTE PROMO PANAGICA : Les vêtements de golf PANAGICA sont en promotion à -25% sur ALLSPORTSHOP.fr. Découvrez la collection Hommes et Femmes. LIMAR: Les casques de vélos LIMAR sont disponibles en promotion sur ALLSPORTSHOP.fr jusqu'à -42% Vêtements de golf PANAGICA 25 modèles disponibles Homme et Femme.Polos Jupes Pulls Pantalons -25% VOIR LEPRODUIT Casques de vélos LIMAR Plusieurs modèles disponibles dans différents coloris Jusqu'à -42% VOIR LEPRODUIT NOUVEAU ORBITA : ALLSPORTSHOP.fr met à votre dispositions la gamme complète du spécialiste Portugais du cycle. Venez découvrir les VTT, Vélos de course, Fixies, Vélos Électriques, VTC, Tandems, Vélos de Cross ainsi que les vélos pour enfants. OREGON: Un coach sportif qui vous accompagne partout, tout le temps et qui calcule vos cycles de sommeil ? Il existe et il se nomme OREGON. Gamme Cycle ORBITA VTT, Courses, Fixie, Électrique, VTC, Tandem, Cross et Kids. À partir de 110,00€ VOIR LEPRODUIT Bracelet d'activités électronique OREGON Le Coach sportif personnel à votre poignet 79,00€ VOIR LEPRODUIT ENTREPRISE FRANÇAISE SATISFAIT OU REMBOURSÉ PAIEMENT 100%SÉCURISÉ PAIEMENT PAYPAL PAIEMENT 3DSECURE ALLSPORTSHOP SUR FACEBOOK Consulter la version en ligne Pour être certain de bien recevoir nos messages, ajoutez Allsportshop à votre carnet d'adresses. Se désinscrire de cette newsletter
Re: HAProxy 1.5 release?
On Wed, Jun 18, 2014 at 08:52:01AM -0400, Patrick Hemmer wrote: Err, pardon the typo, 1.5 :-) This typo is a proof that we're close :-) I merged today what I think might be the last commit. I'm working on a human-readable changelog right now (ie: something that will also enlight people who are not power users about the changes). I realized that the web site is significantly outdated, I'll have to update it a little bit so that newcomers are not too much confused (ie: remove old stuff and performance reports). Willy
effect of adding `cookie` option to server
Hi! I am trying to figure out what the effect of adding the `cookie` option to a `server` config line is. According to this chunk of documentation: https://cbonte.github.io/haproxy-dconv/configuration-1.4.html#5-cookie This value will be checked in incoming requests, and the first operational server possessing the same value will be selected. However, doesn't this require me enabling a cookie load balancing algorithm? What I mean is, if I don't explicitly set any load balancing algorithm and the default (roundrobin) is chosen, it seems as though setting the cookie actually as no effect. In fact, setting the cookie should have no effect unless I specify `appsession`, `cookie`, or `balance uri` (or one of the other persistent `balance` algorithms) in a backend. Is that correct? Another way to phrase the question: is it true that the two listen stanzas below actually behave identically in every respect? The only change is the addition of the `cookie` param to each server. listen helloworld bind :80 mode http option httplog server srv1 10.0.2.15:9494 check inter 1s rise 1 fall 1 server srv2 10.0.2.15:9495 check inter 1s rise 1 fall 1 AND listen helloworld bind :80 mode http option httplog server srv1 10.0.2.15:9494 check inter 1s rise 1 fall 1 cookie srv1 server srv2 10.0.2.15:9495 check inter 1s rise 1 fall 1 cookie srv2 thanks! --igor
Re: haproxy real sever status
On Mon, Jun 16, 2014 at 04:47:02PM +0300, Muhammad Ammad Shah wrote: HI, i have two web servers, could you please help me to find a way to know the status of real servers are up or down. Please do not hijack existing threads when asking an unrelated question, not only it makes your question harder to find, but it further confuses people who try to help in the other thread. Instead of replying to a random existing mail, simply send a new one. Thanks, Willy
Re: use_backend
On Wed, May 28, 2014 at 02:15:47PM +0200, Steven Van Ingelgem wrote: How many entries can I add in 1 acl? Because I splitted 1 of the acls up in 14 lines, with each line about 40 items. I think I did it so a human could still read the configuration file, but does that matter for HAProxy? The default limit is about 1024 chars per line, which is far more what a normal humain can maintain, so your principle of keeping it maintainable is the best one. There is no limit on the number of lines nor total terms however. Regards, Willy
Re: Lot of 503 errors - cannot explain why...
On Sat, May 31, 2014 at 09:22:12PM +0200, Jakov Sosic wrote: On 05/27/2014 08:36 PM, Willy Tarreau wrote: I don't see why GoogleBot would see them since they should only affect the offending clients. Is it by any chance possible that my ISP is somehow screwing up connections? Because I see this kind of aborts/503s even from regular clients fetching regular stuff? Could be possible, but that sounds really strange. You could easily check though, if you own a machine somewhere outside your ISP's network. Simply send a request from there to your site and sniff at both ends. You'll see if the trace matches or not. It could be possible that the ISP is running a misconfigured transparent proxy which systematically closes the request path after sending the request (as haproxy used to do with option forceclose in early version 1.1 12 years ago). Or maybe it's part of an IDS or anti-ddos mechanism that's automatically enabled when they run into trouble. I've talked to ISP technicians and what they told me is that company has bandwith cap at XYZ Mbits, and once that limit is reached additional packets are simply dropped. So, packets dropping at peaks seems promising as explanation of some of the behaviour we have observed... Wow, I didn't know there were still people doing that ugly thing. In general they're doing this with cheap switches with very short queues instead of routers with deep queues, and the marvellous news here is that such switches start to consider the limit reached during very short bursts of a few packets sometimes, resulting in the absolute impossibility to reach even near the capped bandwidth without already dropping a lot of packets. The traffic shaping we noted in the roadmap file a few years ago was aimed at exactly this problem. By having haproxy pace the traffic before it becomes packets, it will be possible to constantly stay below the threshold of such crappy equipments. But that was postponed for 1.6, maybe even later. Thanks for letting us know! Willy
Re: Rewrite domain.com to other domain.com/dir/subdir
Hi Matt,
[ please avoid top-posting, it's very unconvenient to reply with the context,
rearranging your mail now ]
On Wed, May 28, 2014 at 11:40:55PM +0200, Matt . wrote:
2014-05-28 23:28 GMT+02:00 Bryan Talbot bryan.tal...@playnext.com:
The example in the reqirep section of the documentation seems to mostly do
what you're asking.
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#reqirep
Does that not work?
This will rewrite foo.com/baz.jpg - newdomain.com/com/foo/baz.jpg
reqirep ^Host:\ foo.com Host:\ newdomain.com
reqirep ^GET\ /(.*) GET\ /com/foo/\1
Yes I cam up to that part, but about the search in the map, do I need
to do this twice ?
You could. In fact there's a trick you could use now which consists in
using the base sample as a key for two maps. base contains the
concatenation of the host name and the URI. For example :
GET /foo/bar HTTP/1.1
Host: blah.example.org
Then base will equal blah.example.org/foo/bar. I think you see
where I'm going now. You can simply build two maps, one for the
uri part and one for the Host part, both indexed on the base, and
retrieve each part from there.
Hmmm thinking a bit more about it, I'm seeing an issue here. I don't
see how I could replace part of a URI using sample fetch expressions :-(
Well, if you don't have too many URIs to replace, you could at least apply
them based on a condition (eg: if you only need to insert a few prefixes,
you could simply condition a few reqirep rule on the output of the URI map).
For example it could be more or less like this :
reqirep ^GET\ /(.*) GET\ /foo/\1 if { base,map(prefix.map) foo }
reqirep ^GET\ /(.*) GET\ /bar/\1 if { base,map(prefix.map) bar }
reqirep ^GET\ /(.*) GET\ /com/\1 if { base,map(prefix.map) com }
http-request set-header Host %[base,map(host.map)]
There will be a few tricks that might be needed (basically, the base for
setting the Host header will change due to the former rewrite), but before
going into details, I'd need to know if you think this is something that
could be acceptable.
Regards,
Willy
Re: Stick tables
Hi Michal, On Mon, May 26, 2014 at 05:00:31PM +0200, Lazy wrote: Is it normal that http table exp counter gets updated even if src address is whitelisted sc1_get_gpc0(backend) 0 ? (...) I'm seeing a few things that confuse me in your config : tcp-request content track-sc1 src if !whitelist acl whitelist sc1_get_gpc0(backend) gt 0 So basically you enable tracking on sc1 if sc1_get_gpc0() == 0. But sc1_get_gpc0() does not exist yet (since sc1 not yet tracked) thus the ACL will always return not found and the track-sc1 rule will not be applied. That could explain your observation, because below then whitelist will not match and the or will be evaluated : acl flag_ok sc1_inc_gpc0(backend) ge 0 acl rm_black src_clr_gpc0(http) ge 0 acl whitelist sc1_get_gpc0(backend) gt 0 acl cauth_ok cookie_auth http-request allow if whitelist or cauth_ok flag_ok rm_black Is it possible to force expire an entry using an acl ? No but you have sc*_clr_* which might possibly do what you need. Is it possible to define multiple stick tables storing gpc0 in a single frontend, No you need at least one proxy (frontend or backend) per stick-table, there's a 1:1 relation. However, I'd suggest that you create backends for this, because a frontend requires at least a bind line while a backend does not require anything, and is perfectly fine with only a stick-table. So simply create 3 backends if you want 3 values stored in parallel for the same key. now it's using one from the frontend and one from the backend but how to define sc2 without adding another backend ? A backend is free, don't bother with this. Simply create it this way for instance : backend table-1 stick-table type ip size 1m expire 10m store gpc0 And that's all. Hoping this helps, Willy
Re: effect of adding `cookie` option to server
Hi Igor, On Wed, Jun 18, 2014 at 01:10:25PM -0700, Igor Serebryany wrote: Hi! I am trying to figure out what the effect of adding the `cookie` option to a `server` config line is. According to this chunk of documentation: https://cbonte.github.io/haproxy-dconv/configuration-1.4.html#5-cookie This value will be checked in incoming requests, and the first operational server possessing the same value will be selected. However, doesn't this require me enabling a cookie load balancing algorithm? A cookie persistence method, not load balancing. Cookie provides the exception to load balancing : if a cookie is found, then no load balancing happens and the correct server is picked instead. But you're right, you need to set a cookie directive. I'm used to use this : cookie SRV insert indirect nocache What I mean is, if I don't explicitly set any load balancing algorithm and the default (roundrobin) is chosen, it seems as though setting the cookie actually as no effect. That's true. In the past we used to reject such a configuration, but recently we relaxed the check because there was no other way to keep the per-server cookie values when temporarily disabling cookie persistence for debugging or whatever. So now such a configuration is perfectly valid but the cookie is never looked up. In fact, setting the cookie should have no effect unless I specify `appsession`, `cookie`, or `balance uri` (or one of the other persistent `balance` algorithms) in a backend. Is that correct? No, it's unrelated to the balancing algorithms nor to appsessions, it's only used by the cookie directive. Hoping this helps, Willy
Re: oscp and others
Hi Thomas, On Wed, Jun 18, 2014 at 12:20:11AM +0200, Thomas Heil wrote: Hi, I think dnssec and DANE could be an alternate path for checking certificates even for HAPRoxy. Of course dnssec is a burden but after that it can be very useful. How do you think about it? Not checked yet. However, we just merged the joint work of Dirkjan and Emeric who completed a really nice thing together. We can now load OCSP responses from files, send them to clients and update them from the CLI. Next step will probably be to have an HTTP client to automatically perform the refreshes, though it will only be usable for people where the load balancer has access to remote sites. Cheers, Willy
Nous prenons en charge pour vous 1000 euros de bon d'achat
Title: Gagnez 1000 de bon d'achat Cliquez ici pour lire cet e-mail dans votre navigateur. Bonjour,Vous avez t slectionn pour tenter de gagner 1000 de bon d'achat.Identifiez-vous ici pour voir les dtails: participez Si vous ne voulez plus recevoir d’email de notre part, dsabonnez vous ici
Re: HAProxy 1.5 release?
Hi y'all! Thanks for the responses and pointers. Willy-- if there's anything we can to do help with this, please let us know! You have no idea how much we're looking forward to this release! Thanks, Stephen On Wed, Jun 18, 2014 at 1:09 PM, Willy Tarreau w...@1wt.eu wrote: On Wed, Jun 18, 2014 at 08:52:01AM -0400, Patrick Hemmer wrote: Err, pardon the typo, 1.5 :-) This typo is a proof that we're close :-) I merged today what I think might be the last commit. I'm working on a human-readable changelog right now (ie: something that will also enlight people who are not power users about the changes). I realized that the web site is significantly outdated, I'll have to update it a little bit so that newcomers are not too much confused (ie: remove old stuff and performance reports). Willy -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807
