Re: [PATCH v2] *_dom matching header functions now also split on :
On Fri, Sep 09, 2011 at 09:03:15PM +0200, Willy Tarreau wrote: Hi Finn, Yes, thanks for the update, I noticed this too and fixed it during tests. I'm OK with this method. I have just replaced the macros with inline functions (which I verified produced the same code) because the result is more readable. And I removed the typedef, as I consider that typedefs for scalars just add obfuscation, but I know it's a matter of taste. I'm about to merge the attached patch, unless you raise your hand in a very short time ;-) Thanks, it's much nicer like this! The typedef was only there to support the two different macro versions since they needed different types in the delimiter. - Finn Arne
Re: [PATCH v2] *_dom matching header functions now also split on :
On Sat, Sep 10, 2011 at 09:59:46AM +0200, Finn Arne Gangstad wrote: On Fri, Sep 09, 2011 at 09:03:15PM +0200, Willy Tarreau wrote: Hi Finn, Yes, thanks for the update, I noticed this too and fixed it during tests. I'm OK with this method. I have just replaced the macros with inline functions (which I verified produced the same code) because the result is more readable. And I removed the typedef, as I consider that typedefs for scalars just add obfuscation, but I know it's a matter of taste. I'm about to merge the attached patch, unless you raise your hand in a very short time ;-) Thanks, it's much nicer like this! The typedef was only there to support the two different macro versions since they needed different types in the delimiter. Perfect. I've merged it. Thanks for the quick feedback, Willy
How to block empty User-Agent strings?
Hi, I have been using haproxy for my lab servers for a while and I like it. But i have seen in my web server logs (iis) that I get empty User-Agent strings on most attempts to access files that not exists. Some examples: #Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken 2011-09-09 00:42:30 W3SVC143517500 WEB2 192.168.20.31 GET /muieblackcat - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 136 18031 2011-09-09 00:42:34 W3SVC143517500 WEB2 192.168.20.31 GET /phpMyAdmin-2.6.4-pl3/libraries/dbg/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 169 21437 2011-09-09 00:42:34 W3SVC143517500 WEB2 192.168.20.31 GET /old/padmin/libraries/dbg/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 159 20500 2011-09-09 00:42:34 W3SVC143517500 WEB2 192.168.20.31 GET /xampp/phpmyadmin/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 160 18140 2011-09-09 00:42:40 W3SVC143517500 WEB2 192.168.20.31 GET /php-my-admin/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 156 19046 2011-09-09 00:42:40 W3SVC143517500 WEB2 192.168.20.31 GET /typo3/phpmyadmin/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 160 18031 2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET /admin/pma/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 153 21796 2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET /admin/phpmyadmin/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 160 20843 2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET /sql/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 146 19953 2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET /php/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 146 19015 2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET /PHPMYADMIN/+/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 158 18187 How do i block empty User-Agent strings i haproxy.cfg? I have tried with the example in the documentation for haproxy: # ignore user-agents reporting any flavour of Mozilla or MSIE, but # block all others. reqipass ^User-Agent:\.*(Mozilla|MSIE) reqitarpit ^User-Agent: Best Regards Mattias Edman Menco AB Mobil: 073 561 90 26 Tel: 08-551 178 10 E-post: mattias.ed...@menco.se Web: http://www.menco.se/ Adr: Dackevägen 33, 177 34 Järfälla Org nr: 556700-0053
Re: How to block empty User-Agent strings?
Hi Matthias,
On Sat, Sep 10, 2011 at 10:19:28PM +0200, mattias.ed...@menco.se wrote:
Hi,
I have been using haproxy for my lab servers for a while and I like it.
But i have seen in my web server logs (iis) that I get empty User-Agent
strings on most attempts to access files that not exists.
Some examples:
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem
cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie)
cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes
time-taken
2011-09-09 00:42:30 W3SVC143517500 WEB2 192.168.20.31 GET /muieblackcat - 80
- 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0 1236 0 136 18031
2011-09-09 00:42:34 W3SVC143517500 WEB2 192.168.20.31 GET
/phpMyAdmin-2.6.4-pl3/libraries/dbg/setup.php - 80 - 219.94.198.229 HTTP/1.1
- - - www.ifj.se 404 0 1236 0 169 21437
2011-09-09 00:42:34 W3SVC143517500 WEB2 192.168.20.31 GET
/old/padmin/libraries/dbg/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - -
www.ifj.se 404 0 1236 0 159 20500
2011-09-09 00:42:34 W3SVC143517500 WEB2 192.168.20.31 GET
/xampp/phpmyadmin/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - -
www.ifj.se 404 0 1236 0 160 18140
2011-09-09 00:42:40 W3SVC143517500 WEB2 192.168.20.31 GET
/php-my-admin/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - -
www.ifj.se 404 0 1236 0 156 19046
2011-09-09 00:42:40 W3SVC143517500 WEB2 192.168.20.31 GET
/typo3/phpmyadmin/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - -
www.ifj.se 404 0 1236 0 160 18031
2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET
/admin/pma/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se
404 0 1236 0 153 21796
2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET
/admin/phpmyadmin/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - -
www.ifj.se 404 0 1236 0 160 20843
2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET
/sql/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0
1236 0 146 19953
2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET
/php/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - - www.ifj.se 404 0
1236 0 146 19015
2011-09-09 00:42:44 W3SVC143517500 WEB2 192.168.20.31 GET
/PHPMYADMIN/+/scripts/setup.php - 80 - 219.94.198.229 HTTP/1.1 - - -
www.ifj.se 404 0 1236 0 158 18187
How do i block empty User-Agent strings i haproxy.cfg?
I have tried with the example in the documentation for haproxy:
# ignore user-agents reporting any flavour of Mozilla or MSIE, but
# block all others.
reqipass ^User-Agent:\.*(Mozilla|MSIE)
reqitarpit ^User-Agent:
I think it does not work because in fact you don't have an empty User-Agent
but you have no user-agent at all.
Please try this instead :
reqtarpit ^ if { hdr_cnt(user-agent) eq 0 }
Which means tarpit anything if the number of user-agent headers is null.
Regards,
Willy
Re: Increase log size in config.
On Thu, Sep 08, 2011 at 04:53:18PM +0200, Damien Hardy wrote: By the way, haproxy could be configured with default max size at 1024 and if we want to increase it by config it may be available (as long as we are not using a real syslog server). My concern is about network device with UDP frame bigger than 1024 if anybody is aware of potential probleme that could occure I think syslog-ng can read much larger datagrams. You have quite a low risk of loss when sending to localhost over UDP. The loss rate over Unix is much higher due to the default tiny socket buffers for these sockets. I remember someone said several months ago that he changed the default size to more than 1024 and used it with success. Regards, Willy
Re: Error 504
Hi Christophe, On Thu, Sep 08, 2011 at 06:54:29AM -0700, Christophe Rahier wrote: Yes ... Is it possible to improve my config? Well, your config already allows a server to take as much as 50 seconds to respond, and still some requests don't respond within this delay. This is really huge and probably nobody will wait that long anyway. It's absolutely required to fix the application. You could make use of halog to report the slowest URLs : halog -ua log-file | less The format is a bit raw but you'll get the URLs sorted by average response time. Quite often when people encounter 504, it only happens on a very small set of URLs, sometimes even only one. Most often this is due to poorly written SQL queries which take ages to complete, but any number of causes is possible of course. Regards, Willy
Re: Can HAProxy behave like NginX with option proxy_next_upstream?
Hi Ernesto, On Thu, Sep 08, 2011 at 10:32:05AM -0400, Ernesto Rodriguez Reina wrote: Hi Everyone! We move from nginx to haproxy for load balancing and all have been great! All we do with nginx (as balancer) we have been able to do it using haproxy, except for reprocessing a request on http error 5* from upstream servers as we did with nginx using option proxy_next_upstream. Is there any way to archive with haproxy the same that with nginx proxy_next_upstream [1]? No, once the request has been sent, it's not in haproxy anymore and cannot be retried. The only retry we can do consists in trying to connect to another server if a connection fails. Also, you should keep in mind that a very small set of requests are allowed to be retransmitted (they're called idempotent requests in HTTP terminology) and quite commonly the ones you'd be interested in are not allowed and doing so would be dangerous. Regards, Willy
[ANNOUNCE] haproxy 1.5-dev7
Hi all, Five months have elapsed since 1.5-dev6. A massive amount of changes was merged since then. Most of them were cleanups and optimizations. A number of changes were dedicated to making listeners more autonomous. The immediate effect is a more robust handling of resource saturation, and the second effect is the removal of the 10-years old maintain_proxies() function which was harming performance and hard to get over. Halog was improved too (faster with more filters). A significant number of external contributions were merged, among them the stats socket updates to clear session-table keys by values. There are too many changes to list, but nothing too dangerous, so I'd say it's the 1.5-dev version I trust the most today. I'm planning on putting all the focus on server-side keep-alive again. Some of the remaining issues have been overcome. Surely there are still a number, but we can't know if we don't try :-) Do not hesitate to give 1.5-dev7 a try. I'm currently updating all 1.5 I have to it. site index : http://haproxy.1wt.eu/ sources : http://haproxy.1wt.eu/download/1.5/src/devel/ changelog : http://haproxy.1wt.eu/download/1.5/src/CHANGELOG Cheers, Willy
Re: scope of acls
Hi Hank, On Thu, Sep 08, 2011 at 03:26:39AM -0700, Hank A. Paulson wrote: I was wondering if acls that I create in the frontend should be available in backends, too? I was getting errors when I tried but the error disappeared when I either moved the reqadd/rspadd to the frontend or if I used a predefined acl like LOCALHOST. No, an ACL is only visible in the section where it is declared. Regards, Willy
Re: [ANNOUNCE] haproxy 1.5-dev7
I forgot to add something : I have updated the README file to request a change in the format of the subject in patches. To put it short, I'd like that we avoid the square brackets for the parts we want to keep in the commit message. The reason is that Git either removes all words enclosed within brackets, or keeps them all. When a patch series is submitted, the [PATCH] prefix is added and I have to remove it by hand for each patch. Also, right now it's not easy to indicate the criticity of a bug, so it as time to suggest a change. Anyway it's not critical if you forget, it's just something I'd appreciate. All the details are in the README. I'll try not to forget to do it myself ;-) Thanks, Willy
