The 503's you show are from clients disconnecting very shortly after they've sent the request but before haproxy can connect to a backend server. The client closed the TCP connection without waiting for a response. The client probably didn't actually receive a 503 but haproxy seems to log it that way because it would have sent a 503 in that case if it could.
The 504's occur after the server-timeout of 30 seconds had passed without the backend returning a response. 30 seconds is a long time for clients to wait for most web resources, so I'd claim that your application is broken if it's taking that long to respond. Since there are a low number of connections to the backends, I hope that they're not overloaded. -Bryan On Thu, Dec 29, 2011 at 10:42 AM, Benoit GEORGELIN (web4all) < benoit.george...@web4all.fr> wrote: > Here another extract from log on specific website, as you can see, all > images are in 503 error.. > > "GET /inbreeding.php?nid=280240&nidp=&nidm=&nbgen=12 HTTP/1.1" > Dec 29 19:40:34 127.0.0.1 haproxy[24349]: > 193.247.39.42:58371[29/Dec/2011:19:40:34.355] public webserver/http-16 > 43/0/-1/-1/43 503 288 - > - CCVN 458/458/44/7/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/menu_gfx.png HTTP/1.1" > Dec 29 19:40:34 127.0.0.1 haproxy[24349]: > 193.247.39.42:58381[29/Dec/2011:19:40:34.477] public webserver/http-16 > 64/0/-1/-1/65 503 288 - > - CCVN 463/463/55/9/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/banniere/Bannierenoel2010.gif HTTP/1.1" > Dec 29 19:40:34 127.0.0.1 haproxy[24349]: > 193.247.39.42:58387[29/Dec/2011:19:40:34.815] public webserver/http-16 > 172/0/-1/-1/172 503 288 > - - CCVN 452/452/36/8/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/bbc/bbc_bg.gif HTTP/1.1" > Dec 29 19:40:35 127.0.0.1 haproxy[24349]: > 193.247.39.42:58390[29/Dec/2011:19:40:35.057] public webserver/http-16 > 111/0/-1/-1/111 503 288 > - - CCVN 450/450/38/8/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:35 127.0.0.1 haproxy[24349]: > 193.247.39.42:58391[29/Dec/2011:19:40:35.221] public webserver/http-16 > 144/0/-1/-1/146 503 288 > - - CCVN 444/444/38/9/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:35 127.0.0.1 haproxy[24349]: > 193.247.39.42:58392[29/Dec/2011:19:40:35.266] public webserver/http-16 > 101/0/-1/-1/102 503 288 > - - CCVN 443/443/37/9/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:35 127.0.0.1 haproxy[24349]: > 193.247.39.42:58394[29/Dec/2011:19:40:35.429] public webserver/http-16 > 122/0/-1/-1/122 503 288 > - - CCVN 441/441/35/7/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/banniere/Bannierenoel2010.gif HTTP/1.1" > Dec 29 19:40:35 127.0.0.1 haproxy[24349]: > 193.247.39.42:58395[29/Dec/2011:19:40:35.596] public webserver/http-16 > 63/0/-1/-1/63 503 288 - > - CCVN 444/444/34/7/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:35 127.0.0.1 haproxy[24349]: > 193.247.39.42:58396[29/Dec/2011:19:40:35.597] public webserver/http-16 > 72/0/-1/-1/72 503 288 - > - CCVN 442/442/34/7/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/img/wobody.jpg HTTP/1.1" > Dec 29 19:40:35 127.0.0.1 haproxy[24349]: > 193.247.39.42:58397[29/Dec/2011:19:40:35.717] public webserver/http-16 > 131/0/-1/-1/131 503 288 > - - CCVN 442/442/37/8/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:35 127.0.0.1 haproxy[24349]: > 193.247.39.42:58399[29/Dec/2011:19:40:35.737] public webserver/http-16 > 112/0/-1/-1/112 503 288 > - - CCVN 441/441/37/8/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:36 127.0.0.1 haproxy[24349]: > 193.247.39.42:58401[29/Dec/2011:19:40:35.900] public webserver/http-16 > 144/0/-1/-1/145 503 288 > - - CCVN 447/447/36/9/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:36 127.0.0.1 haproxy[24349]: > 193.247.39.42:58402[29/Dec/2011:19:40:35.950] public webserver/http-16 > 102/0/-1/-1/102 503 288 > - - CCVN 445/445/39/10/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:36 127.0.0.1 haproxy[24349]: > 193.247.39.42:58405[29/Dec/2011:19:40:36.296] public webserver/http-16 > 238/0/-1/-1/238 503 288 > - - CCVN 452/452/42/8/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > Dec 29 19:40:36 127.0.0.1 haproxy[24349]: > 193.247.39.42:58408[29/Dec/2011:19:40:36.621] public webserver/http-16 > 88/0/-1/-1/88 503 288 - > - CCVN 445/445/42/8/0 0/0 {www.golf5forum.fr} "GET > /Themes/maclike_20g/images/theme/main_block.png HTTP/1.1" > > > ------------------------------ > *De: *"Benoit GEORGELIN (web4all)" <benoit.george...@web4all.fr> > *À: *haproxy@formilux.org > *Envoyé: *Jeudi 29 Décembre 2011 13:19:04 > *Objet: *How to explain 503 and 504 error. > > > Hi All . > > I would like to understand why i have a lot of 503 and 504 error from > haproxy. > > > *Regarding the documentation :* > > 503 when no server was available to handle the request, or in response to > monitoring requests which match the "monitor fail" condition > > 504 when the response timeout strikes before the server responds > > In my case, there are 6 webservers running apache2 and they are available. > But 503 errors are in the log. > For 504 error, the same, i can't explain why a lot of my website have this > error. > > *Here is an extract of the haproxy.log (debug level)* > > Dec 29 18:54:18 127.0.0.1 haproxy[24349]: 66.249.72.182:47472 > [29/Dec/2011:18:53:48.816] public webserver/http-16 0/0/0/-1/30000 504 270 > - - sHNN 517/517/46/6/0 0/0 {www.webpedigrees.com} "GET > /inbreeding.php?nid=446368&nidp=&nidm=&nbgen=12 HTTP/1.1" > Dec 29 18:54:47 127.0.0.1 haproxy[24349]: 94.142.131.19:52345 > [29/Dec/2011:18:54:46.191] public webserver/http-19 942/0/-1/-1/943 503 288 > - - CCNN 504/503/41/5/0 0/0 {www.cordeaucou.com} "POST > /photos_mariage_zoom_comment.php?idp=374&image=336&x=318&tr=v HTTP/1.0" > Dec 29 18:55:16 127.0.0.1 haproxy[24349]: 207.46.199.52:49370 > [29/Dec/2011:18:55:16.121] public webserver/http-14 0/0/0/20/20 503 410 - - > --NI 465/463/35/13/0 0/0 {judaicultures.info} "GET > /les-arts-judaiques/arts-artistes-artisans/evenements-litteraires-parutions/Parution-du-Midrash-Rabba-sur-l?debut_article_rubrique_numerotes=20 > HTTP/1.0" > Dec 29 18:55:39 127.0.0.1 haproxy[24349]: 66.249.72.225:36265 > [29/Dec/2011:18:55:09.550] public webserver/http-17 0/0/4/-1/30006 504 270 > - - sHNN 488/488/43/12/0 0/0 {www.webpedigrees.com} "GET > /inbreeding.php?nid=446374&nidp=&nidm=&nbgen=11 HTTP/1.1" > Dec 29 18:55:51 127.0.0.1 haproxy[24349]: 94.142.131.19:49698 > [29/Dec/2011:18:55:50.811] public webserver/http-19 942/0/-1/-1/943 503 288 > - - CCNN 423/422/43/3/0 0/0 {www.jeunes-sapeurs-pompiers.fr} "POST > /index.php?option=com_joomlaboard&Itemid=43&func=post HTTP/1.0" > Dec 29 18:56:45 127.0.0.1 haproxy[24349]: > 213.111.38.127:60110[29/Dec/2011:18:56:39.165] public webserver/http-17 > 6585/0/-1/-1/6585 503 > 288 - - CCVN 491/491/49/6/0 0/0 {demaistre.fr} "GET > /gallery2/d/49262-2/20110917_163242_IMG_2776.jpg HTTP/1.1" > Dec 29 18:57:28 127.0.0.1 haproxy[24349]: > 178.33.204.50:38933[29/Dec/2011:18:56:58.380] public webserver/http-14 > 0/0/0/-1/30000 504 270 > - - sHNN 456/448/28/8/0 0/0 {meteor-center.com} "POST > /blog/wp-cron.php?doing_wp_cron HTTP/1.0" > Dec 29 19:12:04 127.0.0.1 haproxy[24349]: > 193.199.4.148:52226[29/Dec/2011:19:12:03.482] public webserver/http-18 > 1189/0/-1/-1/1189 503 > 288 - - CCVN 536/536/41/9/0 0/0 {www.playzgame.com} "GET > /online-flash-games/picture_70x60/scnclrx_70x60.jpg HTTP/1.1" > Dec 29 19:15:18 127.0.0.1 haproxy[24349]: > 41.110.231.242:1613[29/Dec/2011:19:14:45.657] public webserver/http-19 > 3183/0/2/-1/33186 504 > 270 - - sHVN 453/453/44/2/0 0/0 {www.thenia.net} "POST > /phpwebgallery/picture.php?/2247/category/28 HTTP/1.1" > > > *Most of the time on 503 error, it's CCNN* > > the TCP session was unexpectedly aborted by the client. > the proxy was waiting for the CONNECTION to establish on the server. The > server might at most have noticed a connection attempt. > the client provided NO cookie. This is usually the case for new visitors, > so counting the number of occurrences of this flag in the logs generally > indicate a valid trend for the site frequentation. > NO cookie was provided by the server, and none was inserted either. > > > *For 504 it's sHNN* > > the server-side timeout expired while waiting for the server to send or > receive data. > the proxy was waiting for complete, valid response HEADERS from the server > (HTTP only). > the client provided NO cookie. This is usually the case for new visitors, > so counting the number of occurrences of this flag in the logs generally > indicate a valid trend for the site frequentation. > NO cookie was provided by the server, and none was inserted either. > > For example, this url is always in 504 error : > http://www.webpedigrees.com/inbreeding.php?nid=446594&nidp=&nidm=&nbgen=11 > I don't understand why, specifically this url (in this case) > > Is any one have time to try to explain how to solve or how to find where > can be the problem... Because now this is the only one problem on my server > and i can't solve it myself :( > Thank you very much ! > > Cheers, > > Benoît Georgelin > *** > * > >