Hi, Got a few cycles to do some testing/verification. Sure enough when we pass a long URL through haproxy/lighttpd, we see this in the haproxy log:
Sep 19 22:45:14 127.0.0.1 haproxy[4195]: 172.28.xx.xx:55500 [19/Sep/2012:22:45:14.428] kernel_partner_ft kernel_partner_ft/<NOSRV> -1/-1/-1/-1/0 400 187 - - PR-- 0/0/0/0/0 0/0 "<BADREQ>" From your message, you say that haproxy will be marked with PR if blocked by haproxy, so this looks like haproxy is confirmed to be the cause of our problem. Correct? We're going to draft and test a configuration that increases maxrewrite, and perhaps bufsize. Any thoughts to modifying haproxy to limit allocating such large buffer areas for only certain frontends, instead of being only a global tunable? Tuning haproxy globally to solve a problem with processing only one frontend/backend seems rough. - Kevin On Sep 10, 2012, at 4:11 PM, Willy Tarreau wrote: > Hi, > > On Mon, Sep 10, 2012 at 08:20:02PM +0200, Baptiste wrote: >> On Mon, Sep 10, 2012 at 7:24 PM, Lange, Kevin M. (GSFC-423.0)[RAYTHEON >> COMPANY] <[email protected]> wrote: >>> Hi, >>> Our public-facing service provides a REST api to search for products >>> (geospatial science data), which requires in many cases very long URLs to >>> craft the search. We seem to be hitting a limit of around 8K before we >>> receive a 400 Bad request from lighttpd. We're trying to determine if >>> lighttpd is causing this, or haproxy. We have a dev/test stack >>> (lighttpd/haproxy on Linux) and an OPS stack of the same. Our OPS stack we >>> thought we had test results of the URL length maximum, but after we upgraded >>> haproxy to the latest on our OPS stack, we noticed that people began >>> complaining about the URL length issue (maximum 8k). Is there a >>> configurable item in haproxy which would limit a URL length to ~8k? >>> Suggestions from searches show that tune.bufsize might control this. We'd >>> like to offer our customers a 10k length for REST api calls. >>> - Kevin >>> >>> Kevin Lange >>> [email protected] >>> [email protected] >>> W: +1 (301) 851-8450 >>> Raytheon | NASA | ECS Evolution Development Program >>> https://www.echo.nasa.gov | https://www.raytheon.com >>> >> >> Hi, >> >> The 400 error may be issued by HAProxy. >> To know it, you should enable logs and share them here, so we'll know >> the real reason of the error. > > I confirm that, a request blocked by haproxy will be marked "PR" in the > logs while it will be normal "--" if it's emitted by the server. > >> I would have say like Richard: give a try to a bigger tune.bufsize . > > I'd say that if you need 10k, set your bufsize to the default 16k and > set tune.maxrewrite to 1k, you'll end up with 15kB for a complete > request which will be enough to store the large URL. > > Regards, > Willy > Kevin Lange [email protected] [email protected] W: +1 (301) 851-8450 Raytheon | NASA | ECS Evolution Development Program https://www.echo.nasa.gov | https://www.raytheon.com
smime.p7s
Description: S/MIME cryptographic signature
