Hello Dan,
On Wed, Apr 20, 2022 at 05:27:40PM +0000, Moore, Dan [TREAS] wrote:
> Hello all,
>
> My problem is that my AppDev group has some web applications generating
> reports that are taking a very long time to run which is resulting in 504s.
> An example httplog entry is below.
>
> Apr 19 10:53:42 xxxxxxxxx haproxy[6173]: XX.XX.XX.XX:2499
> [19/Apr/2022:10:52:32.339]
> http-in-VIPX-XXXXXXXXXX-UAT Cluster_XXXXXXXXXX_UAT/hostname.com
> 0/0/0/-1/70001 504 262 - - sH-- 3/3/0/0/0 0/0
> "GET /XXXXXXX/Reports/SubmitReportDownload?FileTypeID=1&ReportTypeID=1
> HTTP/1.1"
>
> My thinking was that this was due to the option "timeout server 70s" in my
> defaults (hence the 70001 Ta value). There's also the "sH--" session
> termination state which, by my interpretation, means "server side timeout
> waiting for response Headers" which also seems to point to the server side
> timeout.
You're absolutely right, this is exactly what causes this status code
and termination flags to appear here.
> Because of these, I decided to leave the default at 70 seconds and
> override it in the backend by setting "timeout server 120s" (the 120s
> proposed by AppDev). This had no effect so I tried multiple values (and some
> other timeout parameters) all to no effect. Finally, given this is my
> Dev/UAT NLB, I tried changing the default "timeout server" to 120s and,
> still, no effect.
What do you mean by "no effect" ? Do you mean "it continues to fire at 70s"
or "it still fires but after 120s now" ? Because if you're seeing "sH" with
"timeout server 120s" it should fire at 120s.
> The web server in this case is IIS and AppDev says the
> reports work going directly so it seems to be something specific to my
> haproxy config (the relevant part of which is below).
Does it only happen once in a while or does it happen often ? I'm asking
because then it looks slightly related to an issue we're having here by
which some requests sent over HTTP/2 occasionally experience a delay
that corresponds to the connect timeout:
https://github.com/haproxy/haproxy/issues/1468
We've been suspecting that a wakeup event is missing somewhere and that
sometimes it would be caught by the next firing timeout, but for now we
failed to reproduce it and to figure what could be happening. It would
be interesting to know if you're also seeing long connection delays that
match your connect timeout or if you're only seeing response timeouts.
Oh by the way, you forgot to report the version you're using, please
send the output of "haproxy -vv".
Regards,
Willy
