On Fri, Sep 13, 2013 at 12:40:45PM +0400, Sergey Pinchuk wrote:
>
> >Yes, the show sess should help a lot with all the flags that are dumped.
> >
> >And clearly a pcap could help as well ! Sergey, this time we don't
> >need the data in the pcap file, so if you can take one, you do not
> >need to use -s0, you'll get a smaller file and won't disclose possibly
> >sensitive information.
> >
> Ok, for now I have got pcap file with host 194.186.69.10 that generated
> almost simultaneously 4 "suspended" CLOSE_WAIT sockets on ports 1363,
> 1365, 1369, 1371 (all other traffic excluded). Hope it helps. This bug
> is very difficult to reproduce and we will try to get strace and unix
> socket info of problem.
Thank you. These ones just show empty connections that are closed by the
client soon after opening. In tcp mode, what normally happens is that the
close is forwarded to the other side, and when the other side closes in
turn, the connection to the client finishes as well.
So we may have various possible cases here :
- it is possible that the close happens before haproxy manages to connect
to the server and that we end up in a rare case where nothing else moves.
- for whatever reason the close is not forwarded to the server but still
the timeout is cleared. I don't think this happens thanks to your
graph that shows CLOSE_WAIT only taking a lot of room.
- maybe it's only the server that does not close in response. It could
be possible because then the connection to the server would be in
FIN_WAIT2 and vanish quickly enough not to appear in a significant
way on your graph. Still, haproxy's server timeout should trigger
in this case.
- maybe the server correctly responds and haproxy forgets to forward.
That's quite strange as well but should be studied.
It would be nice if you could check on the server side what the situation
looks like (server responds data maybe, etc...).
In parallel, I'll try to reproduce this behaviour here.
Thanks,
Willy
