On Mon, Dec 12, 2016 at 1:44 PM, Hannes Frederic Sowa
<han...@stressinduktion.org> wrote:
On 12.12.2016 19:05, Josef Bacik wrote:
On Fri, Dec 9, 2016 at 11:14 PM, Eric Dumazet
<eric.duma...@gmail.com>
wrote:
On Fri, 2016-12-09 at 19:47 -0800, Eric Dumazet wrote:
Hmm... Is your ephemeral port range includes the port your load
balancing app is using ?
I suspect that you might have processes doing bind( port = 0) that
are
trapped into the bind_conflict() scan ?
With 100,000 + timewaits there, this possibly hurts.
Can you try the following loop breaker ?
It doesn't appear that the app is doing bind(port = 0) during normal
operation. I tested this patch and it made no difference. I'm
going to
test simply restarting the app without changing to the SO_REUSEPORT
option. Thanks,
Would it be possible to trace the time the function uses with trace?
If
we don't see the number growing considerably over time we probably can
rule out that we loop somewhere in there (I would instrument
inet_csk_bind_conflict, __inet_hash_connect and inet_csk_get_port).
__inet_hash_connect -> __inet_check_established also takes a lock
(inet_ehash_lockp) which can be locked from inet_diag code path during
socket diag info dumping.
Unfortunately we couldn't reproduce it so far. :/
Working on getting the timing info, will probably be tomorrow due to
meetings. I did test simply restarting the app without changing to the
config that enabled the use of SO_REUSEPORT and the problem didn't
occur, so it definitely has something to do with SO_REUSEPORT. Thanks,
Josef
