Re: haproxy 2.0.x: another case of high CPU consumption
On Wed, Aug 14, 2019 at 12:52:00AM +0200, Cyril Bonté wrote: > And I confirm that commit 5b06cb33eb (BUG/MEDIUM: mux_h1: Don't bother > subscribing in recv if we're not connected) fixes the issue ;-) Great, thanks for confirming (and for reporting the issue in the first place)! Willy
Re: haproxy 2.0.x: another case of high CPU consumption
Hi again, Le 13/08/2019 à 11:57, Willy Tarreau a écrit : On Tue, Aug 13, 2019 at 11:49:20AM +0200, Willy Tarreau wrote: Excellent, I think you caught it! I can reproduce it here, except that it doesn't last long, as soon as I get the socket error it's done. So we indeed broke something in the connection setup. Bug kindly brought to you by your benevolent dictator : commit d58f27feadbc71c947fa0810f49552a94c60dc9a (refs/bisect/bad) Author: Willy Tarreau Date: Mon Jun 3 10:12:22 2019 +0200 MINOR: mux-h1: don't try to recv() before the connection is ready Just as we already do in h1_send(), if the connection is not yet ready, do not proceed and instead subscribe. This avoids a needless recvfrom() and subscription to polling for a case which will never work since the request was not even sent. I think the subscription prevents us from performing a synchronous operation somewhere, I'll check. And I confirm that commit 5b06cb33eb (BUG/MEDIUM: mux_h1: Don't bother subscribing in recv if we're not connected) fixes the issue ;-) -- Cyril Bonté
Re: haproxy 2.0.x: another case of high CPU consumption
On Tue, Aug 13, 2019 at 11:49:20AM +0200, Willy Tarreau wrote: > Excellent, I think you caught it! I can reproduce it here, except that > it doesn't last long, as soon as I get the socket error it's done. So > we indeed broke something in the connection setup. Bug kindly brought to you by your benevolent dictator : commit d58f27feadbc71c947fa0810f49552a94c60dc9a (refs/bisect/bad) Author: Willy Tarreau Date: Mon Jun 3 10:12:22 2019 +0200 MINOR: mux-h1: don't try to recv() before the connection is ready Just as we already do in h1_send(), if the connection is not yet ready, do not proceed and instead subscribe. This avoids a needless recvfrom() and subscription to polling for a case which will never work since the request was not even sent. I think the subscription prevents us from performing a synchronous operation somewhere, I'll check. Willy
Re: haproxy 2.0.x: another case of high CPU consumption
Hi Cyril, On Tue, Aug 13, 2019 at 11:38:23AM +0200, Cyril Bonté wrote: > Hi Willy, > > This morning, I found another case where haproxy 2.0 abnormally consumes the > cpu. It occured on my laptop when my network was not setup yet. > > I could find a simple reproducer : > listen test > mode http > bind : > server s1 10.0.0.1: > > then, curl http://localhost:/ > > 10.0.0.1 is unreachable and the connection stays in SYN_SENT, but haproxy > progressively raises to 100% CPU even after stopping curl. > running a second curl will raise the CPU up to 200%, etc... Excellent, I think you caught it! I can reproduce it here, except that it doesn't last long, as soon as I get the socket error it's done. So we indeed broke something in the connection setup. > I think I'll have time to investigate tonight but maybe you'll find the > issue and the fix before me ;) Let's try at least :-) > It looks to be very specific to the 2.0 branch, I can't reproduce it in 1.9 > and 2.1. Great, that's useful. I suspect the fdcache removal in 2.1 helps a lot here ;-) Thanks! Willy
