Hi! Thank you very much for the patches. Looks like they helped.
2018-03-29 14:25 GMT+05:00 Christopher Faulet <[email protected]>: > Le 28/03/2018 à 14:16, Максим Куприянов a écrit : > >> Hi! >> >> I'm sorry but configuration it's too huge too share (over 100 different >> proxy sections). This is also the reason I can't exactly determine the >> failing section. Is there a way to get this data from core-file? >> >> 2018-03-28 11:18 GMT+03:00 Christopher Faulet <[email protected] >> <mailto:[email protected]>>: >> >> Le 28/03/2018 à 09:36, Максим Куприянов a écrit : >> >> Hi! >> >> Yesterday one of our haproxies (1.8.5) with nbthread=8 set in >> its config stuck with 800% CPU usage. Some responses were served >> successfully but many of them just timed out. perf top showed >> this: >> 59.19% [.] thread_enter_sync >> 32.68% [.] fwrr_get_next_server >> >> >> Hi, >> >> Could you share your configuration please ? It will help to diagnose >> the problem. In your logs, what is the values of srv_queue and >> backend_queue fields ? >> >> > Hi, > > Ok, I partly reproduce your problem using a backend, with an hundred > servers and a maxconn to 2 for each one. In this case, I observe same CPUs > consumption. I have no timeouts (it probably depends on your values) but > performances are quite low. > > I think you're hitting a limitation of the current design. We have no > mechanism to migrate entities between threads. So to force threads wakeup, > we use the sync point. It was not designed to be called very often. In your > case, it eats all the CPU. > > I attached 3 patches. They add a mechanism to wakeup threads selectively > without any lock or loop. They must be applied on HAProxy 1.8 (it will not > work on the upstream). So you can check if it fixes your problem or not. It > will be useful to validate it is a design limitation and not a bug. > > This is just an experimentation. I hope it works well but I didn't do a > lot of testing. If yes, I'll then discuss with Willy if it is pertinent or > not to do the threads wakeup this way. But, in all cases, it will probably > not be backported in HAProxy 1.8. > > -- > Christopher Faulet >
