I wonder if you in that specific example (the stackexchange one) are hitting some limit in the dns resolver since "domains/s=161.98" which is the max value there would be something like (1 event for dns reply, 1 for connect, 1 for write is what I'm guesstimating at here) ~486 events in epoll per second is not much events at all.
I wouldn't be at all surprised if public DNS servers such as the Cloudflare and Google one used in your example have some kind of rate limiting. /HH Den ons 1 juni 2022 kl 19:42 skrev James Read via curl-library < curl-library@lists.haxx.se>: > > > On Wed, Jun 1, 2022 at 2:22 PM Aleh Linkin <aleh.lin...@idt.net> wrote: > >> Thanks for the update. Could you be a bit more specific? What do you mean >> under "problem with the TCP/IP stack"? If I'd like to use >> curl_multi_socket_action I had to sacrifice some CPU? >> > > Some time ago I posted a question related on stackoverflow: > https://stackoverflow.com/questions/70584121/why-doesnt-my-epoll-based-program-improve-performance-by-increasing-the-number > > > The question includes no cURL based code which demonstrates the > limitations of epoll with the current TCP/IP stack. I don't know exactly > what the problem with the stack is. I'm still working on it. > > James Read > > >> >> On Wed, Jun 1, 2022 at 4:04 PM James Read <jamesread5...@gmail.com> >> wrote: >> >>> The problem is not with cURL. I have also run into these problems using >>> cURL. I recoded using sockets and epoll and got the same results. The >>> problem is with the TCP/IP stack. >>> >>> On Wed, Jun 1, 2022 at 1:46 PM Aleh Linkin via curl-library < >>> curl-library@lists.haxx.se> wrote: >>> >>>> 1. easy_handles >>>> ``` >>>> SET_OPT(CURLOPT_ERRORBUFFER, m_errorBuffer); >>>> setVerbose(true); >>>> setUrl(url); >>>> setTimeout(timeout); >>>> setConnectTimeout(connectTimeout); >>>> setProxy(proxy); >>>> setProxyPort(proxyPort); >>>> setHttpHeaders(httpHeaders); >>>> SET_OPT(CURLOPT_WRITEFUNCTION, writeFunction); >>>> SET_OPT(CURLOPT_WRITEDATA, m_requestState); >>>> SET_OPT(CURLOPT_DEBUGFUNCTION, CurlRequest::debugFunction); >>>> SET_OPT(CURLOPT_DEBUGDATA, this); >>>> SET_OPT(CURLOPT_PRIVATE, this); >>>> SET_OPT(CURLOPT_NOSIGNAL, 1L); >>>> ``` >>>> >>>> 2. HTTPS. TLS based on NSS SSL backend >>>> >>>> 3. Yes. I've used perf and valgrind-cachgrind but I can't take the >>>> reason from results. For example perf part: >>>> ``` >>>> Samples: 1K of event 'cycles:uppp', Event count (approx.): 9397031894 >>>> Children Self Command Shared Object Symbol >>>> - 23.25% 2.43% lecappqry.1 [unknown] [.] >>>> 0000000000000000 >>>> - 21.58% 0 >>>> 5.29% _int_malloc >>>> 1.42% free >>>> 1.13% Curl_http >>>> 1.07% _int_free >>>> + 0.88% __clock_gettime >>>> 0.72% _IO_vsnprintf >>>> 0.71% __libc_calloc >>>> + 5.07% 0.00% lecappqry.1 [unknown] [k] >>>> 0x0000000000667849 >>>> + 5.07% 0.00% lecappqry.1 [unknown] [k] >>>> 0x00007f33507ad900 >>>> + 5.07% 4.87% lecappqry.1 lecappqry_grpc [.] >>>> dprintf_formatf >>>> + 4.88% 4.71% lecappqry.1 libc-2.17.so [.] _int_free >>>> ``` >>>> >>>> On Mon, May 30, 2022 at 9:07 PM Patrick Schlangen <patr...@schlangen.me> >>>> wrote: >>>> >>>>> Am 30.05.2022 um 16:23 schrieb Aleh Linkin via curl-library < >>>>> curl-library@lists.haxx.se>: >>>>> > >>>>> > I understand. Thank you. >>>>> > But why multi_socket_action eats about twice time more CPU then >>>>> curl_multi_perform called in cycle with 10ms pause? >>>>> >>>>> Can you supply some more details about how the easy handle is >>>>> configured and the type of transfers you are running? Is it HTTP? Are you >>>>> using TLS? Did you use perf or a similar profiler yet to see where time is >>>>> spent? >>>>> >>>>> Thanks, >>>>> >>>>> Patrick >>>>> >>>>> -- >>>> Unsubscribe: https://lists.haxx.se/listinfo/curl-library >>>> Etiquette: https://curl.haxx.se/mail/etiquette.html >>>> >>> -- > Unsubscribe: https://lists.haxx.se/listinfo/curl-library > Etiquette: https://curl.haxx.se/mail/etiquette.html >
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
