On Wed, 9 Aug 2017, fan fan via curl-library wrote:

In my application, I using libcurl to send 40 HTTP requests in every one second. As I test, this consume much CPU, my application such as:


 main thread for sending request and using epoll listen fds whether read
or write.

   while(running) {
         int ret = epoll_wait(epollFd, events, maxEventNum, 10); //10ms

Unconditionally returning every 10 milliseconds like this will make you spend a lot of CPU in vein. Don't do that. libcurl tells you how long it wants you to wait at most. No need to spin around a lap when there's nothing for libcurl to do.

Do you also call libcurl to tell there's a timeout if the 10 milliseconds lapse or when do you do that?

         for (i = 0; i < ret; i++) {
              if (events[i].events & EPOLLIN) {
                     curl_multi_socket_action   // set CURL_CSELECT_IN
                     curl_multi_info_read  // for reading

I think you should put the curl_multi_info_read() call outside of the loop checking for activities, since it might not have anything to return and then its a bit of a waste to call it up to 40 times.

But sure, curl_multi_socket_action() may of course consume CPU since that's the function that will handle the traffic on the socket you give it (and possibly on a few others if necessary).

As I test,  send 100 or 200 or 300 or 400 or 500 in parallel  maybe consume
cpu 100%.

On modern hardware that shouldn't happen.

I found there are too much EPOLLIN event, this consume much cpu.

What do you mean with too much EPOLLIN event? If libcurl tells you to wait for that, shouldn't you then be glad you get it so that you can download the data?

1. Does my use libcurl wrong?

Ineffective at least. But you also didn't show the whole program so its not possible to tell the whole story.

2. Have any performance measurement for libcurl process in parallel?

It should easily handle thousands of parallel connections, CPU wise. Very likely tens of thousands too. Of course they may have to struggle with each other for other resources (like bandwidth).

--

 / daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Reply via email to