On Thu, Nov 9, 2023 at 8:02 AM Luis Carlos Chalaca Figueira via curl-library <curl-library@lists.haxx.se> wrote:
> On 01/11/23 18:32, Dmitry Karpov via curl-library wrote: > > -----Original Message----- > From: Luis Carlos Chalaca Figueira <luisccfigue...@gmail.com> > <luisccfigue...@gmail.com> > Sent: Wednesday, November 1, 2023 9:18 AM > To: libcurl development <curl-library@lists.haxx.se> > <curl-library@lists.haxx.se> > Cc: Dmitry Karpov <dkar...@roku.com> <dkar...@roku.com> > Subject: [EXTERNAL] Re: "Could not resolve host" error only on the multi > interface > > On 31/10/23 18:32, Dmitry Karpov via curl-library wrote: > > I would recommend to try without explicitly setting DNS server and timeout - > just to use the default DNS system settings. > Maybe, the "8.8.8.8" is not routable in your setup. > > Also, I would run Wireshark or tcpdump to see if your DNS queries/response > were actually sent/received on the wire. > > Thanks, > Dmitry Karpov > > -----Original Message----- > From: curl-library <curl-library-boun...@lists.haxx.se> > <curl-library-boun...@lists.haxx.se> On Behalf Of > Luis Carlos Chalaca Figueira via curl-library > Sent: Tuesday, October 31, 2023 10:08 AM > To: curl-library@lists.haxx.se > Cc: Luis Carlos Chalaca Figueira <luisccfigue...@gmail.com> > <luisccfigue...@gmail.com> > Subject: [EXTERNAL] "Could not resolve host" error only on the multi > interface > > Hello everybody, > > > I'm building a web crawler. > > Started with python but the need for speed led me to libcurl. > > The obstacle I'm having is: when using the multi interface a big part of the > the connections cloe because of the error "Could not resolve host:" > > What I've tried: > > 1 - Confirmed that all URLs are correct. > > 2 - Confirmed that using the easy interface URLs are resolved. > > 3 - Confirmed that other programs like dig, resolvectl, nslookup can resolve > URLs that multi interface couldn't. > > 4 - Set a "reputable" DNS server doing: curl_easy_setopt(handles[i], > CURLOPT_DNS_SERVERS, "8.8.8.8");. > > 5 - In the file /etc/nsswitch.conf, line "hosts: files > nDNS[NOTFOUNDED=return] dns" I deleted [NOTFOUNDED=return]. > > 6 - I've turned verbose on - Doesn't help me much. > > I don't know what to do more as it seems a problem inside the library, so i > posted here before going digging in the library. > > > System info: > > O.S. - Ubuntu 22.04.3 LTS > > Harware - HP HP Pavilion Laptop 14-ce3xxx > > Processor - Intel® Core™ i7-1065G7 CPU @ 1.30GHz × 8 > > libcurl version - 7.81.0 > > > Attached I send the source code file I'm using do debug this problem. > > > Best Regards, > > Luis Figueira > > > Dimitry, > > Thanks for the response, > > However it looks like my e-mail was not clear, maybe because English is my > second language. > > Those 6 steps I took were made after I realized that most requests names > could not be resolved therefore they are not the problem. However I've reset > all of them to the default state. > > I've used tcpdump and the output only revealed what the the system sends > packets only to one address, the stub address (Which discovered by using > nslookup). Below an exctract from the output: > > 19:09:50.875611 IP6 ubuntu.39274 > 2001:8a0:776c:b200::1.domain: 53886+ AAAA? > e10952.a.akamaiedge.net. (41) > 19:09:50.876703 IP6 ubuntu.51246 > 2001:8a0:776c:b200::1.domain: 27951+ AAAA? > e-cultura.pt. (30) > 19:09:50.880193 IP6 ubuntu.58635 > 2001:8a0:776c:b200::1.domain: 52688+ AAAA? > orientacao-vocacional.com.pt. (46) > 19:09:50.881306 IP6 2001:8a0:776c:b200::1.domain > ubuntu.47511: 24563 > 0/1/0 (104) > 19:09:50.882880 IP6 2001:8a0:776c:b200::1.domain > ubuntu.35687: 64715 > 0/1/0 (110) > 19:09:50.883430 IP6 2001:8a0:776c:b200::1.domain > ubuntu.43052: 35782 > 0/1/0 (106) > > What I will do now is ask curl to give me every type of error code for each > request and see if I can get what is happening. > > -- > Cumprimentos, > Luis Figueira > > " What I will do now is ask curl to give me every type of error code for each > request and see if I can get what is happening." > > I don't think libcurl can easily do that. It works with different DNS > backends (i.e. c-ares or libc) and they don't provide the same error > information. > > In the past, I proposed to add an "info" option which would return a resolver > error code, so it could be logged on remote devices and reported, > but the proposal was rejected because of differences between multiple > resolver backends. > > Thanks, > Dmitry > > > It looks like it might be the fact that my code is multi-threaded that is > creating these errors, because using both interfaces single-threaded does > not yield any error but threading any of them leads to errors. > > Therefore I wave 2 questions: > > 1 - About multithreaded name resolving: - I would say Ubuntu 22.04.3 LTS > provides thread safe name resolving system calls ? - How can I find out? > Yes. glibc provides that just fine. it also provides an old messy api that you should never use. curl configure script picks the right version unless there is something very wrong with the environment. > 2 - About multithreading TLS: - folowed the threaded-ssl example but got > the compiling error: > > /usr/bin/ld: crawler.o: undefined reference to symbol ' > CRYPTO_malloc@@OPENSSL_3.0.0' > /usr/bin/ld: /lib/x86_64-linux-gnu/libcrypto.so.3: error adding symbols: > DSO missing from command line > collect2: error: ld returned 1 exit status > make: *** [Makefile:5: crawler] Error 1 > > > CRYPTO_malloc is exported by -lcrypto (libcrypto.so.3.x) the linker no longer forgives you for not adding -lcrypto explicitly on the command line even if it knows what to do..this expected, the amount of bugs the old implicit behaviour provoked beggars belief.
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
