On Mon, 22 Oct 2012 22:07:16 +0200 Dan Fandrich <[email protected]> wrote: > On Mon, Oct 22, 2012 at 02:24:01AM +0200, Thorben Thuermer wrote: > > on http://curl.haxx.se/libcurl/features.html we find: > > "libcurl is designed and implemented entirely thread safe." > > "libcurl uses certain system calls to obtain information. Some of the most > > crucial ones are the name resoluition calls (the gethostby* family)." > > > > i just debugged a problem in a multithreaded application > > ( https://github.com/volkszaehler/vzlogger , thread in german: > > > > http://volkszaehler.org/pipermail/volkszaehler-users/2012-October/000619.html > > ) > > that appers to be caused by libcurl's (ab)use of alarm() to timeout > > gethostbyname() not being thread-safe. > > (as described here: https://bugzilla.redhat.com/show_bug.cgi?id=539809 ). > > > > there appears to be a race-condition when two threads run requests > > simultaneously, which reproducibly leads to a crash with > > the logging function being called with an invalid buffer/file, > > without any actual dns-timeout having occured. > > (the application only accesses localhost, which is defined in /etc/hosts, > > curiously the problem persists if a numeric IP is used.) > > If the problem exists even with a numeric IP, then it's unlikely to be a > DNS issue since numeric addresses bypass the external resolver. Are you > making SSL connections by chance? There is more you may need to do in that > case; http://curl.haxx.se/libcurl/c/libcurl-tutorial.html#Multi-threading
no, plain http. > > the problem disappeared when i compiled libcurl without HAVE_ALARM. > > CURLOPT_NOSIGNAL ought to do the same at run-time, i found that option in the meantime, yes. > but it's odd that > this occurs with numeric addresses (assuming SSL is not involved). i just checked by placing a breakpoint in alarm(), alarm() is called even when connecting to numeric IP addresses, when gethostbyname itself is apparently not called. (that was my point actually, the alarm() stuff is non-threadsafe, regardless of gethostbyname - thus the default resolver is not threadsafe, regardless of the OS-provided gethostbyname, unlike what the features page states. but as you point out below, one should be setting URLOPT_NOSIGNAL anyway.) > > i'm not sure if this is a libcurl bug or just a documentation issue, > > but it has cost us some time to track down... > > This is from curl_easy_setopt(3) > http://curl.haxx.se/libcurl/c/curl_easy_setopt.html > > CURLOPT_NOSIGNAL > > Pass a long. If it is 1, libcurl will not use any functions that install > signal handlers or any functions that cause signals to be sent to the > process. This option is mainly here to allow multi-threaded unix > applications to still set/use all timeout options etc, without risking > getting signals. (Added in 7.10) > > And this is from libcurl-tutorial(3) > http://curl.haxx.se/libcurl/c/libcurl-tutorial.html#Multi-threading > > When using multiple threads you should set the CURLOPT_NOSIGNAL option > to 1 for all handles. ok, so it is documented, just not where i was looking (i was looking for resolver-related issues after seeing the backtrace). i did not write the application itself, i'll blame the original developer for not reading the provided documentation on threaded usage. ;) > libcurl now gives a number of choices of DNS resolver back-end at > compile time, some of which are immune to this problem. yes, i found those. but it's rather unfortunate that this is compile-time so most users would have to build libcurl from source to take advantage of this. > >>> Dan thanks for the heads up, - T. ------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
