Re: Username and password in URL stopped working with proxy
On Fri, 2 Aug 2019, Mike Crowe via curl-library wrote: I've managed to reproduce the problem against our Squid proxy with the the curl command line tool in the current state of master: curl --proxy http://proxy:3128/ --anyauth http://unittest:password@host/ And you're using curl 7.65.3 ? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
2000 contributors today
Hi, As of today, we count no less than 2000 contributors who have helped out making curl into what it is today. Thank you everyone! https://daniel.haxx.se/blog/2019/08/01/2000-contributors/ -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Can you help us curl up in Berlin 2020?
Hi friends, curl up is our annual low key curl developers conference. We do it once a year and the public vote says the spring is still the best time. We've been between 20 and 30 persons each year. We're now thinking of hosting curl up in Berlin in May 2020. (*thinking* being the operative word, nothing is decided yet!) For this to happen and work out nicely, I would appreciate help priarily with suggestions on a suitable venue for us and ideally one or two volunteers "on the ground" in this city that can help us out. To help us out we have some docs in the curl up wiki [1], like the Venue requirements [2] and for inspiration the agendas from earlier curl ups like 2019 [3] and 2018 [4]. We're aiming for Berlin this time after the vote back in April 2019 [5]. For additional inspiration, we have recordings from many of the presentations done at previous curl ups saved on the curl web site[6]. (For more detailed and in-depth discussions of curl up, what to do there, the agenda and all of those things, consider subscribing to and taking such topics to the curl-meet [7] mailing list.) Thanks for considering! [1] = https://github.com/curl/curl-up/wiki [2] = https://github.com/curl/curl-up/wiki/venue-requirements [3] = https://github.com/curl/curl-up/wiki/2019 [4] = https://github.com/curl/curl-up/wiki/2018-agenda [5] = https://github.com/curl/curl-up/wiki/poll-2020 [6] = https://curl.haxx.se/docs/videos/ [7] = https://cool.haxx.se/mailman/listinfo/curl-meet -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
CUSTOMREQUEST + FOLLOWLOCATION
Hey team, I have PR #3803 [1] queued up that I'm looking for some thumbs up or down on. The problem it addresses: A user sets CUSTOMREQUEST to a string and enables FOLLOWLOCATION. When libcurl follows a redirect, it will use the CUSTOMREQUEST even in the susequent followed-to URL which in some/many cases is not what the user want and is not really what the HTTP response code says either. This is occasionally also seen when users superfluously use -X on curl command lines... The proposed change: The PR introduces a new bit for the CURLOPT_FOLLOWLOCATION option that if set, will make libcurl *NOT* use the custom method string when it follows a redirect when the HTTP response code says so. To ponder about: 1. Is it worth the added complexity with yet another knob? 2. This doesn't solve the -X issue since we don't want to break the existing functionality where users want the current behavior. Should we add a -X alternative for this? Thoughts? [1] = https://github.com/curl/curl/pull/3803 -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Procedure to use libcurl without CURL_GLOBAL_SSL
On Wed, 31 Jul 2019, Dipak B via curl-library wrote: - What are the steps to initialise libcurl without CURL_GLOBAL_SSL? Could not find answer to this after spending time on this. Since 7.57.0, CURL_GLOBAL_SSL has no meaning for libcurl. And before that, it wasn't clearly documented what it meant anyway. Mentioned here: https://curl.haxx.se/docs/knownbugs.html#CURL_GLOBAL_SSL -The documentation states that when CURL_GLOBAL_SSL is not used then initialisation of SSL layer needs to be done by application or atleast outside of libcurl. Before 7.57.0, yes. But it was not clear exactly what it meant, correct. It was highly error-prone and mostly artifacts of past decisions not reconsidered in time. - My understanding is to call OpenSSL APIs for initialisation of TLS but then I will have to link to OpenSSL too. Yes. But I would not go down that route if I were you, I would rather work with a modern libcurl version and if you're missing something from the TLS usage within libcurl (like FIPS) then work on fixing that inside libcurl proper. Presently, I am linking with static libcurl only. Will this work? Is this the correct understanding? Sure, since libcurl prusumably can use openssl then your app can too, if you just link it correctly (and include the right headers etc). - What are steps to initialise TLS layer without using CURL_GLOBAL_SSL? That's unsupported behavior these days. To go down that route, I would advice you to consider paid support (for example as explained in my signature) as I doubt many people here feel like doing archeology like that just for the fun of it... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: libcurl in fips mode
On Tue, 30 Jul 2019, Dipak B via curl-library wrote: Can you please help me with the following question? How do I use libcurl in FIPS mode? libcurl has no special provisions for FIPS. If any source code changes or function invokes are necessary, you need to make them. OpenSSL FIPS support seems to only exist in the outdated 1.0.2 version and according to https://www.openssl.org/docs/fips/UserGuide-2.0.pdf just linking with a FIPS OpenSSL 1.0.2 is not enough. It then also needs FIPS_mode_set() to be called. (That's a 225 page document and I only skimmed it very casually so I'm far from being knowledgable in this area.) It would probably be suitable to have curl's configure be able to detect that function and be able to use it. But I'm hesitant to add support for that now since OpenSSL 1.0.2 is old and reading on the openssl site it seems they intend to do FIPS differently going forward. A possibly more reliable way forward right now would be to instead switch to wolfSSL that offers a FIPS version of their current version that is supported. a. Could not find curlopt_xxx for FIPS mode. Apology if this is not needed. Normally I would guess that you want FIPS if the FIPS-enabled library was used in the build so such an option wouldn't be used, but I've not received much feedback on this topic from FIPS-using curl users so I'm mostly guessing. b. Checking if VTLS interface can be leveraged to pass socket for which FIPS is configured. I don't think that sounds like a viable way forward. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: resume of interrupted transfers from/to SMB shares not working as I had hoped
On Tue, 30 Jul 2019, Alfred Eisenberg (aeisenbe) via curl-library wrote: Does the latest curl library fully support resume of interrupted uploads to SMB shares? I don't think it does. I find no references to resume at all in the lib/smb.c source code... :-( -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: curl_multi_poll: a sister to curl_multi_wait() that waits more
On Tue, 30 Jul 2019, Christopher Head via curl-library wrote: When I saw the sentence, “If no extra file descriptors are provided and libcurl has no file descriptor to offer to wait for, this function will return immediately,” in the documentation, I originally thought this meant if there are no easy handles in the multi handle, but I see now that it actually might also mean it returns immediately if there are easy handles in the multi handle that are carrying out transfers, but none of them actually want their FD to be selected on right now. Yes correct, that is exactly what it means. If you can think of a way to say that in a way that is clearer to the reader, please suggest! I was wondering what the implications were for CURLOPT_MAX_RECV_SPEED_LARGE? That is described as “If a download exceeds this speed… the transfer will pause,” but from inspecting the code, I do not think the word “pause” here refers to “curl_easy_pause.” I think it actually means an internal-to-libcurl delay, after which the easy handle automatically continues transferring data That is exactly what happens. The application just sets the option and then it makes the transfer just like normally and libcurl will internalls do short pauses when it has to in order to keep the transfer speed below the specified limit. When a transfer is determined to have been over the limit and it needs to wait a little to have it cool down, the socket is removed from the set of sockets to wait for... (as an aside, I think this might be worth rewording in the documentation, if my interpretation is correct that it is not related to curl_easy_pause). I hear you. I never thought about it that way! Maybe avoid the word "pause" there and just say "slow down" or something? if I now understand correctly, with no FDs selectable, curl_multi_wait will return immediately; does this include if all easy handles are throttling? Yes How do I know that this happened, so that I know I have to sleep outside curl_multi_wait rather than just carrying on immediately? It's difficult as there's no clear indication it happens. Like there's no return code or state that tells us this. Which of course is one reason why I propose this new API so that we can have this sorted out. One way I've used myself is simply have it time-based. If the function returns (repeatedly a few times) within N milliseconds and reports no sockets to wait for, call curl_multi_timeout() to get a suitable timeout and simply sleep for that many milliseconds. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: curl_multi_poll: a sister to curl_multi_wait() that waits more
On Mon, 29 Jul 2019, m brandenberg via curl-library wrote: This function works identically to curl_multi_wait() - EXCEPT - for the case when there's nothing to wait for internally, as then this function will by itself wait for a "suitable" short time before it returns. So 'wait' polls and 'poll' waits? Hehe, yeah pretty much. I struggled to come up with another name and even considered curl_multi_wait2(), but this is the best I've come up with so far! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
curl_multi_poll: a sister to curl_multi_wait() that waits more
Hi team, Very soon after we introduced curl_multi_wait() I realized it had an awkward behavior that makes it tricky to use at times. Today I wrote up an initial proposal for a drop-in function replacement that probably is more what application authors actually want and need - and some probably already thought curl_multi_wait() worked like this. I'm open for all and any feedback! Nothing in this is written in stone yet... See the relatively small patch in PR 4163: https://github.com/curl/curl/pull/4163 DESCRIPTION Repeatedly we see problems where using curl_multi_wait() is difficult or just awkward because if it has no file descriptor to wait for internally, it returns immediately and leaves it to the caller to wait for a small amount of time in order to avoid occasional busy-looping. This is often missed or misunderstood, leading to underperforming applications. This change introduces curl_multi_poll() as a replacement drop-in function that accepts the exact same set of arguments. This function works identically to curl_multi_wait() - EXCEPT - for the case when there's nothing to wait for internally, as then this function will by itself wait for a "suitable" short time before it returns. This effectiely avoids all risks of busy-looping and should also make it less likely that apps "over-wait". This also changes the curl tool to use this funtion internally when doing parallel transfers and changes curl_easy_perform() to use it internally. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: How to do optimal FTP upload for multiple files?
On Thu, 25 Jul 2019, Taras Kushnir wrote: (Please stop the top-posting, it is highly confusing) Resume code originates from https://curl.haxx.se/libcurl/c/ftpuploadresume.html Right, but then it wasn't libcurl doing it, it was your application... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: How to do optimal FTP upload for multiple files?
On Wed, 24 Jul 2019, Taras Kushnir wrote: I disabled UPLOAD_BUFFERSIZE and after upgrade 3 different users reported that now they upload without issues (libcurl 7.65.3). That certainly indicates we have a bug in there somewhere... The thing is that libcurl uploads whole file (confirmed by parsing Content-Length from), however, it times out and then it tries to resume transfer, however, 100% of the file is transferred. Are you sure this is actually libcurl that tries to resume? I can't recall that we have code that does this. Do you have any recommendations? Is it safe to hack it in a way if we confirm 100% is transferred by parsing content-length, but no server confirmation (226, if I’m not mistaken), is it still safe to consider it as “done” or not really? Under typical circumstances it would not, but just within given context, when for some reason we time out after. This is probably the same control connection time-out I've already spoken of. When the transfer takes time, some networks will have killed the control connection by the time the transfer is done and then libcurl will get a time-out on that one. If that's a situation you can detect and then ignore, then it is up to you to do that. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Regarding Async DNS resolver
On Tue, 23 Jul 2019, Amit wrote: I have added the changes under compile time switch (HAVE_SOCKET) to fix compilation on Windows. I think you meant HAVE_SOCKETPAIR ? But also, you reintroduced several of the code style fixes. I propose you run 'make checksrc' to have the build point out some of the code style nits. Can you please submit this is a PR instead? That's the preferred way to submit changes as it will test and verify your changes before merge etc. I have done the changes only for HAVE_GETADDRINFO. Let me know if you would like me to consider the use-case of gethostbyname_thread as well ? I don't consider that a requirement, but more of a bonus point thing. If you're up to it, it would certainly make the fix even better but there should be a rather small amount of users on that code path these days. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Regarding Async DNS resolver
On Mon, 22 Jul 2019, Amit wrote: I have done the changes as per your suggestion - that is, to create socket pair during async thread initialization, return the read socket fd to client application and write dummy data to signal client that socket is readable. I did the testing on version which I am using (7.57.11) and verified the changes. Could you please review the attached patch file (generated after merging the changes to latest libcurl) ? . Thanks! This is a great start. First, I had to edit it slighly to fix some checksrc warnings and I changed Curl_resolver_getsock() to not do the short expire things if there is a file descriptor to wait for as that would defeat its purpose a bit. I created a PR out of this for you so that it'll run all the tests first: https://github.com/curl/curl/pull/4140 It needs to go green in all the tests to be subject for merge. Since this code seems to be using socketpair() unconditionally (ie also on Windows) I assume this won't build on Windows. It need to make that conditionally on the presense of that function adding #ifdef HAVE_SOCKETPAIR perhaps? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: How to do optimal FTP upload for multiple files?
On Fri, 19 Jul 2019, Taras Kushnir wrote: I didn't investigate curl commandline tool upload time on my machine since for me priority #1 is to fix upload errors of users of my app (since they will just stop using it for upload). I will do so right after finding some solution to this problem. Trying the same operation with the curl tool would be a way to diagnose what you're experiencing. If curl doesn't show the problem, then it is specific to your use of libcurl. If curl *does* show it, then it seems likely to be a more generic libcurl issue. I understand about NAT/firewall/etc. timeout, but the thing is that libcurl 7.57 works for them right at the same time (of course, it could be a very big coincidence). I'm not suggesting it is a coincidence. You already say it uploads much slower, so the risk for the control connection to time out is therefore much higher so I think the correlation seems rather likely. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: How to do optimal FTP upload for multiple files?
On Fri, 19 Jul 2019, Taras Kushnir wrote: On the same machine at the same time, app with libcurl 7.57 uploads without timeout and app with libcurl 7.65 and UPLOAD_BUFFERSIZE change uploads with timeout. And how long did the uploads take in both these cases? Was the timeout always the control connection one Daniel showed? Since I’m not a big expert in libcurl (that’s why I’m seeking your help in this mailing list) of course I thought that UPLOAD_BUFFERSIZE is the problem (because that’s all that I personally changed). I understand that, but it is important to keep an eye on *all* the details and facts so that we don't accidentally focus on the wrong things. If changing UPLOAD_BUFFERSIZE actually makes the upload slower, it can of course explain why the control connection would time out if that slowness pushes the transfer time over the "magic threshold" (which to us happens to be unknown). It of course doesn't at all explain *why* changing that size makes the upload slower, because as I've said already, it really shouldn't. If you do FTP upload with the curl command line tool, does it also show the same kind of "slowness" ? Of course, I can just try to increase the timeout in my app and see if it helps, but at least it’s important (and interesting) to understand “why?”. The timeout in question here is most likely in a router, NAT or firewall between the client and the server so there's no timeout in the app that you can change for this! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: How to do optimal FTP upload for multiple files?
On Fri, 19 Jul 2019, Daniel Jeliński wrote: Here's what we found in the logs: 18:54:41.002 T#12216 Connectivity::my_trace - "== Info: We are completely uploaded and fine" 18:54:41.002 T#12216 Connectivity::my_trace - "== Info: Remembering we are in dir \"\"" 18:54:51.012 T#12216 Connectivity::my_trace - "== Info: FTP response timeout" 18:54:51.012 T#12216 Connectivity::my_trace - "== Info: control connection looks dead" This does indeed look like the control connection has been dropped while the transfer was done. At least that's the symptom of that. The problem is that the first line above (completely uploaded and fine) is logged when the OS accepts the last application buffer into OS buffer. And Windows accepts buffers whole - send never returns a partial result, it's either all or nothing. So we log that we are finished while we still have 1MB outstanding on data connection. That would be highly annoying and special, and this winsock docs[1] says otherwise: On nonblocking stream oriented sockets, the number of bytes written can be between 1 and the requested length, depending on buffer availability on both the client and server computers. [1] = https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-send Curl FTP expects a response on control connection within 10 seconds after it sends the last data buffer, and then declares the connection dead. While we could probably modify that timeout, we have no way to tell how much time is enough. Right, but longer than 10 seconds response time in a case where the control connection actually is still alive should be extremely rare. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
[RELEASE] curl 7.65.3
Hi friends, Another day, another release. This is just a very small fix for the annoying progress meter regression in 7.65.2. A single-fix release. Get it as always from https://curl.haxx.se/ curl and libcurl 7.65.3 Public curl releases: 184 Command line options: 221 curl_easy_setopt() options: 268 Public functions in libcurl: 80 Contributors: 1991 This release includes the following bugfixes: o progress: make the progress meter appear again [1] This release includes the following known bugs: o see docs/KNOWN_BUGS (https://curl.haxx.se/docs/knownbugs.html) This release would not have looked like this without help, code, reports and advice from friends like these: Chih-Hsuan Yen, Daniel Stenberg, (2 contributors) Thanks! (and sorry if I forgot to mention someone) References to bug reports and discussions on issues: [1] = https://curl.haxx.se/bug/?i=4122 -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: How to do optimal FTP upload for multiple files?
On Thu, 18 Jul 2019, Daniel Jeliński via curl-library wrote: As for the connection timeout, it appears to be a well known problem with FTP on slow connections with oversized buffers. I just found a 10 year old message describing what looks like the same problem: https://curl.haxx.se/mail/archive-2009-08/0060.html I believe that's a misunderstanding. That's a known problem with long-going FTP transfers - totally independent of buffer size. Since FTP uses two connections, sometimes when a transfer takes more than a certain time, a network equipment like a NAT or a firewall closes the control connection due to inactivity before the transfer is done. But that doesn't seem to be what Taras has described here. Since libcurl uses non-blocking sockets internally I can't see any reason why a larger upload buffer would cause a greater risk for any timeout. What am I missing? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: [RELEASE] curl 7.65.2
On Thu, 18 Jul 2019, Christian Schmitz wrote: I seem to have broken the progress meter output! :-/ But only for the tool, right? So libcurl users can ignore it? The default progress meter is displayed by libcurl itself so it affects all applications that use it but yeah, it is probably most notable in the curl tool. I don't have any numbers for how common it is used by other applications. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: [RELEASE] curl 7.65.2
Hey, I'm proposing a 7.65.3 release within shortly due to this: https://github.com/curl/curl/issues/4122 I seem to have broken the progress meter output! :-/ -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Request: Make HTTP2 library version available via API
On Thu, 18 Jul 2019, Mac-Fly wrote: I understand the point of you question. Hence I am running 2 Open-Source projects myself (one is quite large) and hardly find the time for requests in their feature and bug trackers. If by any chance possible, I would prefer that a member of the CURL team might take the action and I don't need to work myself into the internals of CURL. You might know that situation too well, too. :-) Absolutely! Here's my PR adding nghttp2 version info: https://github.com/curl/curl/pull/4121 -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Request: Make HTTP2 library version available via API
On Wed, 17 Jul 2019, Mac-Fly via curl-library wrote: thanks for asking. Yes - its for better representation in an UI where the long string that curl_version() returns is not well suited. (Imagine a list/table together with other libs.) I could RexEx this string though. But as the other libs are available in the version struct I thought nghttp2 might just have been missed. We could easily add it to the curl_version_info_data struct in a coming version. Feel like working on that? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
[RELEASE] curl 7.65.2
ost on github, aasivov on github, AceCrow on Github, Alessandro Ghedini, Alex Grebenschikov, Ben Voris, Bjarni Ingi Gislason, Brian Carpenter, Caleb Raitto, Christopher Head, Cliff Crosland, Daniel Gustafsson, Daniel Stenberg, Gaël Portay, Gergely Nagy, Giorgos Oikonomou, Gisle Vanem, Jakub Zakrzewski, James Brown, Jan Chren, jonrumsey on github, Juergen Hoetzel, Koen Dergent, Kunal Ekawde, Kyle Edwards, Linos Giannopoulos, Marcel Raad, Michael Brehm, Michael Wallner, MrSorcus on github, Niklas Hambüchen, Patrick Monnerat, pendrek at hackerone, Peter Simonyi, Ray Satiro, Richard Alcock, Rich Mirch, Shankar Jadhavar, sstruchtrup on github, Stefano Simonelli, Stephan Szabo, Steve Holme, Thomas Gamper, Tom van der Woerdt, Vasily Lobaskin, Viktor Szakats, (46 contributors) Thanks! (and sorry if I forgot to mention someone) References to bug reports and discussions on issues: [1] = https://curl.haxx.se/bug/?i=3993 [2] = https://curl.haxx.se/bug/?i=3986 [3] = https://curl.haxx.se/bug/?i=4000 [4] = https://curl.haxx.se/bug/?i=3994 [5] = https://curl.haxx.se/bug/?i=3999 [6] = https://curl.haxx.se/bug/?i=3975 [7] = https://curl.haxx.se/bug/?i=4009 [8] = https://curl.haxx.se/bug/?i=4006 [9] = https://curl.haxx.se/mail/lib-2019-06/0019.html [10] = https://curl.haxx.se/bug/?i=3984 [11] = https://curl.haxx.se/bug/?i=3903 [12] = https://curl.haxx.se/bug/?i=3833 [13] = https://curl.haxx.se/bug/?i=4027 [14] = https://curl.haxx.se/bug/?i=4021 [15] = https://curl.haxx.se/bug/?i=4020 [16] = https://curl.haxx.se/bug/?i=4017 [17] = https://curl.haxx.se/bug/?i=4012 [18] = https://curl.haxx.se/bug/?i=4013 [19] = https://curl.haxx.se/bug/?i=4010 [20] = https://curl.haxx.se/bug/?i=4033 [21] = https://curl.haxx.se/bug/?i=4031 [22] = https://curl.haxx.se/bug/?i=4034 [23] = https://curl.haxx.se/bug/?i=3950 [24] = https://curl.haxx.se/bug/?i=4029 [25] = https://curl.haxx.se/bug/?i=3706 [26] = https://curl.haxx.se/bug/?i=4035 [27] = https://curl.haxx.se/bug/?i=4037 [28] = https://curl.haxx.se/bug/?i=4036 [29] = https://curl.haxx.se/bug/?i=4023 [30] = https://curl.haxx.se/bug/?i=4048 [31] = https://curl.haxx.se/bug/?i=4042 [32] = https://curl.haxx.se/bug/?i=4055 [33] = https://curl.haxx.se/bug/?i=4050 [34] = https://curl.haxx.se/bug/?i=4051 [35] = https://curl.haxx.se/bug/?i=4070 [36] = https://curl.haxx.se/bug/?i=4068 [37] = https://curl.haxx.se/bug/?i=4066 [38] = https://curl.haxx.se/bug/?i=4061 [39] = https://curl.haxx.se/bug/?i=4065 [40] = https://curl.haxx.se/bug/?i=4075 [41] = https://curl.haxx.se/bug/?i=4076 [42] = https://curl.haxx.se/bug/?i=4072 [43] = https://curl.haxx.se/bug/?i=4073 [44] = https://curl.haxx.se/bug/?i=4079 [45] = https://curl.haxx.se/bug/?i=4086 [46] = https://curl.haxx.se/bug/?i=4090 [47] = https://curl.haxx.se/bug/?i=4087 [48] = https://curl.haxx.se/bug/?i=4053 [49] = https://curl.haxx.se/bug/?i=4054 [50] = https://curl.haxx.se/bug/?i=4046 [51] = https://curl.haxx.se/bug/?i=4094 [52] = https://curl.haxx.se/bug/?i=4100 [53] = https://curl.haxx.se/bug/?i=4103 [54] = https://curl.haxx.se/bug/?i=4095 [55] = https://curl.haxx.se/bug/?i=4107 [56] = https://github.com/curl/curl/commit/48b9ea4#commitcomment-34084597 [57] = https://curl.haxx.se/bug/?i=4096 [58] = https://curl.haxx.se/bug/?i=4101 [59] = https://curl.haxx.se/bug/?i=4102 [60] = https://curl.haxx.se/bug/?i=4099 [61] = https://curl.haxx.se/bug/?i=4117 [62] = https://curl.haxx.se/bug/?i=3814 [63] = https://curl.haxx.se/bug/?i=4118 [64] = https://curl.haxx.se/bug/?i=3389 [65] = https://curl.haxx.se/bug/?i=4110 [66] = https://curl.haxx.se/bug/?i=4097 -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: How to do optimal FTP upload for multiple files?
On Tue, 16 Jul 2019, Taras Kushnir via curl-library wrote: The reason why I’m asking this is the following: I got reports from users that my app “uploads very slowly” so I started this investigation. Default upload code heavily (like x3-x4 on my machine) underperforms FileZilla upload and commandline “ftp" utility even for 1 file to 1 server (that eliminates my theoretical threading implementation bugs etc.). I've not seen/heard this before. What libcurl version on what platform? Have you seen this "slowness" on more than one version/platform? Is this against any or particular servers? When you do these "slow" uploads, what's the RTT to server, roughly? If, when doing FTP upload for 1 file for 1 server, I’m setting CURLOPT_UPLOAD_BUFFERSIZE, upload speed increases for some users of my app to “almost satisfactory”. When you set it to what value? What does "almost satisfactory" mean in terms how it compares to other tools then? However, same setting cause FTP response timeouts on _every_ transfer for other users (who, if I understand correctly, have much lower bandwidth). That sounds really weird and would rather imply a (separate) bug. On my machine (macbook pro 2015) and bandwidth I feel almost no difference when playing with UPLOAD_BUFFERSIZE. In my own tests, UPLOAD_BUFFERSIZE mostly make a difference when you have near infinite bandwidth, like uploading on localhost. (And when when using SFTP, but I assume we're not talking about that here.) Therefore comes a question: is there any idiomatic way of using libcurl for such cases that will solve upload speed issues for users both with big and small bandwidth? Increasing buffer size shouldn't hurt transfer speeds for any users! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Regarding Async DNS resolver
On Tue, 16 Jul 2019, Amit wrote: I don't actually advocate using 100ms all the time. It needs to be shorter at first and then increase over time. curl_multi_timeout() return such timeouts. Thanks, will use curl_multi_timeout() instead of fixed 100 msec timer. Just note that curl_multi_timeout() only started to return those suitable timeouts during the threaded name resolver in 7.60.0, so in earlier versions you need to do that yourself. why do you think notifying DNS-completed in resolver thread is a bad idea ? Because it would be surprising to applications (since no callbacks in libcurl is ever called from another thread) and introduce the typical threading issues requiring mutexes, locks etc and risk causing race conditions in the app. Anyways, application would be handling that DNS-completed event in CURL thread context. If you call libcurl from thread A and the callback comes from thread B (that curl itself created), then surely those are not using the same thread contexts? The *better* fix would be to instead use a pipe/sockpair to signal that the name resolve is complete and then have the application be able to wait on that. Are you suggesting that let CURL internally create pipe/socket pair for DNS resolution and application wait on the same fd ? I'm suggesting that would be one way to offer a solution. And on DNS resolution, CURL will write to the pipe, which will cause application thread to wake up and application can trigger the CURL to resume the connection ? Correct. Regarding your concern on Windows, I am not sure about the CURL development process, but since I am working on linux, would like to know if It if okay to implement it only for linux ? If you are fine, then I can explore more in this direction. Sure, a solution that only works for non-Windows is totally fine, as Windows users could then either just remain the current method or a Windows developer could work on providing the corresponding solution for that platform. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Regarding Async DNS resolver
On Tue, 16 Jul 2019, Amit wrote: 1> During testing, I have seen that DNS is getting resolved in 40 msec but request is going out of the box only in the next poll iteration. Since there are other high priority tasks running in the system, sometime this 100 msec timer event is getting processed with delay which is adding additional 100 - 200 msec delay and impacting the UI (user interface) experience. Right, and I don't actually advocate using 100ms all the time. It needs to be shorter at first and then increase over time. curl_multi_timeout() return such timeouts. 2. The another issue with 100 msec polling is that my application will keep on polling every 100 msec and the thread in which CURL is running at MEDIUM priority, this would create issue for other low priority threads. Really? When curl ends up not doing anything because there's nothing to do yet, it takes a fraction of a millisecond on most hardware. It seems unlikely to have a measurable impact. IMO it would be better to wake up the thread only when DNS is resolved rather than polling every 100 msec. Agreed, that is much better. But in this case it is rather difficult. The name resolving is done in a separate thread and there's no signalling done when the resolve phase is completed - exactly because it doesn't use any socket/file descriptor internally. libcurl itself polls that information when the resolver thread is used. I think a DNS-completed callback from the resolver-thread would be a bad idea. The *better* fix would be to instead use a pipe/sockpair to signal that the name resolve is complete and then have the application be able to wait on that. But that's problematic on Windows. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Regarding Async DNS resolver
On Mon, 15 Jul 2019, Amit via curl-library wrote: I would like to seek your opinion/thoughts if it is good idea to re-use the existing callback to notify the client about DNS resolution ?. I don't think that sounds like a good idea, no. To me it sounds like you're looking to patch libcurl rather than to improve your own application logic. Can't you just fix your one-second wait to avoid this problem? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Async thoughts
Hi team, Recently I've talked to some users who, independently of each other, have brought up or touched the idea of (easier) async transfers with libcurl. The easy interface is fine and yeah, easy, but synchronous, and the multi interface is powerful and non-blocking and all but not always very easy to "just do async" with. Is there a way and a "need" for easier async with libcurl? If so, how should the API for this work? Is there an existing async API somewhere that's good or useful that we can learn from or be inspired by? If your dream async API came true, how would it work? The probably most basic way I can think of is firing off a separate thread with the "start transfer" call and then have have a way to probe how the transfer is going and finally a way to signal the completion of the transfer. Thoughts? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Proposal: Removal of CURLPROTO_GOPHER from default redir_protocols
On Thu, 4 Jul 2019, Linos Giannopoulos via curl-library wrote: Therefore, I would propose to remove CURLPROTO_GOPHER from the default value of CURLOPT_REDIR_PROTOCOLS P.S: The same holds for other protocols such as CURLPROTO_DICT I approve. But how about instead discussing what protocols we should keep allowing by default? Is anyone ever using redirects (and expecting them to work) to anything but HTTP, HTTPS and FTP ? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: segfault in libcurl during install
On Fri, 28 Jun 2019, Chris Murphy via curl-library wrote: Upstream bug tracking this problem, the developers say the segfault is happening in libcurl and to ask about it here. https://github.com/flatpak/flatpak/issues/2977 Downstream bug contains gdb backtrace that I can't parse, but it's the 2nd attachment in this bug report https://bugzilla.redhat.com/show_bug.cgi?id=1723242 As always, we would much appreciate if we would have a way to reproduce that doesn't involve a lot of extra layers of software. If you can do that, and this still happens with current git master (as I think this problem *might* have been fixed there) then please file an issue on github! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: bind address lockup
On Thu, 27 Jun 2019, Robert NEMKIN wrote: Hope I'm right, but bind() is for server/receiving sockets. I want to bind to a specific interface as client socket. It's the same bind() call. No setsockopt for this. I know, that setsockopt is thread safe according to the POSIX standard. But is the linux kernel (4.19.37-5) implementation correct? This isn't using setsockopt, this is bind. Sometimes it took more than a minute to start the transfer, sometimes it never starts. That certainly indicates something odd! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: bind address lockup
On Thu, 27 Jun 2019, Robert NEMKIN via curl-library wrote: I don't know if setsockopt bind address is thread-safe? Assuming you speak of Linux, I certainly presume bind() is thread-safe. But your way of saying this makes me curious. Why do you call it setsockopt bind address? Where exactly does it hang? And for how long? Maybe I can use some workaround? E.g. download a little testfile sequential to do the bindaddress serialized with keepalive, then start the multithreaded download? You could add one transfer at a time and only start the next once the previous has started to get data as then you know they don't do bind() at the same time. But I wouldn't feel entirely safe with that since then we don't know for sure where the problem is or that this is actually a working fix or just accidentally working... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: curl, governance, money and the future
On Fri, 11 Jan 2019, Daniel Stenberg via curl-library wrote: Back in January we got this response from the Software Freedom Conservancy: Conservancy's Evaluation Committee really likes curl and would like us to move forward. Time passed and in spite of several new prope emails from me, we got no responses back until now. They now say: the earliest we could work on bringing in a new project is September. I'm disappointed by their silence and slowness and I consider this a not so subtle hint that the SFC is not for us. I will not push forward on this. We already have open collective as a good way to receive donations and hold funds for us (https://opencollective.com/curl). In the "governance" suggestion document from October 2018 [*] we proposed forming a "steering committe", a "core team", if you want so that decisions about money and "larger" issues in the project could be handled by someone more than just me but still perhaps a smaller team than the entire curl-library list, and I still think that's a good idea. I'll probably get back to this topic later this summer (as I'm just about to take off on vacation now). [*] = https://curl.haxx.se/mail/lib-2018-10/att-0033/curl__governance__money_and_the_f uture.pdf -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Error 58 with CURLOPT_SSLCERT
On Tue, 25 Jun 2019, Praveen Pvs wrote: Wanted to open a socket using COM library and pass that FD to CURL for communication pass so that COM library that we have takes care of the loading keys/cert and does the handshakes. ... Now i am seeing that, CURL also trying to do handshake and it is failing. If you're using a HTTPS URL, libcurl will still know it is TLS and it will of course do a TLS handshake. Make it a HTTP URL if you'd rather skip the TLS part. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Error 58 with CURLOPT_SSLCERT
On Fri, 21 Jun 2019, Praveen Pvs wrote: using which TLS backend? It would be auto negotiated. TLS1_2 support is there I meant backend as in TLS library. Based on your error texts I assume OpenSSL. But based on your error message: Curl Error Buffer: unable to set private key file: '/mnt/flash/system/rkeys/usr1/key0.der' type DER We can tell that this is the code that runs: https://github.com/curl/curl/blob/1e9769639b57ffd4f587d39efafaae6ff1d75c4a/lib/vtls/openssl.c#L852-L858 ... so clearly OpenSSL doesn't approve of that key file. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
curl: Windows OpenSSL engine code injection
Windows OpenSSL engine code injection = Project curl Security Advisory, June 24th 2019 - [Permalink](https://curl.haxx.se/docs/CVE-2019-5443.html) VULNERABILITY - A non-privileged user or program can put code and a config file in a known non-privileged path (under `C:/usr/local/`) that will make curl automatically run the code (as an openssl "engine") on invocation. If that curl is invoked by a privileged user it can do anything it wants. This flaw exists in the official curl-for-windows binaries built and hosted by the curl project (all versions up to and including 7.65.1_1). It **does not** exist in the curl executable shipped by Microsoft, bundled with Windows 10. It possibly exists in other curl builds for Windows too that uses OpenSSL. The curl project has provided official curl executable builds for Windows since [late August 2018](https://daniel.haxx.se/blog/2018/08/27/blessed-curl-builds-for-windows/). There exists proof of concept exploits of this flaw. INFO This bug sneaked in partly due to insecure default build options in OpenSSL when built cross-compiled and partly due to a misleading commit message in the curl commit that made it possible to disable this feature. This bug does not exist in the curl or libcurl source code but in the scripts for the Windows build. The Common Vulnerabilities and Exposures (CVE) project has assigned the name CVE-2019-5443 to this issue. CWE-94: Code Injection Severity: High AFFECTED VERSIONS - - Affected versions: all curl-for-windows downloads before **7.65.1_2**. THE SOLUTION Replace your downloaded curl version on Windows with the updated download package from the [curl site](https://curl.haxx.se/windows/). The build fix for curl-for-win correcting this flaw is in [this commit](https://github.com/curl/curl-for-win/commit/51b658a76594942cf1d6f227d8fc4732bb8ec277). It completely disables curl's ability to load an OpenSSL config when invoked. RECOMMENDATIONS -- We suggest you take one of the following actions immediately, in order of preference: A - Upgrade to a fixed curl executable B - Remove curl executable downloaded from curl.haxx.se and instead use the one shipped by Microsoft in Windows 10 TIMELINE The issue was reported to the curl project on June 12, 2019. The fix was done, verified and communicated with the reporter on June 12, 2019. While planning the release schedule of this advisory and coordinating with other affected projects, we discovered that this exact flaw had already been published and discussed in public before we were informed about it. A few other OpenSSL-using projects on Windows also had already fixed their builds for this exact problem. Realizing this, we switched gears and decided to publish as soon as possible to minimize user impact. curl 7.65.1_2 for Windows was uploaded and made available on June 21 2019 - the older, vulnerable builds, were removed from the site at the same time. This advisory was posted on June 24th 2019. CREDITS --- Reported by Rich Mirch. OpenSSL patch by Viktor Szakats. Thanks a lot! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Questions about split client/host in runtests
On Fri, 21 Jun 2019, Stephan Szabo via curl-library wrote: We're starting to maintain an internal cross-compiled port for libcurl and we've started getting tests running using the split client/host support in runtests, but we had a question about the feature checking done by the script. Cool! I'm sure you might end up needing to fix the scripts a bit to get that going. When checking for features, is the intent that these are checked on the client or that the client and the host have the same feature set available? Okay with this terminology: client - the machine that runs curl host - the machine that runs the test suite ... the script should only care about the features of the client's curl installation. I don't think the host should even have to have curl installed. It seems like curl -version is run on the client, but server/disabled and curl -M (for manual support check) is currently run on the host. I'd say that's a result of us basically never running it on host/client like this so these mistakes can easily slip in without us noticing! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Error 58 with CURLOPT_SSLCERT
On Thu, 20 Jun 2019, Praveen Pvs via curl-library wrote: I am experiencing error code 58 when i try to set the SSL certificate in my code. Which libcurl version? using which TLS backend? on what platform? Did you try converting and using the key in PEM format?à -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Using the X.509 certificates of the Windows certificate store via libcurl
On Tue, 18 Jun 2019, * * via curl-library wrote: Sending an HTTPS GET request with that code produces error CURLE_PEER_FAILED_VERIFICATION (60) with the error message "Cert verify failed: BADCERT_NOT_TRUSTED". So, obviously the certificates are not properly taken into account during the TLS handshake. I don't know a lot about Windows and extracting the CA store on Windows, but I think this problem happens because you haven't actually got the full cert store in that function. At least David Weisgerber's code from two years ago seems to do more: https://curl.haxx.se/mail/meet-2017-03/0030.html (That example is for OpenSSL but I think the Windows parts should be similar.) -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: multi threaded curl segmentation fault
On Tue, 18 Jun 2019, Siarhei Siniak wrote: Yeah, vcpkg is using 1.0.2 openssl: ... Daniel, coud you please send me the link about these openssl requriements? https://curl.haxx.se/libcurl/c/threadsafe.html is our main docs for what to think about to use libcurl multi-threaded. The OpenSSL docs concering this for 1.0.2 seems to have been removed from openssl.org so those links on that page are now dead! The link to the example setup might be the most useful there: https://curl.haxx.se/libcurl/c/opensslthreadlock.html -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: multi threaded curl segmentation fault
On Tue, 18 Jun 2019, Siarhei Siniak via curl-library wrote: I'm experiencing a rare segmentation fault within libcurl method. A stack trace is as follows: #0 0x7f6f80308b98 in getrn () at lib.so #1 0x7f6f80308ffd in lh_delete () at lib.so #2 0x7f6f8030a0b1 in int_thread_del_item () at lib.so #3 0x7f6f8030b130 in ERR_remove_thread_state () at lib.so #4 0x7f6f80288876 in Curl_close () at lib.so #5 0x7f6f8026177d in curl_easy_cleanup () at lib.so The stack trace varies, but is within Curl_close(). Is it always in OpenSSL? What OpenSSL version? If before 1.1, do you have the necessary OpenSSL mutex callbacks setup? curl:x64-linux 7.61.1-2 -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Nmake install prefix
On Sat, 15 Jun 2019, Thomas Gamper via curl-library wrote: As I was working on a script to automate the nmake based build of libcurl, I noticed that the installation directory for the built files is hardwired. Would the attached patch, that allows one to supply a install prefix, be considered a worthwile addition to the nmake build system? I don't use this build method myself but this patch seems like small and simple enough that I think we should merge it. Any Windows developer who would object? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Stuck in poll on network disconnect
On Fri, 14 Jun 2019, Magdy, Marco wrote: I've created a PR [2] that fixes the problem and passes the unit tests. I noticed you also closed it again, but the reasoning surprised me. - you withdrew it after you noticed the speedcheck call in the TOOFAST state, but in your use case surely your transfer is never in TOOFAST? - isn't your transfer in the PERFORM state when it gets stuck? If so, the Curl_speedcheck call is done in Curl_readwrite() so it *should* already detect the situation... (but since you see this issue, something must be wrong) -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Stuck in poll on network disconnect
On Thu, 13 Jun 2019, Magdy, Marco wrote: I'm a little confused. The commit SHA(s) you referenced above show disabling poll on MacOS. And the problem does _not_ happen on MacOS. Sorry, I was confused and mixed up your issue with another one when I responded! So, it seems that poll is the problem. Is there a way I can turn it off to test that theory? I really don't think poll is the problem, poll on linux works rock solid and has done so for decades. No, it rather seems that the Curl_speedcheck() function isn't doing what it's supposed to do in your case but I have no idea why... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Stuck in poll on network disconnect
On Thu, 13 Jun 2019, Magdy, Marco via curl-library wrote: timeout to 0 and relying on the low-speed-time and low-speed-limit options to determine if a connection has stalled. ... However, we're encountering a problem (that happens 50-ish% of the time) when the connection is lost; we see libcurl stuck with the following stack trace: poll 0x7f2441f4e74d This is curious. Since a34c7ce754 curl should build without using poll and instead use select if built with configure (and since 6aa9cfa2b also in cmake). poll on macOS has a long and sad story so everyone is typically better off avoiding it there. But since you say you can reproduce on Linux too, I doubt that's the reason. So, is this a bug? What is your guidance otherwise? It certainly sounds like bug, but I don't recognize it as something we've fixed. Please file on issue on github and tell us how we can proceed to reproduce it! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: How to run test 307?
On Wed, 12 Jun 2019, Marcel Raad via curl-library wrote: I've never seen a test run with test 307 enabled. The precheck checks for "openssl" in "curl -s --engine list". I don't think I've seen it run for a long time either! The precheck is there because the test cases uses that openssl engine so it if isn't there, there's no point in trying the test ase. And the avilable dynamic engines are always "capi" and "padlock". I haven't found any info about the "openssl" engine. Maybe that one isn't around anymore in current openssl versions? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: SMTP server response timeout
On Wed, 12 Jun 2019, Christian Schmitz via curl-library wrote: One of the issues I see with 7.65.x is the server response timeouts we see with SMTP: ... Has anyone seen that? I have not. How can we reproduce? If there's a way, please submit it as an issue on github so that we don't lose it! The only recent timeout related change I can think of is d93405 [1], but that just reverted the situation back to how things worked before 7.64.1. [1] = https://github.com/curl/curl/commit/d934059afd8568201d0646d6bfd201f39b6bc412 -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: no feature window yet
On Fri, 7 Jun 2019, Daniel Stenberg via curl-library wrote: I'm inclined to just not allow any features at all in this cycle. The segfaults should/might be fixed now. Still, let's keep the feature window closed for this cycle to really make sure we stop the boat from rocking and get back to good old boring stability before we open up the flood gates again. Next curl version will therefore be named 7.65.2 and ship on July 17. I strongly encourage everyone who experiences problems with 7.65.0 or 7.65.1 to check the current master to verify if the problems are gone there or persist - so that we can work on fixing them before next release! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: SMTP RFC 1830
On Wed, 5 Jun 2019, Alan Jones via curl-library wrote: How can I use the cURL SMTP interface in conjunction with the RFC1830 extension? I.e. be able to send large binary MIME messages introduced with the BDAT keyword in place of DATA? libcurl's SMPT code uses 'DATA' only. But if this BDAT thing is in use and is standardized, I can't think of any reason why we couldn't add support for it if there's use case! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Clarification on support for CURLOPT_CAINFO
On Wed, 5 Jun 2019, Richard Alcock via curl-library wrote: CURLOPT_CAINFO option is ignored. Refer to https://curl.haxx.se/docs/ssl-compared.html I think this reference is unfortunate and should be removed. This man page should be stand-alone and contain all the necessary information by itself. If I'm understanding them correctly, these two sources contradict each other, CURLOPT_CAINFO is supported for Schannel and Secure Transport, and it isn't ignored. It isn't the preferred/recommended method for those back-ends, but it is expected to work. I think that's correct. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: the curl user survey 2019 analysis
On Wed, 5 Jun 2019, Kamil Dudka via curl-library wrote: That is not how I understood the question. I thought that YES was reserved for people who maintain some publicly available CI infrastructure for upstream curl, something on https://curl.haxx.se/dev/builds.html or Github-based CI. That's exactly how I intended the question... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: the curl user survey 2019 analysis
On Wed, 5 Jun 2019, Jeffrey Walton wrote: For next year's survey I would be interested to know what percentage of the folks who build cURL from sources run the self tests before installation (re: Building curl). It is merely a a curiosity of mine. I can already guess: a very tiny portion of users. For several reasons: 1. Most users don't build curl at all 2. Many users opt to not run the tests 3. Many users build on systems that can't run the test suite 4. Many users build with build systems that don't offer tests I personally don't think of this as a very big problem. I view the tests as something mostly oriented for us who develop curl to detect problems *before* they are released. The amount of issues we get reported by users who get test failures on their own with a release version is minuscule. And out of those we get reported, it usually turns out to be a problem with the tests rather than with curl... :-/ But it might also show a gap in build procedures. If you find too few folks are performing the 'make check' step, then it might be wise to have the 'make install' recipe run it for them. While I sympathize with the idea, I don't think that's the right way either. Most users who build curl from source build it and use it on a platform that's been tested a million times already. Running the same tests on the same platforms a million extra times is just a waste of time and resources. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
[RELEASE] curl 7.65.1
Hello team! I'm happy to announce another curl release. This time we ship a bunch of bug-fixes and we call the relase 7.65.1. As always, download it from: https://curl.haxx.se/ curl and libcurl 7.65.1 Public curl releases: 182 Command line options: 221 curl_easy_setopt() options: 268 Public functions in libcurl: 80 Contributors: 1965 This release includes the following bugfixes: o CURLOPT_LOW_SPEED_* repaired [6] o NTLM: reset proxy "multipass" state when CONNECT request is done [32] o PolarSSL: deprecate support step 1. Removed from configure [3] o appveyor: add Visual Studio solution build [25] o cmake: check for if_nametoindex() [2] o cmake: support CMAKE_OSX_ARCHITECTURES when detecting SIZEOF variables [17] o config-win32: add support for if_nametoindex and getsockname [8] o conncache: Remove the DEBUGASSERT on length check [27] o conncache: make "bundles" per host name when doing proxy tunnels [21] o curl-win32.h: Enable Unix Domain Sockets based on the Windows SDK version [16] o curl_share_setopt.3: improve wording [34] o dump-header.d: spell out that no headers == empty file [30] o example/http2-download: fix format specifier [4] o examples: cleanups and compiler warning fixes [4] o http2: Stop drain from being permanently set [26] o http: don't parse body-related headers in bodyless responses [28] o md4: build correctly with openssl without MD4 [9] o md4: include the mbedtls config.h to get the MD4 info [10] o multi: track users of a socket better [15] o nss: allow to specify TLS 1.3 ciphers if supported by NSS [20] o parse_proxy: make sure portptr is initialized [23] o parse_proxy: use the IPv6 zone id if given [1] o sectransp: handle errSSLPeerAuthCompleted from SSLRead() [13] o singlesocket: use separate variable for inner loop [31] o ssl: Update outdated "openssl-only" comments for supported backends [33] o tests: add HAProxy keywords [18] o tests: add support to test against OpenSSH for Windows [5] o tests: make test 1420 and 1406 work with rtsp-disabled libcurl [19] o tls13-docs: mention it is only for OpenSSL >= 1.1.1 [29] o tool_parse_cfg: Avoid 2 fopen() for WIN32 o tool_setopt: for builds with disabled-proxy, skip all proxy setopts() [12] o url: Load if_nametoindex() dynamically from iphlpapi.dll on Windows [24] o url: fix bad feature-disable #ifdef [7] o url: use correct port in ConnectionExists() [22] o winbuild: Use two space indentation [11] This release includes the following known bugs: o see docs/KNOWN_BUGS (https://curl.haxx.se/docs/knownbugs.html) This release would not have looked like this without help, code, reports and advice from friends like these: Benbuck Nason, Carlos ORyan, Daniel Stenberg, Dave Reisner, dbrowndan on github, dkwolfe4 on github, Edmond Yu, elsamuko on github, Eric Wu, Frank Gevaerts, Gisle Vanem, Hubert Kario, Jonas Vautherin, Josie Huddleston, Kunal Ekawde, Maksim Stsepanenka, Marcel Raad, Marc Hörsken, Michael Kaufmann, Patrick Monnerat, Ray Satiro, Sergey Ogryzkov, smuellerDD on github, Steve Holme, Tom Greenslade, Tom van der Woerdt, wesinator on github, (27 contributors) Thanks! (and sorry if I forgot to mention someone) References to bug reports and discussions on issues: [1] = https://curl.haxx.se/bug/?i=3482 [2] = https://curl.haxx.se/bug/?i=3917 [3] = https://curl.haxx.se/bug/?i=3888 [4] = https://curl.haxx.se/bug/?i=3919 [5] = https://curl.haxx.se/bug/?i=3290 [6] = https://curl.haxx.se/bug/?i=3927 [7] = https://curl.haxx.se/bug/?i=3924 [8] = https://curl.haxx.se/bug/?i=3923 [9] = https://curl.haxx.se/bug/?i=3921 [10] = https://curl.haxx.se/bug/?i=3922 [11] = https://curl.haxx.se/bug/?i=3930 [12] = https://curl.haxx.se/bug/?i=3926 [13] = https://curl.haxx.se/bug/?i=3932 [14] = https://curl.haxx.se/bug/?i=3653 [15] = https://curl.haxx.se/bug/?i=3952 [16] = https://curl.haxx.se/bug/?i=3939 [17] = https://curl.haxx.se/bug/?i=3945 [18] = https://curl.haxx.se/bug/?i=3949 [19] = https://curl.haxx.se/bug/?i=3948 [20] = https://curl.haxx.se/bug/?i=3916 [21] = https://curl.haxx.se/bug/?i=3951 [22] = https://curl.haxx.se/bug/?i=3956 [23] = https://curl.haxx.se/bug/?i=3959 [24] = https://curl.haxx.se/bug/?i=3960 [25] = https://curl.haxx.se/bug/?i=3941 [26] = https://curl.haxx.se/bug/?i=3966 [27] = https://curl.haxx.se/bug/?i=3962 [28] = https://curl.haxx.se/bug/?i=3968 [29] = https://curl.haxx.se/bug/?i=3938 [30] = https://curl.haxx.se/bug/?i=3964 [31] = https://curl.haxx.se/bug/?i=3970 [32] = https://curl.haxx.se/bug/?i=3972 [33] = https://curl.haxx.se/bug/?i=3985 [34] = https://curl.haxx.se/mail/lib-2019-06/0009.html -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--
Re: CURLSHOPT_LOCKFUNC callback requirements.
On Tue, 4 Jun 2019, Carlos ORyan via curl-library wrote: I guess the bottom line is: should I use different locks for different kinds of data in the same CURLSH* handle? Yes. Each kind should have its own lock since they will be locked/unlocked totally independently of each other. And if so, may I suggest that the documentation could be improved to say: ... When using multiple kinds of data in a CURLSH you must make sure that your callback uses a different lock for each kind of data. With that constraint in mind, use a single lock a time for each CURLSH and kind of data. Thanks. How about this further edited version? The data argument tells what kind of data libcurl wants to lock. Make sure that the callback uses a different lock for each kind of data. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
the curl user survey 2019 analysis
Hej! I've finally managed to sum up what 732 users told us in this year's survey: https://daniel.haxx.se/blog/2019/06/04/curl-user-survey-2019-analysis/ -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: sendto with EBADF
On Tue, 4 Jun 2019, Kunal Ekawde via curl-library wrote: I used curl-master as of 3rd June (wanted to verify a bug #3904), although now original core dump is not seen, after few transfers application is not processing new messages and seem to be taking 100% CPU. I need to check if its libcurl issue or application issue. ... Any idea when such case can occur ? Nope, I've never seen it happen. If you can create small example code that can reproduce the problem then please file an issue and attach the code and I'm sure we can look into it! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: libcurl DNS resolver does it support DNS srv queries to fetch the port number.
On Mon, 3 Jun 2019, Badari Prasad via curl-library wrote: Does libcurl DNS resolver (native DNS resolver, not using c-ares) support fetching port numbers for a domain using srv query ? libcurl doesn't support SRV records (or URI records) at all, no matter which resolver is used. The feature is mentioned in the TODO: https://curl.haxx.se/docs/todo.html#SRV_and_URI_DNS_records -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: TLS v1.2 support in libcurl
On Fri, 31 May 2019, ashish yadav via curl-library wrote: I am looking for version of libcurl which support TLS v1.2. Could anyone please help me in that ? I'm pretty sure all libcurl versions released the last 10 years or so do as long as you build it with a TLS library that supports it. Since 7.34.0, libcurl can be told to use TLS 1.2 explictly. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: connection pooling with http/2 when multiplexing to remote http1.1/http2 server
On Wed, 29 May 2019, Stefan Eissing via curl-library wrote: Libcurl version: 7.52.1-5+deb9u9 + some patches. I think there have been improvements since then. A curl dev might know. I just want to underscore that Stefan is entirely correct here. We've done numerous fixes to HTTP/2 and multiplexing handling since that version so a modern libcurl version might very well perform better. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: CURLPIPE_MULTIPLEX option for http1.1 and 2.0
On Sat, 1 Jun 2019, Kunal Ekawde via curl-library wrote: If we have both HTTP/1.1 and HTTP/2.0 transfers happening over same multi handle (can it not ?) , Yes it can. can we safely set CURLPIPE_MULTIPLEX for CURLMOPT_PIPELINING. Yes you can. We don't want to have HTTP1.1 pipelining but need HTTP/2 multiplexing.Since multi handle is same, wont setting the CURLPIPE_MULTIPLEX option impact both HTTP1.1 & HTTP/2 transfers ? There's no multiplexing in HTTP/1 so it could only do multiplexing for those requests and connections that negotiate HTTP/2. But yes, it is a boolean switch that applies the all handles/transfers done with that multi handle. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: delayed patch release
On Wed, 29 May 2019, Daniel Stenberg via curl-library wrote: I'll assess the situation and get back with a new release date. New release date for 7.65.1 is Wednesday June 5. That's a full week later than first planned but gives us time to not rock the boat immediately before the release. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Fwd: ESNI initiative
On Wed, 29 May 2019, Niall O'Reilly via curl-library wrote: I'm not sure whether posting to this list is the right thing to do; apologies if not. Seems like the exact right thing to do! The DEfO project (https://defo.ie) has work in progress to integrate prototype OpenSSL support for ESNI into a number of applications. Not surprisingly, curl is high on our list. We're still exploring how the curl application and library are structured and built. One of us will say more when we have something presentable. Awesome! And do feel free to share your code as early as you think is convenient and useful as usually the earlier we can feed back comments on design and think, the better it tends to get. Additionally, you might also get help from other fellow contributors who want to see ESNI support done. Following the existing pattern for the `ALTSVC` feature, these might be (for example), ``` #define CURL_VERSION_ESNI (1<<25) /* ESNI support */ ``` and `USE_ESNI`. If this is premature, or doesn't make sense, I'ld appreciate being told. Feedback is not necessarily confirmation. It looks perfect to me. Matches how we usually do these things! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
delayed patch release
Hey, I was about to release 7.65.1 this morning, but I won't. We had too many changes land just in the last hours and we've discovered a new segfault[3961] in the CI builds that we need to have under control first before I can do a release I can firmly stand behind. 3961 = https://github.com/curl/curl/issues/3961 I'll assess the situation and get back with a new release date. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
We offer commercial curl support
Hello cURL community! (consider this a one-time mail only, I will not repeat this on these mailing lists again) As I am now part of the wolfSSL family (which we all are thrilled about!), we at wolfSSL would like to open our arms to the entire cURL community and let you know that commercial support from basic up to full 24x7x365 is now available for cURL! Whatever you, your company or your engineering team need help with to get the full capacity out of curl, we can help you. We also recently released TinycURL for small embedded systems, showing that curl can serve your use cases even there! The regular open source support and availability of curl will not go away. The commercial support wolfSSL offers is beyond that. It is dedicated, non-disclosed with guaranteed delivery times etc for you and your company to rely on. Additionally, team wolfSSL is ready to support cURL users with new features and performance optimizations. Get in touch to discuss today, at supp...@wolfssl.com Available support packages: https://www.wolfssl.com/products/support-and-maintenance/ We love you! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
bug-fixes galore, no feature-window yet
Hi, I just want to prematurely say that we DO NOT open the feature window tomorrow when I will upload 7.65.1. The amount of bug-fixes we have managed to merge just these last 6 days since 7.65.0 is huge (22 and counting). We need to hold off and see that we don't get any new alarming bugs filed for a few days until we open for features again. I'll get back with updated info on this on Monday the 3rd of June. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Tests 1455 and 1456 failing with --disable-proxy
On Mon, 27 May 2019, Marcel Raad via curl-library wrote: the HAProxy tests 1455 and 1456 have been failing with proxy support disbled since commit https://github.com/curl/curl/commit/e91e48161235272ff485ff32bd048c53af731f43#diff-72ee0eef31d053c9e6d4fb22e81d2407R1700. Unfortunately, they are only labeled as "HTTP" and "HTTP GET", which I don't want to disable globally in my HTTP-only autobuilds Should we add the existing "proxy" label to them, a new one, or both? I think they should indeed have the 'proxy' keyword set, but why not also add a 'haproxy' one so that it gets easier to test or exclude just those... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: A quick follow-up release next week
On Sat, 25 May 2019, Daniel Gustafsson wrote: I favor option D (with the commit message clearly stating that the revert isn’t due to the commit in itself, but due to proess). Thanks Ray and Daniel. I've now reverted the SASL authzid commits by merging Ray's commit for it. We'll get them back in again after the 7.65.1 release. To take a play from another project I’m involved in, another way to deal with this would be to branch off every release with a standardized name Yeah, that's one way. I have however tried to avoing creating branches for release works mostly because we're a small team and I rather want us as a team work on the next release, whichever it is, rather than to see us split up and have some go merge in new features in the master branch while some others work on fixing up a release in another branch. But I also don't want to get stuck in a particular workflow just because that's the way've done it before... Maybe we need some sort of automated.. matrix? .. of sorts which maps the options we have in CI and what we have in autobuilds, to be able to find blindspots? That'd be really useful. I'm not sure how we would go ahead to create one though! And also, some of the problems this time were because people do build combinations with don't build in the CI builds plus and using TLS libraries built with non-default build options which we don't test with either. We also have to accept the fact that we offer way many more build combinations than we can ever test so we need to be clever about it. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
[Final reminder] The curl user survey 2019
Hi team, The curl user survey will be taken offline at the Euro midnight tomorrow. Please give us your input before then! https://daniel.haxx.se/blog/2019/05/13/the-curl-user-survey-2019/ This is the primary way for us to get user input and feedback about what you features you are actually using in curl, what we should do next and what works and doesn't work in the project! We've already received more than 700 responses, which is more than last year. Totally awesome! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: A quick follow-up release next week
On Sat, 25 May 2019, Jeffrey Walton wrote: 7.65.1 seems more natural to me. But I appreciate the need to adhere to policies and procedures. Are you allowed to label it "experimental" to sidestep the min version bump? Yes, good point, I figure that could also be an option. But it also feels a little like a "hack" just to circumvent our own rules. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: A quick follow-up release next week
On Sat, 25 May 2019, Jeffrey Walton wrote: We already merged support for CURLOPT_SASL_AUTHZID (and the accompanying --sasl-authzid), more or less accidentally. Forgive my ignorance... Did this break an ABI? Or did it break something that already exists? Or did it add additional functionality? Or something else? It it's a new feature, or a "change" as I sometimes call it, as opposed to just a bug-fix. According to our version numbering "rules" [1], we bump the release number for the next release if that happens. That would mean 7.66.0. If we revert this change, we only have bug-fixes merged since 7.65.0 and we would call the next release 7.65.1. [1] = https://curl.haxx.se/docs/versions.html -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: A quick follow-up release next week
On Sat, 25 May 2019, Ray Satiro wrote: I prefer either of A) branch off and revert; or D) revert in master. My suggestion is if you are going to revert then roll them up into 1 like I did rather than have 5 reverts. I think it's clearer that way if you're reading the history to see what happened and why. Good advice I think. Let's see if someone else weighs in as well. After your input I think I'm on (D) now to make the git history easier to follow. To be clear I'm not suggesting wait a week before a patch release I'm suggesting wait a week without any major reported problem in the latest release before you open the feature window. Each time a release goes out --including a patch release-- reset again. I hear you. My minor variation of that was to suggest that we can have a week post-release as a rule and allow us to overrule the full week with a public message saying the feature window has opened again - should we feel that need. Should we then extend the feature period one week at the far end so that it remains four weeks or should we narrow it down to 3:5 ? I'm rather indifferent and can think of reasons for either way. For reference I read the last 5 years of release tags [1] and urgent patches have typically been within 5-10 days with just one at 2 days. This is a subject I've brought up on curl up every year so far. The graph I've shown has been "days between curl releases" and the latest version I've shown looks like this: https://daniel.haxx.se/media/days-between-releases-since-2010.png It shows a clear improvement since about a year back as we've done much fewer "panic releases" (follow-up releases within a short period of time from the previos release) recently. I credit this enhancement primarily to our extended use of CI builds which nowadays makes us land much better commits with fewer follow-up needs. It has however been made clear that the CI builds aren't covering all the setups that our user base is using - a number of issues showed up this time around in build combos and setups that we simple didn't test prior to release. More test coverage and more CI builds can help here. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: A quick follow-up release next week
On Fri, 24 May 2019, Ray Satiro via curl-library wrote: See upstream/jay/test [1] for a combined reversion. I also tested reapply with cherry-pick if you want to do it all in master instead (in other words revert then release then cherry pick). Right, doing it all in master is also an option. I take it you favor that approach? I'll admit it is attractive. Which rules make you lean towards C? It felt like it might be the one with the least resistance and work. Also how was adding the SASL authzid support accidental? Because I first didn't clearly say no and I didn't stop it when the first commit went in. Maybe add a week cooling off period after the release before opening the feature window so it can be determined whether an urgent patch release is necessary. Yeah. Or if not a week, we could have the opening of the feature window to be explict so that someone (like me) should announce it as open before we consider it so. Then it could be made a week if we need it, or just three days for the times when we consider that good enough... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Edit cookies easily?
On Wed, 22 May 2019, Scott Ellentuch via curl-library wrote: Would have been nice to manipulate in memory AND write out when closed. :) I might just have to do all cookie handling in program. Not terrible, but just wanted to see if there was an alternative. libcurl offers the CURLINFO_COOKIELIST option to read all the cookies: https://curl.haxx.se/libcurl/c/CURLINFO_COOKIELIST.html ... and the corresponding CURLOPT_COOKIELIST to add or replace existing cookies: https://curl.haxx.se/libcurl/c/CURLOPT_COOKIELIST.html -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
A quick follow-up release next week
Hi, Due to the number of tiny annoying glitches in the 7.65.0 release, we'll ship a follow-up release on Wednesday. Hold off all feature merges until then please. We already merged support for CURLOPT_SASL_AUTHZID (and the accompanying --sasl-authzid), more or less accidentally. Should we: A) branch off in git and release 7.65.1 with those commits reverted B) keep them in master and release 7.65.1 with those included C) keep them in master and call the release 7.66.0 For simplicity and sticking to our own rules, I personally lean towards C. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Clarification in man page for CURLOPT_WRITEFUNCTION for nmemb==0
On Thu, 23 May 2019, Nicolas Roeser via curl-library wrote: | This function may be called with zero bytes data if the transferred | file is empty. Ahh, good to know. But _still_, I am not sure whether the function may be called with 0 bytes _if the transferred file is *not* empty._ I think it doesn't specifically specify that because we haven't wanted to make that promise. Partly because it seems that applications don't really *need* that promise and people have not asked for it. Until now. 3) Depending on the protocol [or something else], this function may or will never be called with zero bytes if the transferred file is not empty. I want to say this as that's basically what is implied now and it makes the least amount of promise and we don't have to change anything... =) This said, I don't think it currently actually ever calls the callback with zero bytes other than the specific use case already mentioned above! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
RE: [RELEASE] curl 7.65.0
On Wed, 22 May 2019, Daniel Stenberg via curl-library wrote: I suspect one of these options --disable-proxy, --disable-crypto-auth and --disable-cookies is what makes curl set an option that now returns an error instead of being silently accepted and ignored. I could add that since neither of these options are possible to figure out from the outside if they were used in the build, I've not worked on running the tests when they are enabled - which is why this problem slipped through my fingers. And clearly I didn't look at the autobuilds close enough! =( -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
RE: [RELEASE] curl 7.65.0
On Wed, 22 May 2019, Marcel Raad wrote:
I agree. My HTTP-only OpenSSL autobuild (
https://curl.haxx.se/dev/log.cgi?id=20190522044629-2216) is also broken
because of " curl: (48) An unknown option was passed in to libcurl", but I
haven't had time to take a closer look yet.
I suspect one of these options --disable-proxy, --disable-crypto-auth and
--disable-cookies is what makes curl set an option that now returns an error
instead of being silently accepted and ignored.
If you want a blunt tool to figure out which option curl sets that libcurl
doesn't like, here's a debug patch to get you started:
index 745b4546e..e1a10efe9 100644
--- a/src/tool_setopt.c
+++ b/src/tool_setopt.c
@@ -630,10 +630,11 @@ CURLcode tool_setopt(CURL *curl, bool str, struct
GlobalConfig *config,
char *escaped = NULL;
CURLcode ret = CURLE_OK;
va_start(arg, tag);
+ fprintf(stderr, "OPT: %s\n", name);
if(tag < CURLOPTTYPE_OBJECTPOINT) {
/* Value is expected to be a long */
long lval = va_arg(arg, long);
long defval = 0L;
const NameValue *nv = NULL;
--
/ daniel.haxx.se | Get the best commercial curl support there is - from me
| Private help, bug fixes, support, ports, new features
| https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: [RELEASE] curl 7.65.0
Hi, We've already seen a few bugs and pull-requests popping in to fix quirks in the 7.65.0 release. Due to this, I think we should consider doing a follow-up patch release within a week or so where these are addressed. This shouldn't warrant any need to move any other release dates or planning. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
[SECURITY ADVISORY] curl: TFTP receive buffer overflow
TFTP receive buffer overflow Project curl Security Advisory, May 22nd 2019 - [Permalink](https://curl.haxx.se/docs/CVE-2019-5436.html) VULNERABILITY - libcurl contains a heap buffer overflow in the function (`tftp_receive_packet()`) that recevives data from a TFTP server. It calls `recvfrom()` with the default size for the buffer rather than with the size that was used to allocate it. Thus, the content that might overwrite the heap memory is entirely controlled by the server. The flaw exists if the user selects to use a "blksize" of 504 or smaller (default is 512). The smaller size that is used, the larger the possible overflow becomes. Users chosing a smaller size than default should be rare as the primary use case for changing the size is to make it larger. It is rare for users to use TFTP across the Internet. It is most commonly used within local networks. We are not aware of any exploit of this flaw. INFO This bug was introduced in January 2009 in [commit 0516ce7786e95](https://github.com/curl/curl/commit/0516ce7786e95). The Common Vulnerabilities and Exposures (CVE) project has assigned the name CVE-2019-5436 to this issue. CWE-122: Heap-based Buffer Overflow Severity: 1.8 (Low) AFFECTED VERSIONS - - Affected versions: libcurl 7.19.4 to and including 7.64.1 - Not affected versions: libcurl < 7.19.4 and >= libcurl 7.65.0 libcurl is used by many applications, but not always advertised as such. THE SOLUTION A [fix for CVE-2019-5436](https://github.com/curl/curl/commit/2576003415625d7b5f0e390902f8097830b82275) RECOMMENDATIONS -- We suggest you take one of the following actions immediately, in order of preference: A - Upgrade curl to version 7.65.0 B - Apply the patch to your version and rebuild C - do not use TFTP with curl TIMELINE The issue was reported to the curl project on April 29, 2019. The patch was communicated to the reporter on April 29, 2019. We contacted distros@openwall on May 15. curl 7.65.0 was released on May 22 2019, coordinated with the publication of this advisory. CREDITS --- Reported by l00p3r. Patch by Daniel Stenberg Thanks a lot! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
[SECURITY ADVISORY] curl: Integer overflows in curl_url_set
Integer overflows in `curl_url_set()` = Project curl Security Advisory, May 22nd 2019 - [Permalink](https://curl.haxx.se/docs/CVE-2019-5435.html) VULNERABILITY - libcurl contains two integer overflows in the `curl_url_set()` function that if triggered, can lead to a too small buffer allocation and a subsequent heap buffer overflow. The flaws only exist on 32 bit architectures and require excessive string input lengths. We are not aware of any exploit of this flaw. INFO There are two entry points to this issue, on 32 bit architectures. By asking libcurl to parse a string, passing in a string longer than 2GB to this API: `curl_url_set(uh, CURLUPART_URL, "string", 0);` triggers the bug. Asking libcurl to update a URL with a new string, and URL encoded it in the process, by passing in a string longer than 1.33GB to this API: `curl_url_set(uh, CURLUPART_*, "string", CURLU_URLENCODE);` triggers the bug. This bug was introduced in August 2018 in [commit fb30ac5a2d](https://github.com/curl/curl/commit/fb30ac5a2d63773c52). The Common Vulnerabilities and Exposures (CVE) project has assigned the name CVE-2019-5435 to this issue. CWE-131: Incorrect Calculation of Buffer Size Severity: 3.7 (Low) AFFECTED VERSIONS - - Affected versions: libcurl 7.62.0 to and including 7.64.1 - Not affected versions: libcurl < 7.62.0 and >= libcurl 7.65.0 libcurl is used by many applications, but not always advertised as such. THE SOLUTION A [fix for CVE-2019-5435](https://github.com/curl/curl/commit/5fc28510a4664f4) is already merged. RECOMMENDATIONS -- We suggest you take one of the following actions immediately, in order of preference: A - Upgrade curl to version 7.65.0 B - Apply the patch to your version and rebuild TIMELINE The issue was reported to the curl project on April 24, 2019. The patch was communicated to the reporter on April 25, 2019. We contacted distros@openwall on May 15. curl 7.65.0 was released on May 22 2019, coordinated with the publication of this advisory. CREDITS --- Reported by Wenchao Li. Patch by Daniel Stenberg Thanks a lot! -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
[RELEASE] curl 7.65.0
y unused code [22] o progress: acknowledge CURL_DISABLE_PROGRESS_METER [78] o proxy: acknowledge DISABLE_PROXY more o resolve: apply Happy Eyeballs philosophy to parallel c-ares queries [3] o revert "multi: support verbose conncache closure handle" [69] o sasl: Don't send authcid as authzid for the PLAIN mechanism as per RFC 4616 o sasl: only enable if there's a protocol enabled using it o scripts: fix typos o singleipconnect: show port in the verbose "Trying ..." message o smtp: fix compiler warning [15] o socks5: user name and passwords must be shorter than 256 [8] o socks: fix error message o socksd: new SOCKS 4+5 server for tests [31] o spnego_gssapi: fix return code on gss_init_sec_context() failure [53] o ssh-libssh: remove unused variable [83] o ssh: define USE_SSH if SSH is enabled (any backend) [57] o ssh: move variable declaration to where it's used [83] o test1002: correct the name o test2100: Fix typos in test description o tests/server/util: fix Windows Unicode build [21] o tests: Run global cleanup at end of tests [29] o tests: make Impacket (SMB server) Python 3 compatible [11] o tool_cb_wrt: fix bad-function-cast warning [5] o tool_formparse: remove redundant assignment [83] o tool_help: Warn if curl and libcurl versions do not match [28] o tool_help: include for strcasecmp [4] o transfer: fix LGTM alert "Comparison is always true" [14] o travis: add an osx http-only build [80] o travis: allow builds on branches named "ci" o travis: install dependencies only when needed [24] o travis: update some builds do Xenial [30] o travis: updated mesalink builds [35] o url: always clone the CUROPT_CURLU handle [26] o url: convert the zone id from a IPv6 URL to correct scope id [89] o urlapi: add CURLUPART_ZONEID to set and get [59] o urlapi: increase supported scheme length to 40 bytes [84] o urlapi: require a non-zero host name length when parsing URL [73] o urlapi: stricter CURLUPART_PORT parsing [33] o urlapi: strip off zone id from numerical IPv6 addresses [49] o urlapi: urlencode characters above 0x7f correctly [9] o vauth/cleartext: update the PLAIN login to match RFC 4616 [27] o vauth/oauth2: Fix OAUTHBEARER token generation [6] o vauth: Fix incorrect function description for Curl_auth_user_contains_domain [68] o vtls: fix potential ssl_buffer stack overflow [76] o wildcard: disable from build when FTP isn't present o winbuild: Support MultiSSL builds [34] o xattr: skip unittest on unsupported platforms [20] This release includes the following known bugs: o see docs/KNOWN_BUGS (https://curl.haxx.se/docs/knownbugs.html) This release would not have looked like this without help, code, reports and advice from friends like these: Aron Bergman, Brad Spencer, cclauss on github, Dan Fandrich, Daniel Gustafsson, Daniel Stenberg, Eli Schwartz, Even Rouault, Frank Gevaerts, Gisle Vanem, GitYuanQu on github, Guy Poizat, Isaiah Norton, Jakub Zakrzewski, Jan Ehrhardt, Jeroen Ooms, Jonathan Cardoso Machado, Jonathan Moerman, Joombalaya on github, Kamil Dudka, Kristoffer Gleditsch, l00p3r on hackerone, Leonardo Taccari, Marcel Raad, Mert Yazıcıoğlu, nevv on HackerOne/curl, niner on github, Olen Andoni, Omar Ramadan, Paolo Mossino, Patrick Monnerat, Po-Chuan Hsieh, Poul T Lomholt, Ray Satiro, Reed Loden, Ricardo Gomes, Ricky Leverence, Rikard Falkeborn, Roy Bellingan, Simon Warta, Steve Holme, Taiyu Len, Tim Rühsen, Tom van der Woerdt, Tseng Jun, Viktor Szakats, Wenchao Li, Wyatt O'Day, XmiliaH on github, Yiming Jing, (50 contributors) Thanks! (and sorry if I forgot to mention someone) References to bug reports and discussions on issues: [1] = https://curl.haxx.se/bug/?i=3709 [2] = https://curl.haxx.se/bug/?i=3707 [3] = https://curl.haxx.se/bug/?i=3699 [4] = https://curl.haxx.se/bug/?i=3715 [5] = https://curl.haxx.se/bug/?i=3718 [6] = https://curl.haxx.se/bug/?i=2487 [7] = https://curl.haxx.se/bug/?i=3724 [8] = https://curl.haxx.se/bug/?i=3737 [9] = https://curl.haxx.se/bug/?i=3741 [10] = https://curl.haxx.se/bug/?i=3651 [11] = https://curl.haxx.se/bug/?i=3731 [12] = https://curl.haxx.se/bug/?i=3736 [13] = https://curl.haxx.se/bug/?i=3723 [14] = https://curl.haxx.se/bug/?i=3732 [15] = https://curl.haxx.se/bug/?i=3729 [16] = https://curl.haxx.se/bug/?i=3720 [17] = https://curl.haxx.se/bug/?i=3738 [18] = https://curl.haxx.se/bug/?i=3744 [19] = https://curl.haxx.se/bug/?i=3743 [20] = https://curl.haxx.se/bug/?i=3759 [21] = https://curl.haxx.se/bug/?i=3758 [22] = https://curl.haxx.se/bug/?i=3739 [23] = https://curl.haxx.se/bug/?i=3725 [24] = https://curl.haxx.se/bug/?i=3721 [25] = https://curl.haxx.se/bug/?i=3654 [26] = https://curl.haxx.se/bug/?i=3753 [27] = https://curl.haxx.se/bug/?i=3757 [28] = https://curl.haxx.se/bug/?i=3774 [29] = https://curl.haxx.se/bug/?i=3783 [30] = https://curl.haxx.se/bug/?i=3777 [31] = https://curl.haxx.se/bug/?i=3752 [32] = http
The release comes tomorrow
Hi, We're just finalizing the last few things on the release and it is scheduled to ship tomorrow. I will live-stream this event on twitch starting 08:30 (UTC+2) and go over the news and some of the bug-fixes I think are most noteworthy. Also a good time for me to address your questions if you have such. Streamed here: https://www.twitch.tv/curlhacker Older live-streams are archived and available after the fact: https://www.youtube.com/user/danielhaxxse/ -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
[Reminder] The curl user survey 2019
On Mon, 13 May 2019, Daniel Stenberg wrote: https://daniel.haxx.se/blog/2019/05/13/the-curl-user-survey-2019/ We're approaching 400 responses. Please consider giving us YOUR point of view as well. The survey will be up for another week. This is the primary way for us to get user input and feedback about what you features you are actually using in curl, what we should do next and what works and doesn't work in the project! In 2018, we got 670 respones during the two week survey period. Can we beat that this year? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: SIZEOF_LONG undefined when cross-compiling for iOS
On Fri, 17 May 2019, Jonas Vautherin via curl-library wrote: I have been trying to cross-compile curl for iOS, using leetal's toolchain [1] (and another, similar one that has the same issue). When building for the "Xcode" cmake generator (cmake -G Xcode <...>), curl build successfully. But when building for the "Unix Makefiles" generator, I get an error saying that SIZEOF_LONG is undefined. Seems lame. The curl cmake file has always checked that size since the first day we had one. Although I don't know how good cmake is at doing that for cross-compiles... I see that SIZEOF_LONG is defined in curl_config.h.cmake [2], but I don't really get where it comes from. In cmake builds, here: https://github.com/curl/curl/blob/f3e0f071b14fcb46a453f69bdf4e062bcaacf362/CMakeLists.txt#L798 (and obviouly similarly by configure, but that's not the topic here) Should I set it somewhere manually? You should at least set it somewhere. Exactly what the best way is for you to set it, I leave to you. And if yes, it is always right to say that it should be "8" on 64-bits archs, and "4" on 32-bits archs (as described here [3])? All 64 bit iOS targets will have it at 8 as far as I know, yes. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Curl_socket_check is getting stuck for more then 2 min in poll
On Wed, 15 May 2019, ravi prakash Bajpai via curl-library wrote: In Curl_socket_check we are passing *timeout_ms* value in argument . In what cases it can wait for more then 2 min. What do you do to make this happen? On what platform? Which libcurl version and what libssh2 version? #1 0x7f27302fb43f in *Curl_socket_check *() from /lib64/libcurl.so.4 #2 0x7f273030382e in ssh_easy_statemach.constprop.1 () from /lib64/libcurl.so.4 This stack trace you show reveals a function name that that was renamed in February 2013 (commit c25383ae1), in the 7.30.0 release, so you're obviously using something older. That means you're using a curl that is more than six years old. Presumably you're then also using a fairly old SSH library. I think a sensible first approach would be to upgrade both, if even in a test environment, and see if the problem is still there. Because if its not, I don't think you'll find us very interested in hunting for this issue... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Nominating polarssl for deprecation
On Wed, 15 May 2019, Daniel Gustafsson via curl-library wrote: Agreed, +1 on marking as deprecated. I note there are Daniel x 3 in favor! =) I filed a PR for step 1: https://github.com/curl/curl/pull/3888 -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Libcurl.so Lib File
On Wed, 15 May 2019, Iftekhar via curl-library wrote: I have a Linux based machine and want to use https in that machine, I do app development for that machine on windows using eclips c/c++. I am looking for header and library file(.SO) to support http and https. It will be a great help to me if you can share these files to me if you have, or you can guide me how can I get it. Iftekhar, you should preferably use your Linux distros preferred way of installing it. Not download it on the side from somewhere else. Your Linux distribution has a package manager, search for "libcurl-dev" or "libcurl devel" in there and install that package. On Redhat/centos you can install with yum. With Debian/Ubuntu to install with apt/apt-get. On other distros there are other tools. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Nominating polarssl for deprecation
Hi, The polarssl TLS library has not had an update in over three years. The last release was done on January 7 2019 [1]. That library has been superceded by the mbedtls library, which is the current incarnation of it. In curl we support both, with two separate implementations. I propose that we move to DEPRECATE PolarSSL support in curl because it seems unlikely that this library is a good choice for users to get proper TLS security and support today and at the same time there are plenty of good and updated alternatives. I consider it likely that the existing users of curl + polarssl out there are stuck on old curl versions and when they eventually manage to update curl they should also be able to update their TLS library. If nobody objects, I will file a PR soon that makes the polarssl disabled in code but still present, and mark it for complete removal six months after the release where it is marked as deprecated. Possibly then marked deprecated in the July 17 2019 release, removed from the code after January 17 2020. Opinions? [1] = https://tls.mbed.org/tech-updates/releases -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: crash seen in multi_socket
On Sun, 12 May 2019, Kunal Ekawde wrote: Ok, I shall try to update to 7.64.1 but that would take sometime to integrate, sanitize and load run. Note: I don't think upgrading will fix this particular issue. but using this patch resulted in trap #2. Based on trap #2 backtrace, I didn't understand that backtrace. Any chance you can rebuild your libcurl and keep the debug symbols so that we can see exactly from what source lines the functions are called? Maybe even stop there with gdb and do 'bt full'! Why exactly is it crashing there? Is that a NULL pointer dereference or something? I've update the patch to 'return result;' //CURLM_OK instead of continue. I m yet to verify the minimal load run with the server abruptly terminating case, which most likely resulted in trap #2. Do you think 'result result;' is not correct there ? It is not correct. We need the rest of the logic in the function to run. One thing missing there would be updating running_handles. For example, yes, but we also need to take care of the timers that might have expired. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: cURL Asan testing and LIB ordering
On Mon, 13 May 2019, Ray Satiro via curl-library wrote:
COMMON="-fsanitize=address,undefined -fno-sanitize-recover
-fno-omit-frame-pointer -Wformat -Werror=format-security
-Werror=array-bounds -g"
CPPFLAGS="${CPPFLAGS:-}" \
CFLAGS="${CFLAGS:-} $COMMON" \
CXXFLAGS="${CXXFLAGS:-} $COMMON" \
LDFLAGS="${LDFLAGS:-} $COMMON -Wl,-rpath,/usr/local/ssl/lib
-Wl,-rpath,/usr/local/lib" \
./configure --enable-debug \
--with-ssl=/usr/local/ssl \
"$WITH_NGHTTP2" \
"$@"
And here's how I mostly do it when running my sanitizer builds:
CC=clang-8
CFLAGS="-fsanitize=address,undefined,signed-integer-overflow
-fno-sanitize-recover=undefined,integer -Wformat -Werror=format-security
-Werror=array-bounds -g"
CXXFLAGS="-fsanitize=address,undefined -fno-sanitize-recover=undefined,integer
-Wformat -Werror=format-security -Werror=array-bounds -g"
LDFLAGS="-fsanitize=address,undefined -fno-sanitize-recover=undefined,integer"
LIBS="-ldl -lubsan"
./configure --disable-shared --enable-debug
--enable-maintainer-mode --enable-ipv6 --enable-ares=$HOME/src/c-ares
--with-gssapi --enable-werror --with-ntlm-auth
--with-libmetalink=/home/daniel/build-libmetalink-bzr
--with-nghttp2=$HOME/build-nghttp2 --prefix=$HOME/test-curl-install
--without-libssh2 #--with-libssh2=$HOME/src/libssh2
--
/ daniel.haxx.se | Get the best commercial curl support there is - from me
| Private help, bug fixes, support, ports, new features
| https://www.wolfssl.com/contact/---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: The curl user survey 2019
On Mon, 13 May 2019, Daniel Stenberg via curl-library wrote: (and yes, I will repeat this request a few times over the next two years during which the survey will be up) Hm no, hehe, I meant two *weeks*... -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
The curl user survey 2019
Hi all, Please take a few minutes off your busy schedule and tell us how you use curl, what you're missing in curl and what we should add to curl in the future! You find the link to the survey here: https://daniel.haxx.se/blog/2019/05/13/the-curl-user-survey-2019/ Thanks! (and yes, I will repeat this request a few times over the next two years during which the survey will be up) -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: crash seen in multi_socket
On Sat, 11 May 2019, Kunal Ekawde via curl-library wrote:
I'm using libcurl - 7.64.0 with nghttp2 for http2 call flow. For http/1.1
this crash is not seen.
I recommend using 7.64.1!
I tried with following fix:
/* the socket can be shared by many transfers, iterate */
for(e = list->head; e; e = e->next) {
data = (struct Curl_easy *)e->ptr;
// crash fix - temp
if (!data)
continue;
I think this should do a 'break' instead of a 'continue'. After all, this
happens because the connection died so there is no more transfers on the same
connection.
While this would just be defensive fix, the root cause fix could be
elsewhere also, request to please check / comment.
This is a bit tricky to reproduce so I haven't managed to do this myself yet.
I'm just reading your description and the code and thinking.
If you add the break like this in there, does this fix the problem for you?
Does it have any other flaws? I'm thinking we might want to terminate those
remaining transfers on the same socket in a more immediate way...
What do you do to reproduce this case?
--
/ daniel.haxx.se | Get the best commercial curl support there is - from me
| Private help, bug fixes, support, ports, new features
| https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
soon time for the annual user survey
Hi team, I just want to point out that I'll soon publish this year's edition of the curl user survey. I'm curious if anyone has any thoughts on questions or data that we should add or change this year? The analysis of the 2018 edition can be found here: https://daniel.haxx.se/blog/2018/06/12/curl-survey-2018-analysis/ I've modified some questions ever so slightly so far, so if nothing major happens the 2019 poll will look very similar to the 2018 one. -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Possibly higher bug bounties
Hi, I just wanted to highlight the fact that we just merged updated documentation and updated the information on the hackerone page [1] about our bug bounty. Thanks to Dropbox, we can now offer up to 32,000 USD in reward money if you manage to find a security issue in curl that is graded high or critcal and it matches some further critiera. See [1]. Happy hunting! [1] = https://hackerone.com/curl -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: CURLOPT_VERBOSE and curl_multi_cleanup()
On Wed, 8 May 2019, Kristoffer Gleditsch (kgledits) via curl-library wrote: After https://github.com/curl/curl/pull/3618 and https://github.com/curl/curl/pull/3598, we started getting '* Closing connection 0' messages on STDERR when curl_multi_cleanup() closes the connections in the connection cache. This is because the verbose flag is copied to the closure_handle but fdebug etc. are not. The reason for this is pretty well explained in https://github.com/curl/curl/pull/3598#issuecomment-466301175 , but the result is that we have no way controlling where the verbose output from the closure_handle goes. One workaround is to add and remove a dummy easy_handle with CURLOPT_VERBOSE=0 before doing curl_multi_cleanup(); this will at least silence the output on STDERR. But should there be a proper interface for controlling this? Thanks Kristoffer for clearly explaining the issue. I agree that we need to do something about this. The concept with an internal "close handle" is a bit unfortunate since it has its own set of options and we've opted to let it "inherit" some of the options from the most recently added handle. It's not documented properly anywhere as it was meant to just be an internal design descision that shouldn't affect users, but I think in reality it does. We decided to not copy the debug callback to the close handle since it could come as a surprise to application that it could be called even after the handle in which it was set had been killed. I fear that would be even worse than what we have right now. One *possible* fix to this would be to not to set the VERBOSE bit if the debugcallback is set. Not sure that goes far enough. Another option is to revert that change and go back to the previous behavior, and then add a second bit to the CURLOPT_VERBOSE argument that then could set VERBOSE to the close handle if set... What do you think would be a fair fix for your case? -- / daniel.haxx.se | Get the best commercial curl support there is - from me | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: php curl insecure connection option is skipped
On Wed, 8 May 2019, surya chandrika via curl-library wrote: But am not sure why certificate with correct hostname is not recognized from uploaded certificate. The problem is that you connect to hostname XYZ but the certificate provided by this XYZ doesn't list that name as one of names the certificate is valid for. -- / daniel.haxx.se --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Change Curl_is_absolute_url to allow URI without authority
On Sat, 4 May 2019, Aron Bergman wrote: But why? Can you show us what benefit this brings to anyone? As stated in the first mail, this will make it possible to use protocols which doesn’t use the authority component. I don’t think that any of the protocols that cURL currently implements will have any use of this change. I see it as more of a preemptive change for future protocol implementations. More specifically I’m thinking of the DNS protocol. I'm not a believer in such "preemptive" changes. If the change is good for something that might happen in the future, then I think we should wait with doing it until that possible future event occurs. If you want to implement support in libcurl for a protocol that has a URL scheme for which you need to change the code, then I'm willing to discuss it. And then I think such a discussion should first be about whether we think that particular protocol is a good fit for curl or not! I decided to try to implement the DNS protocol into cURL I'm not convinced DNS is a good fit for curl. If it is, I think we need a slightly larger code than than your proposed change to properly handle that URL syntax (reading RFC 4501). -- / daniel.haxx.se--- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: Ranged PUTs, Content-Range, and Content-Length
On Thu, 2 May 2019, Christopher Head via curl-library wrote: CURLOPT_RANGE specifically as far as I know is only supposed to be used for download, not upload (can anyone confirm?) And yet, looking at the source code, there are explicit conditionals to check for CURLOPT_RANGE and turn it into a Content-Range header in an upload. As I mentioned in the issue [3814], curl does support ranged PUT requests since basically the beginning of time and there are even tests that verify this. 3814 = https://github.com/curl/curl/issues/3814 -- / daniel.haxx.se --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
