Re: Windows users! Help us test upload performance tuning?

2018-08-14 Thread Daniel Stenberg via curl-library
On Tue, 14 Aug 2018, Jan Ehrhardt via curl-library wrote: I did not test if there is a difference on *nix. Did you? Here are the results my tests run just now. Using Linux kernel 4.17. Upload 4GB over plain HTTP to Apache 2.4.34 on localhost - so really 0 RTT. I ran "time curl -sT 4GB

Re: a URL API ?

2018-08-14 Thread Daniel Stenberg via curl-library
On Mon, 13 Aug 2018, Dan Fandrich via curl-library wrote: I'm not sure I see the difference between these two approaches. Can you show them with some example URLs? For example, + and ! are reserved characters in RFC 3986 but unreserved in RFC 2326 (RTSP), so a generic canonicalization might

Re: Windows users! Help us test upload performance tuning?

2018-08-14 Thread Daniel Stenberg via curl-library
On Tue, 14 Aug 2018, Jan Ehrhardt wrote: my tests indicated that increasing this value also influenced the FTP upload speeds. From 10 to 5 seconds on XP in Daniel Jelinski's testcurl uploads. And down to less than a second on Win 7 and Win 10. Right. So yes, there's certainly a valid reason

Re: Windows users! Help us test upload performance tuning?

2018-08-13 Thread Daniel Stenberg via curl-library
On Mon, 13 Aug 2018, Jan Ehrhardt via curl-library wrote: Daniel Stenberg (Thu, 9 Aug 2018 16:55:38 +0200 (CEST)): -#define UPLOAD_BUFSIZE CURL_MAX_WRITE_SIZE +#define UPLOAD_BUFSIZE (512*1024) @Daniel Stenberg: is there a reason that is stopping you from changing this in curl? Yes. (Pretty

Re: Retrieve all addresses mapped to specific host, not just one IP

2018-08-13 Thread Daniel Stenberg via curl-library
On Mon, 13 Aug 2018, myLC--- via curl-library wrote: With a callback you'd make sure that the user takes care of it when the data becomes available in that format - and only then. Yes, but since we need to copy the entire data to the export struct, keeping that memory around a little longer

h2c, fcurl and curl-cheat-sheet

2018-08-13 Thread Daniel Stenberg via curl-library
Hi, The three github respositories named in the subject are now living in the curl organization instead of under me personally on github. It should make them more easily discoverable and accessible. I figure they're better handled by the curl community at large. They're all ready to

Re: Retrieve all addresses mapped to specific host, not just one IP

2018-08-13 Thread Daniel Stenberg via curl-library
On Mon, 13 Aug 2018, myLC--- via curl-library wrote: I think most existing users would presume such data to become available via curl_easy_getinfo(), like the primary IP and friends already are. I'm obvious not that acquainted with curl. I took a glance at the source. Unless I'm mistaken,

Re: a URL API ?

2018-08-13 Thread Daniel Stenberg via curl-library
On Mon, 13 Aug 2018, Dan Fandrich via curl-library wrote: I think you're right, it should work. Documenting (CURLU_URLDECODE|CURLU_URLENCODE) as performing canonicalization is probably all you'd need, besides ensuring decode and encode happen in the correct order. We could perhaps even make

Re: Retrieve all addresses mapped to specific host, not just one IP

2018-08-12 Thread Daniel Stenberg via curl-library
On Sun, 12 Aug 2018, myLC--- via curl-library wrote: However, I wouldn't be opposed somehow exposing the whole list if someone would like to work on it. If this happened through an optional callback after the hostname has been resolved, you wouldn't have to "expose internals", nor would you

Re: a URL API ?

2018-08-12 Thread Daniel Stenberg via curl-library
On Sun, 12 Aug 2018, Ray Satiro via curl-library wrote: I think you overdid it (I know, I'm a little behind on this discussion [1]), a struct would be simpler. curl_url_parse(url, ); curl_url_build(parts, ); Is there really a need for a more expansive API? I don't think exposing such a

Re: a URL API ?

2018-08-12 Thread Daniel Stenberg via curl-library
On Sun, 12 Aug 2018, Dan Fandrich via curl-library wrote: I'm curious whether the API can be used to canonicalize a URL, i.e., URL decode parts that can be decoded without semantic difference but canonicalize those parts that the specs say must be encoded. I think it should be possible The

Re: break out of the curl_easy_perform loop

2018-08-11 Thread Daniel Stenberg via curl-library
On Sat, 11 Aug 2018, Deven via curl-library wrote: Now, I would like to cleanly close the application which uses the curl easy interface. The curl operation happens in a separate thread. Is there a way to cancel the curl operation, break out of the blocking curl easy perform function? This

Re: Windows users! Help us test upload performance tuning?

2018-08-10 Thread Daniel Stenberg via curl-library
On Sat, 11 Aug 2018, Daniel Jeliński wrote: I like that; if we had the sftp_write function acknowledge data as soon as it is put in socket buffer, we could get much faster transfers. In order to avoid errors we would need to wait for all outstanding acks on file close. Something like that,

Re: Windows users! Help us test upload performance tuning?

2018-08-10 Thread Daniel Stenberg via curl-library
On Fri, 10 Aug 2018, Daniel Jeliński via curl-library wrote: TCP window is not a problem, and SSH window doesn't seem to be the problem either. There is definitely still some room for improvement. Absolutely. For example, one fundamental piece that is a rather tough obstacle is the fact that

Re: CURLINFO_SIZE_UPLOAD may not measure correctly when Expect header removed

2018-08-10 Thread Daniel Stenberg via curl-library
On Tue, 7 Aug 2018, cjmsoregan via curl-library wrote: I have run into an issue in libcurl where the uploaded size obtained via CURLINFO_SIZE_UPLOAD is not measured correctly if the Expect header is removed. Thank you! I could reproduce the issue using your code. I first submitted it as a

Re: Issues uploading file to NextCloud server

2018-08-09 Thread Daniel Stenberg via curl-library
On Mon, 6 Aug 2018, Andy Pont wrote: The debug output on the console when I run the script manually returns with "HTTP/1.1 400 Bad Request”. The logging in the Nextcloud instance shows the request being submitted but fails with the error “BadRequest: expected file size X got 0”. I

Re: Windows users! Help us test upload performance tuning?

2018-08-09 Thread Daniel Stenberg via curl-library
On Thu, 9 Aug 2018, Jan Ehrhardt via curl-library wrote: curl plain ftp patched 41 seconds curl patched sftp 1925 seconds Oh what a sad number there... =( A quick little experiment could be to try upping the upload buffer size significantly: diff --git a/lib/urldata.h

Re: a URL API ?

2018-08-09 Thread Daniel Stenberg via curl-library
On Thu, 9 Aug 2018, Daniel Stenberg via curl-library wrote: (replying to myself...9 /* append to query, ask for encoding */ curl_url_set(h, CURLUPART_QUERY, "company=AT", CURLU_APPENDQUERY| - CURLU_URLENCODE with CURLU_APPENDQUERY set, will skip the '=' letter when doing th

Re: a URL API ?

2018-08-09 Thread Daniel Stenberg via curl-library
On Thu, 9 Aug 2018, Daniel Jeliński via curl-library wrote: char *append = "=44"; Well assuming we want to use the API to build URL based on HTML form with GET action, curl_url_query_append suggested by Geoff would be much nicer. Yes, you're right. I've taken a more generic approach that

Re: Pipelining is a pain, can we ditch it now?

2018-08-09 Thread Daniel Stenberg via curl-library
On Thu, 9 Aug 2018, Radu Hociung via curl-library wrote: I have looked at every one of those reports, and I found no unresolved bugs. Care to list the remaining unaddressed issues? Off the top of my head, here are a few things I think are important to get fixed to increase my

Re: a URL API ?

2018-08-09 Thread Daniel Stenberg via curl-library
On Thu, 2 Aug 2018, Geoff Beier wrote: The setters would be important to us. I might be bikeshedding here, but the ability to add to the query would be very nice. So something like curl_url_query_append(urlp, "numitems", 3) Returning to this, as I've polished the API a bit over the last few

Re: Windows users! Help us test upload performance tuning?

2018-08-09 Thread Daniel Stenberg via curl-library
On Thu, 9 Aug 2018, Daniel Jeliński via curl-library wrote: There's no way for curl/libssh2 to put more than 16 KB on the wire. It's right there in libssh2 docs [1]: [libssh2_sftp_write] will return a positive number as soon as the first packet is acknowledged from the server. That's

Re: Curl closing connection in case of incomplete transfer

2018-08-08 Thread Daniel Stenberg via curl-library
On Wed, 8 Aug 2018, Antoine Magaud via curl-library wrote: I'm kinda new to using libcurl, and am facing an issue which I can't around. I am using curl version 7.41.0. That version is 3.5 years old and has no less than 42 known vulnerabilities! it is related to not reading the whole file,

Re: Application to request http2 content with multiplexed multiple streams

2018-08-08 Thread Daniel Stenberg via curl-library
On Tue, 7 Aug 2018, Rajvardhan Deshmukh wrote: I looks fine, and you can verify yourself by reading the verbose output as it states clearly when it reuses connections. It reuses some connections and most of the times it says " * Connection 75 seems to be dead! * Closing connection 75 " and

Re: Windows users! Help us test upload performance tuning?

2018-08-08 Thread Daniel Stenberg via curl-library
On Wed, 8 Aug 2018, Daniel Stenberg via curl-library wrote: Right, that's a valid remark. But let's address that issue if/when someone actually gets a problem with it... Merged now! -- / daniel.haxx.se --- Unsubscribe: https

Re: Windows and CA certificates

2018-08-08 Thread Daniel Stenberg via curl-library
On Tue, 7 Aug 2018, Daniel Jeliński via curl-library wrote: I recently started using HTTPS functionality with libcurl + openSSL; I noticed that by default this combo does not use Windows certificates, but instead wants to load them from CA bundle. This poses a maintenance problem - the bundle

Re: multi_socket and epoll example

2018-08-08 Thread Daniel Stenberg via curl-library
On Tue, 7 Aug 2018, James Read wrote: As long as libcurl is built to use the threaded resolver (default) or the c-ares backend, it will do name resolving in a non-blocking manner. I just did a standard install using the latest git clone. Then it probably built with the threaded resolver. At

Re: multi_socket and epoll example

2018-08-07 Thread Daniel Stenberg via curl-library
On Tue, 7 Aug 2018, James Read wrote: What about DNS resolution? Is this already asynchronous or do I need to make some changes? As long as libcurl is built to use the threaded resolver (default) or the c-ares backend, it will do name resolving in a non-blocking manner. -- /

Re: Include CA Bundle at Build Time on Windows

2018-08-07 Thread Daniel Stenberg via curl-library
On Tue, 7 Aug 2018, Dillon Korman via curl-library wrote: I do see: --with-ca-bundle=FILE: use the specified file as CA certificate store. CA certificates need to be concatenated in PEM format into this file. But I believe that is for the configure script for nix systems (please correct me

Re: multi_socket and epoll example

2018-08-07 Thread Daniel Stenberg via curl-library
On Tue, 7 Aug 2018, James Read via curl-library wrote: Does anyone have a working example using multi_socket with epoll? Josh Bialkowski provided one not too long ago, available here: https://curl.haxx.se/libcurl/c/ephiperfifo.html -- / daniel.haxx.se

Re: Windows users! Help us test upload performance tuning?

2018-08-07 Thread Daniel Stenberg
On Mon, 6 Aug 2018, Daniel Jeliński wrote: Well I guess the overhead of WSAIoCtl or setsockopt may be non-negligible for fast connections. I can't reproduce this here, can't find a network with low enough bandwidth-delay product. On high BDP the patched version is consistently faster.

Re: a URL API ?

2018-08-06 Thread Daniel Stenberg
On Mon, 6 Aug 2018, Daniel Stenberg wrote: - implement the missing pieces of the API: I've pushed a first PR of the URL API work to make sure all tests and builds are happy: https://github.com/curl/curl/pull/2842 I've listed outstanding work in there. The API works pretty good already

Re: Pipelining is a pain, can we ditch it now?

2018-08-06 Thread Daniel Stenberg
On Mon, 6 Aug 2018, Radu Hociung wrote: We got a fresh new HTTP pipelining bug submitted today [1] I am the submitter of that bug, and now that I understand it better have revised the issue report. It is NOT a pipelining bug, but a plumbing bug. Right, quite ironic I think that it wasn't a

Re: a URL API ?

2018-08-06 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Daniel Stenberg wrote: Good or bad? What would your application need and would this work for that? I've written some initial code for this API [1] now. As I proceed further, I intend to remove the wiki page as I suspect there will be a lot of details that have changed

Re: Windows users! Help us test upload performance tuning?

2018-08-05 Thread Daniel Stenberg
On Sat, 4 Aug 2018, Jan Ehrhardt wrote: curl was consistently a little bit faster than bash/lftp, but that may be related to the sftp encryption (curl ran plain ftp, port 21). curl vanilla and curl patched did not seem to differ. Sometimes patched was faster than vanilla, sometimes the other

Re: Slow Windows uploads (with patch)

2018-08-03 Thread Daniel Stenberg
On Fri, 3 Aug 2018, Thomas Glanzmann wrote: Have you tried to do something like Daniel's upload "auto-tuning" patch but for the download buffers [*]? Or maybe that's not as easily done for downloads... [*] = https://github.com/curl/curl/pull/2762 No, I have not. I just tried to calculate

Re: Windows users! Help us test upload performance tuning?

2018-08-03 Thread Daniel Stenberg
On Fri, 3 Aug 2018, Ray Satiro wrote: 2. Windows Vista SP2, USA, NYC: Microsoft Windows [Version 6.0.6002] generating test file... running vanilla... start:0.484000 total:15.234000 running patched... start:0.453000 total:2.234000 Yikes! At 28 reported numbers, this is the largest improvement

Re: Slow Windows uploads (with patch)

2018-08-03 Thread Daniel Stenberg
On Fri, 3 Aug 2018, Thomas Glanzmann wrote: If you like to run me some more experiements, I can do that. I just compile curl with a higher SO_SNDBUF / SO_RCVBUF, fire up a ec2 AWS instance with windows server 2016 somewhere in the world with a big pipe and high latency and download something

Re: Windows users! Help us test upload performance tuning?

2018-08-03 Thread Daniel Stenberg
On Fri, 3 Aug 2018, Daniel Stenberg wrote: With 14 results so far the patch keeps proving itself a good idea: https://docs.google.com/spreadsheets/d/1xAntVPAggz9gvx7TI_vUF6G6titRAX5tBOZv0JCXkNY/edit?usp=sharing At 21 results, the numbers don't move much. The patched version needs on average

Re: a URL API ?

2018-08-03 Thread Daniel Stenberg
On Fri, 3 Aug 2018, Samuel Hurst wrote: For example, being able to build new URLs from relative ones. I can't quite tell from the examples provided whether curl_url would do relative transformation if the urlhandle is already valid. I can see a use case where I'd want to do the following:

Re: Windows users! Help us test upload performance tuning?

2018-08-03 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Gisle Vanem wrote: But that server is 19 hops away. Testing with the same file via ftp to my server in Denmark (11 hops) shows a bit worse result: running vanilla... start:0,406000 total:1,219000 running patched... start:0,359000 total:1,531000 It would probably be

Re: Windows users! Help us test upload performance tuning?

2018-08-03 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Daniel Stenberg wrote: We want to have more user's experience and results from tests to determine how we should make curl make uploads on windows as fast as possible. I've added all the results received so far into a google docs sheet avialble for viewing on the link

Re: Windows users! Help us test upload performance tuning?

2018-08-02 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Gisle Vanem wrote: But that server is 19 hops away. Testing with the same file via ftp to my server in Denmark (11 hops) shows a bit worse result: running vanilla... start:0,406000 total:1,219000 running patched... start:0,359000 total:1,531000 Seems ftp is maxing my

Re: a URL API ?

2018-08-02 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Geoff Beier wrote: The setters would be important to us. I might be bikeshedding here, but the ability to add to the query would be very nice. So something like curl_url_query_append(urlp, "numitems", 3) That ("numitems", 3) approach is very specific for adding a

Re: --disable-symbol-hiding not working anymore?

2018-08-02 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Dan Fandrich wrote: I found the reason: commit 4b414249 "lib: don't export all symbols, just everything curl_*" This commit completely ignores --disable-symbol-hiding and ends up trying to do what it does but in poorer fashion. Lovely, thanks! Here's my attempt at fixing

Windows users! Help us test upload performance tuning?

2018-08-02 Thread Daniel Stenberg
. -- / daniel.haxx.se -- Forwarded message -- Date: Wed, 1 Aug 2018 16:38:02 From: Daniel Jeliński Reply-To: libcurl development To: libcurl development Subject: Re: Slow Windows uploads (with patch) 2018-07-30 18:09 GMT+02:00 Daniel Stenberg : The tool could try upload with and without

Re: a URL API ?

2018-08-02 Thread Daniel Stenberg
On Wed, 1 Aug 2018, Radu Hociung wrote: a "URL handling" API in libcurl This wheel has already been invented. [1] [1] https://uriparser.github.io/ Are you suggesting this uses an API that would be worth getting inspiration from? Personally I don't see a lot I like there... (Dan already

Re: a URL API ?

2018-08-02 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Dan Fandrich wrote: My first thought it that it's pretty long-winded. I can't think of many situations where you might want only one part of a URL, like the port, but don't want e.g. the host, or the query without the path. What was in the back of my mind for such an API was

Re: Aw: Re: Question about CURLOPT_TLS13_CIPHERS

2018-08-02 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Bernhard Jaeger wrote: > All libcurl options are documented since the version they are introduced! We > even have tests that make sure we can't forget that. As a sidenote CURLOPT_TLS13_CIPHERS was already listed at https://curl.haxx.se/libcurl/c/curl_easy_setopt.html

Re: Question about CURLOPT_TLS13_CIPHERS

2018-08-02 Thread Daniel Stenberg
On Thu, 2 Aug 2018, Bernhard Jaeger wrote: Until libcurl 7.60 the CURLOPT_TLS13_CIPHERS option was necessary to use TLS 1.3 with libcurl. No! CURLOPT_TLS13_CIPHERS was introduced in 7.61.0 (see https://curl.haxx.se/changes.html#7_61_0). If you used it with a prior libcurl version, it did

a URL API ?

2018-08-01 Thread Daniel Stenberg
Hi, In the 2018 user survey, more than 40% of the 395 users who answered the question said they'd use a "URL handling" API in libcurl if one existed. I gave it some thoughts the other day and I've now jotted down my initial suggestion on how it could be made to work. An API that can parse a

Re: curl 7.61.0 - test 1139 fails using --without-libidn

2018-07-31 Thread Daniel Stenberg
On Sun, 22 Jul 2018, David Chapman wrote: "make test" does not leave a tests/log/stderr1139, but if I go into the "tests" directory and run "./runtests.pl 1139", the resulting "stderr1139" file has: --disallow-username-in-url is not in curl.1 (but in tool_getparam.c tool_help.c)

Re: --disable-symbol-hiding not working anymore?

2018-07-31 Thread Daniel Stenberg
On Tue, 31 Jul 2018, Ran Mozes wrote: Somewhere along the way this behavior has changed and in the latest version these functions will show in the „.so“ file as „LOCL“ (hidden). I don't think its due to curl's configure though. I ran 'git diff curl-7_45_0.. -- m4/curl-compilers.m4' and it

Re: How to get list of files in directory smb

2018-07-30 Thread Daniel Stenberg
On Mon, 30 Jul 2018, Artem Smirnov wrote: I'm trying to get a list of files in the directory with libcurl from the SMB server. But I have a error: It is not supported, but we would love to. Feel free to work on it. This item is already listed in our TODO:

Re: General query about SNI implementation.

2018-07-30 Thread Daniel Stenberg
On Mon, 30 Jul 2018, Alisha Joshi wrote: I have referred to this : https://curl.haxx.se/mail/archive-2015-01/0042.html and tried using it in my application and it works. However, I find it worrysome that CURLOPT_RESOLVE option pre-populates the DNS cache with entries for the host+port pair.

Re: Slow Windows uploads (with patch)

2018-07-30 Thread Daniel Stenberg
On Mon, 30 Jul 2018, Daniel Jeliński wrote: I'd love to see the patch accepted, but right now it seems that it's only helping me, and that might not be enough to get it in the next release. Any chance you (the plural you) can create some sort of test program that we can ask Windows users to

Re: plan for a patch release

2018-07-29 Thread Daniel Stenberg
On Fri, 20 Jul 2018, Daniel Stenberg wrote: I'm thinking it might be a good idea to spend the entire 8 week period on bug fixes this time to give us some time to get back on track - even though we're now at 6(!) PRs waiting to get merged in the next feature-window... We still have

Re: hiperfifo example code for linux epoll/timerfd

2018-07-28 Thread Daniel Stenberg
On Fri, 27 Jul 2018, Josh Bialkowski wrote: I was trying to work out how to integrate libcurl's multi interface using epoll and timerfd. In the process, I worked through the `hiperfifo.c` demo using these APIs. I figured I would share the code in case someone would like to add it to the list

Re: AW: libcurl with openSSL on windows x64 - no certificates?

2018-07-27 Thread Daniel Stenberg
On Fri, 27 Jul 2018, Ivan Pilipenko wrote: The idea was to specify a directory where our clients could put their certificates in, instead of relying on us shipping an up-to-date cacert.pem file. libcurl offers no option for such a separate place for extra CA certs. Besides, if I specify

Re: FTPS: error 425 on connection re-use - SSL session not being re-used?

2018-07-27 Thread Daniel Stenberg
On Fri, 27 Jul 2018, Ivan Pilipenko wrote: according to [1], the error occurs because proftpd requires the clients to also re-use SSL sessions if the connection is re-used. Setting the corresponding option in proftpd to work around the issue works, so it seems curl is still not re-using SSL

Re: Help

2018-07-26 Thread Daniel Stenberg
On Wed, 25 Jul 2018, CHUONG HOANG wrote: I'm trying to build (cross compile) a libcurl for Embedded Lynxos on Window64. I download curl-7.60.0 and start with configure command and I have error as bellow, what should I do? I don't think trying to cross-compile on Windows for anything

Re: Unknown system time | feature discussion

2018-07-26 Thread Daniel Stenberg
On Wed, 25 Jul 2018, Hagemann Laurin wrote: I'm developing some embedded stuff and system time is always reset on reboot, so time() will return 0 + seconds since boot. (changing the system time on runtime is not an option) If I'm not mistaken, this is a problem not only for cookie expiry but

Re: How to identify HTTP timeout during response phase

2018-07-25 Thread Daniel Stenberg
On Tue, 24 Jul 2018, Gaurav Pruthi wrote: CURL throws CURLE_OPERATION_TIMEDOUT to signify that connection was timed out. Do we have any mechanism to identify as to the timeout occurs while receiving the data and not during sending the data. No it doesn't. However since you can set a callback

Re: Blocking select

2018-07-24 Thread Daniel Stenberg
On Fri, 20 Jul 2018, Christian Schmitz wrote: In select.c ptimeout = (timeout_ms < 0) ? NULL : _tv; I just have a problem with app which blocks. So I wonder if it is a good idea to have select block forever!? That's the underlying select()-wrapper function. The *user* of that function

Re: curl 7.61.0 - test 1139 fails using --without-libidn

2018-07-22 Thread Daniel Stenberg
On Sat, 21 Jul 2018, David Chapman wrote: (unpack archive into empty directory) $ ./configure $ make $ make test (all tests run) $ make clean $ make $ make test (test 1139 fails) If you after this failure checks tests/log/stderr1139, what does that file contain? -- / daniel.haxx.se

Re: Slow Windows uploads (with patch)

2018-07-20 Thread Daniel Stenberg
On Fri, 20 Jul 2018, Daniel Jeliński wrote: 2018-07-19 18:45 GMT+02:00 Daniel Stenberg : https://github.com/curl/curl/pull/2762 Thank you! I see that the test builds succeeded. What's next? I would prefer to get some additional thumbs up or down from other Windows developers. The change

Re: libcurl with openSSL on windows x64 - no certificates?

2018-07-20 Thread Daniel Stenberg
On Tue, 17 Jul 2018, Ivan Pilipenko wrote: Is there any particular reason why setting CURLOPT_CAPATH to the same directory the cacert.pem/ca-certificates.crt lies in didn't work for me? Our documentation (https://curl.haxx.se/libcurl/c/CURLOPT_CAPATH.html) says the option doesn't work at all

Re: Curl_handler.connecting returning CURLE_AGAIN

2018-07-20 Thread Daniel Stenberg
On Tue, 17 Jul 2018, Richard W.M. Jones wrote: I've written my Curl_handler.connecting callback so it returns CURLE_AGAIN if the socket would block. However this causes the whole connection to fail (see debug trace at end). If I change the callback so it loops until *done == TRUE then it

Re: Test 1143 failing in MSYS bash except with -g

2018-07-19 Thread Daniel Stenberg
On Thu, 19 Jul 2018, Marcel Raad wrote: The strange thing is: when running "./runtests.pl -a -p -v -g" in the tests directory, with -g for gdb, the argument is passed correctly. Can anyone imagine where this discrepancy might come from? I'd say because of this runtests.pl line:

Re: Slow Windows uploads (with patch)

2018-07-19 Thread Daniel Stenberg
On Thu, 19 Jul 2018, Daniel Jeliński wrote: Let me know if there are chances for this patch to make its way to the next release. Thanks! I edited your patch slightly and submitted it as a PR to have it test-built by a few machines: https://github.com/curl/curl/pull/2762 -- /

Re: test 1307...internal Curl_fnmatch() testing failures

2018-07-16 Thread Daniel Stenberg
On Tue, 17 Jul 2018, Dan Fandrich wrote: Better yet would be for the logs to be automatically scanned on upload and if a new failure occurs, notify all submitters in the last day or add a comment to the PR. It wouldn't actually be all that hard to do this, but I just haven't found the time.

Re: Avoiding DNS cache sharing

2018-07-16 Thread Daniel Stenberg
On Mon, 16 Jul 2018, Guillaume Quintard wrote: I have an issue with the DNS cache of one easy handle polluting other transfers and I was wonder if you could help me. You clearly wanted this email sent to the curl-library mailing list. curl-users is primarily for issues and discussions

Re: test 1307...internal Curl_fnmatch() testing failures

2018-07-16 Thread Daniel Stenberg
On Mon, 16 Jul 2018, Dan Fandrich wrote: This test has been failing on every single autobuild where it's run for a long time, probably since commit a115c6bb which switched libcurl to using the system fnmatch back in May. Yeah, clearly... I find it hard to understand why no-one has noticed

Re: Any tips or documentation for writing a new Curl_handler (protocol)?

2018-07-11 Thread Daniel Stenberg
On Wed, 11 Jul 2018, Wouter Verhelst wrote: I have done some (minimal) work with the libcurl API in the past though, and even there I think the more common case is "give libcurl a URL, give it a callback, tell it to have fun". I'm not sure that's the best way forward for an NBD client-side

Re: Any tips or documentation for writing a new Curl_handler (protocol)?

2018-07-11 Thread Daniel Stenberg
On Tue, 10 Jul 2018, Richard W.M. Jones wrote: I was wondering if you have any documentation, or general tips, for writing a new handler? I was thinking of implementing an NBD (Network Block Device) client. I'm afraid we don't have much docs for that! :-( I suppose mostly because it

[SECURITY ADVISORY] curl SMTP send heap buffer overflow

2018-07-11 Thread Daniel Stenberg
- It was reported to the curl project on June 11, 2018 We contacted distros@openwall on July X, 2018. curl 7.61.0 was released on July 11 2018, coordinated with the publication of this advisory. CREDITS --- Detected and researched by Peter Wu. Patch by Daniel Stenberg. Thanks a l

[RELEASE] curl 7.61.0

2018-07-11 Thread Daniel Stenberg
out help, code, reports and advice from friends like these: Adrian Peniak, Alejandro R. Sedeño, Andreas Olsson, Archangel_SDY on github, Bernhard M. Wiedemann, Bernhard Walle, Björn Stenberg, bsammon on github, Dagobert Michelsen, Daniel Stenberg, Dario Nieuwenhuis, Dave Reisner, elepho

Re: CURLE_WRITE_ERROR after update to 7.59.0

2018-07-10 Thread Daniel Stenberg
On Tue, 10 Jul 2018, E T wrote: Content-Encoding: binary Your requeset ask for some specific encodings and the server instead sends you this made up content-encoding. libcurl doesn't know about any "binary" encoding and bails out. This is a bad server. A work-around for this particular

Re: Bug: --form-string data malformed if split across transmit blocks

2018-07-10 Thread Daniel Stenberg
On Tue, 10 Jul 2018, scott silveria wrote: I have encountered a bug that seems to occur when using a command-line of this form: Using which curl version? I failed to reproduce your problem with curl 7.60.0 and curl 7.61.0-DEV. -- / daniel.haxx.se

Re: Is it possible to use some custom functions to send and recv from socket?

2018-07-10 Thread Daniel Stenberg
On Mon, 9 Jul 2018, Bernardo Pastorelli wrote: Looking at the source code it appears that libcurl retrains full control on the read and write operations to the socket. Am I missing any alternative that would allow me to gain control of the read and write operations to the socket so that I can

plan for a patch release

2018-07-09 Thread Daniel Stenberg
Hey, There will be a 7.61.0 release on Wedneday, the 11th, this week. But due to the http2 problems (issue #2674 and #2688), the feature window will not be opened again until those bugs as fixed satisfiably and they will most likely be reason enough for a 7.61.1 release. Once that is out of

Re: An "opt-in component" policy idea

2018-07-08 Thread Daniel Stenberg
On Fri, 6 Jul 2018, Dennis Clarke wrote: What if there existed an "opt-in" feature which would enable curl/libcurl to send a usage status message to some end point collection server? I wouldn't be totally against a system that would ask the user to invoke a special command to pass

Re: CURLE_WRITE_ERROR after update to 7.59.0

2018-07-07 Thread Daniel Stenberg
On Fri, 6 Jul 2018, etks...@gmail.com wrote: We updated libcurl from 7.55.1 to 7.59.0 and are now getting CURLE_WRITE_ERROR from curl_easy_perform(). CURLOPT_WRITEFUNCTION and CURLOPT_HEADERFUNCTION are both set and the functions return size*nitems to all calls. I copied the 7.55.1 library

Re: SCP to Cisco device

2018-07-05 Thread Daniel Stenberg
On Thu, 5 Jul 2018, Martin Galvan wrote: #1 0x76e3941a in EVP_MD_CTX_cleanup () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 ... #14 0x77b8e0dd in ?? () from /usr/lib/x86_64-linux-gnu/libssh.so.4 #15 0x77b7aae4 in ssh_connect () from

Re: SCP to Cisco device

2018-07-05 Thread Daniel Stenberg
On Thu, 5 Jul 2018, Martin Galvan wrote: I quickly tested a debug build of libcurl and still got a segfault. Output is: Segmentation fault (core dumped) ... and if you run that again with a debugger, what does the stack trace from the crash show? -- / daniel.haxx.se

Re: CURLOPT_LOGIN_OPTIONS to select multiple ones?

2018-07-05 Thread Daniel Stenberg
On Thu, 5 Jul 2018, Volker Schmid wrote: Try "AUTH=NTLM;AUTH=CRAM-MD5;AUTH=LOGIN;AUTH=PLAIN" thanks, this is really working and solved my problem! Would be really great if the libcurl documentation would mention that option here: https://curl.haxx.se/libcurl/c/CURLOPT_LOGIN_OPTIONS.html

Re: SCP to Cisco device

2018-07-05 Thread Daniel Stenberg
On Thu, 5 Jul 2018, Martin Galvan wrote: curl -v scp://somehost/home/user/martinTest -u user:pass and got a segfault. Same thing happens with localhost. Notice I'm not hitting an assert but having a segfault. I'm pretty sure I would've gotten a segfault too if I didn't run a debug build...

Enable HTTP/2 when using HTTPS, by default

2018-07-05 Thread Daniel Stenberg
Hi, I'd like to switch libcurl's default HTTP version in 7.62.0 to opt in to HTTP/2 when using HTTPS scheme URLs (if built with support for HTTP/2). It is called CURL_HTTP_VERSION_2TLS when set with the API. This is required to make the most of the suggested change of the multiplex default,

Re: SCP to Cisco device

2018-07-05 Thread Daniel Stenberg
On Wed, 4 Jul 2018, Martin Galvan wrote: Hi Daniel, thanks for the answer. Unfortunately I don't have time right now for re-building the SSH libs to debug them. I'm using the libssh/2 versions that I got from the Ubuntu 17.10 libssh-dev and libssh2-1-dev packages, so it's not some exotic

Re: SCP to Cisco device

2018-07-04 Thread Daniel Stenberg
On Tue, 3 Jul 2018, Martin Galvan wrote: Just checked, the same happens for libcurl 7.60: "Failed to recv file" for libssh2, segfault for libssh. With problems related to the SSH libs, it will be helpful if you can try using those libraries only (without curl) to see if you can reproduce the

Re: Include Addiontal Source Files

2018-07-03 Thread Daniel Stenberg
On Tue, 3 Jul 2018, ozhan duz wrote: I am trying to customize libcurl to my own needs. I modifed the Makefile.inc and added additonal source files to LIB_CFILES variable. It worked perfectly for mingw and msvc. I try to do same thing on Linux but gcc does not compile my own source files. It

Re: make "HTTP/0.9" support opt-in ?

2018-07-02 Thread Daniel Stenberg
On Mon, 2 Jul 2018, Dennis Clarke wrote: It may be of value to create a curl/libcurl "policy" wherein the annual survey data may indicate the degree to which a feature is an opt-in or a default feature. In this way one would be able to say that there is less than a 5% usage rate for a given

Re: DEPRECATE.md

2018-07-02 Thread Daniel Stenberg
On Mon, 2 Jul 2018, Rich Gray wrote:   https://curl.haxx.se/dev/deprecate.html I wonder if for depreciation of build options like axTLS, you should arrange a big warning message should anyone try to build it. This may get someone's attention when they might otherwise miss notification on

make "HTTP/0.9" support opt-in ?

2018-07-02 Thread Daniel Stenberg
Hi, We have this bug [1] that shows a short "HTTP/0.9" response and how curl just then ignores the data it receives. HTTP/0.9 is the popular name for the never truly named HTTP version that existed before HTTP/1.0 was born. It has no response headers at all but instead it just sends data

Re: DEPRECATE.md

2018-07-02 Thread Daniel Stenberg
On Mon, 2 Jul 2018, James Fuller wrote: Maybe we could raise an issue for each deprecated item (apologies for being a process nazi ... just want to be able to subscribe to them), Sure, but I think I prefer doing an issue/PR per step/action so that we don't need to have them linger around

DEPRECATE.md

2018-07-02 Thread Daniel Stenberg
FYI, I just added docs/DEPRECATE.md to the git repo, outlining the deprecation plans for axTLS and pipelining. This document is now available on the site here: https://curl.haxx.se/dev/deprecate.html When these features have been deprecated, I think they should still be mentioned in this

Re: Pipelining is a pain, can we ditch it now?

2018-07-01 Thread Daniel Stenberg
On Sat, 30 Jun 2018, Daniel Stenberg wrote: HTTP Pipelining was never enabled by default by the large desktop browsers due to all the issues with it. Both Firefox and Chrome have also dropped Pipelining support entirely since a long time back now. We are in fact over time becoming more

Pipelining is a pain, can we ditch it now?

2018-06-30 Thread Daniel Stenberg
Hi all, We got a fresh new HTTP pipelining bug submitted today [1]. We already have other pipelining bugs listed in the KNOWN_BUGS document [2]. I'm pretty sure there are more pipelining quirks that just aren't documented/stored anywhere. HTTP Pipelining is badly supported by curl in the

Re: libcurl smtp error list?

2018-06-29 Thread Daniel Stenberg
On Fri, 29 Jun 2018, Gisle Vanem wrote: grepping the header file could be to extract the symbols from docs/libcurl/symbols-in-versions instead, which is sort of "pre-grepped" Can we trust your docs matches the reality? :-) (I assume 'symbols-in-versions' is *non* generated). We can be

Re: Aw: Re: Using CURLOPT_SSLVERSION?

2018-06-29 Thread Daniel Stenberg
On Fri, 29 Jun 2018, Bernhard Jaeger wrote: Do I call the option twice with different parameters to achieve that? Maybe update the documentation than aswell. It currently only says you can OR the two versions, there is no mentioning of an AND option. " It is also possible to OR one of the

<    8   9   10   11   12   13   14   15   16   17   >