Soon, the curl mailing lists will move from cool to lists

[Daniel Stenberg via curl-library]

Hello,

This is just an alert about a pending infrastructure change that I'm planning 
to execute soon.


The curl related mailing lists are currently hosted and run on the server 
called cool.haxx.se. This server is being decommissioned after many years of 
hard and faithful service.


All five curl releated lists we host will soon move over to the new host 
lists.haxx.se. I will move over all subscribers so with a little luck you will 
not have to do anything and things will just continue working. Some of you 
might need to adjust some filters and what not.


When will this happen? I'm not exactly sure yet, but I hope to do it within a 
week from now. I will of course post another mail once the transition has been 
done and I will make sure that docs everywhere is updated and that old links 
redirect over to the new place.


Sorry for any inconveniences this will cause. Thank you for flying curl.

PS. All mailing list info is always here: https://curl.se/mail/

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

RE: Connection to file server disconnected while file transfer is in progress

[Daniel Stenberg via curl-library]

On Thu, 26 Aug 2021, vaibhav.taksale--- via curl-library wrote:


Regarding disconnection, here is code flow -
1. Authentication to server successful.
2. Opening data channel for File download from server
3. after 2 minutes, connection timed out.


How does it "time out" ? What does it mean, exactly. A TCP connection where 
the KEEPALIVE detected it as dead? Or is it just a silent TCP connection 
where no packets are sent?


Is it always timing out after exactly two minutes?

Is this SFTP? If so, your ssh library and version are also important details.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: relative performance

[Daniel Stenberg via curl-library]

On Wed, 25 Aug 2021, XSLT2.0 via curl-library wrote:

Summary : IMHO the important thing is to test on a "slow enough" machine, 
assuming the main measure you want is "elapsed".


Slow enough for what? I agree that if we run the tests over an actual physical 
network, a system with a fast enough CPU will saturate the network and then it 
mostly won't matter how fast or not curl is.


For this kind of test to be sensible, we need to make sure to either have a 
faster pipe than can be saturated by a single CPU core or do a test setup that 
can't do it due to complexity. It seems easiest to accomplish this by doing 
transfers on localhost.


When doing transfers on localhost I don't think it matters much exactly how 
fast the CPU is, and I'm convinced we will see deltas between versions 
whichever CPU we use. In fact, I beleive I've already spotted some. I'm just 
not ready yet to draw the conclusions nor to start working on figuring out why 
they exist.


I've provided scripts in the curl/relative directory now that can:

1. build 'sprinter' the test tool
2. build (lib)curl for a number of versions and install them locally
3. run sprinter with each of those built versions

All you need to do is edit the run-many.sh script to make sure it uses a URL 
on your localhost that works, and adjust how many transfers and which 
concurrency to use.


It seems most interesting to do A LOT of smaller transfers with a fairly huge 
concurrency. I've played with doing 100,000 4K transfers at 100 at a time and 
with 6 8GB transfers at 2 at a time, and the latter will mostly just saturate 
the memory bandwidth in the machine.


The output for the sprinter runs is not easily "comparable" yet and there's no 
machine help to detect regressions etc but it's a decent start I think.


I'm curious if others will see the same thing I seem to see right now...

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: relative performance

[Daniel Stenberg via curl-library]

On Tue, 24 Aug 2021, Ben Greear via curl-library wrote:


What is your network-under-test in this case?


Not sure.

I hope to come up with a way where the network and exact server aren't 
terribly important as long as they perform roughly the same on repeated runs. 
It would need some experimenting I think.


If we run "sprinter" using say 10 different libcurl versions on the same host, 
network and test server (probably using localhost or a host that is on a 
network that cannot be saturated by a single CPU core) it might tell us 
something.


I'll probably start out making the necessary scripts and those 10 runs on my 
own machine to see what I learn and then go from there.


Maybe it could be interesting to point those 10 runs on a different 
server/network setup then to see if that changes anything.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

RE: Connection to file server disconnected while file transfer is in progress

[Daniel Stenberg via curl-library]

On Wed, 25 Aug 2021, vaibhav.taksale--- via curl-library wrote:


[Vaibhav] Libcurl version 7.58 Used on Embedded Linux platform on ar758x board.


This should come as no surprise but of course my first recommendation will be 
to upgrade your libcurl to a recent release as whatever problem you experience 
it is most likely addressed (or different) then.



e.g. In this function callback "writeCallback", error return was expected.


I don't understand. The callback is your code. You make it return error!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Connection to file server disconnected while file transfer is in progress

[Daniel Stenberg via curl-library]

On Tue, 24 Aug 2021, vaibhav.taksale--- via curl-library wrote:

Connection to file server disconnected while file transfer is in progress 
CURL does not get this disconnection response and will continue to wait for 
file transfer.


That's a bug then. Either in your code or in libcurl.

Which libcurl version on which platform is this?


Thus, CURL is not attempting reconnection to resume file transfer.


libcurl will not reconnect nor resume anything on error. It should return an 
error.


Steps: successful connection to file server (filezilla) with valid 
credentials File transfer started possible FTP disconnection and file 
transfer is aborted.


Is this an upload or a download? Exactly how is that "FTP disconnection" done?

Can you provide a small stand-alone example program that reproduces the 
problem?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

relative performance

[Daniel Stenberg via curl-library]

Hi all,

For a long time I've wanted to get something done that allows us to compare 
curl's relative performance. Ideally something we can run every once in a 
while to compare that nothing major has turned sour without us being aware of 
it.


A first step would be a tool we can run that measures "relative performance". 
Like doing N transfers of size X and measure how fast it can complete them. 
Running the same tool on the same host with the same server but built to use 
different libcurl versions should then not get noticably worse speeds over 
time. (Barring the difficulty of measuring network things when other programs 
are also running on the test host.)


I'm not sure exactly how to do this, but I have a first shot at such a tool 
written and I figured we can create a new repository for this (curl/relative 
I'm thinking) and perhaps add more smaller tools for various tests as we 
advance. Then work out how to actually run them with different/current 
libcurls.


Thoughts?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

RE: Assistance building curl 7.77.0 with OpenSSL 3.0

[Daniel Stenberg via curl-library]

On Sat, 21 Aug 2021, Randall S. Becker wrote:

The OpenSSL build is definitely 64 bit - I know because I did the 
build/install about 2 weeks ago. If I leave off the -Wlp64, the linker 
complains about mixing 64 (openssl) and 32 models (curl) during the 
configure phase claiming it can't find openssl.


Using -Wlp64 is a mistake. It would simplify if you dropped all custom flags 
and then showed us what fails.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: configure options for various libcurl builds

[Daniel Stenberg via curl-library]

On Sat, 21 Aug 2021, Anmol Mishra via curl-library wrote:


Could there be some guidance about a simple libcurl static build for


For configure builds, both a shared and a static libcurl version are built. 
SMTP is enabled by default.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Assistance building curl 7.77.0 with OpenSSL 3.0

[Daniel Stenberg via curl-library]

On Fri, 20 Aug 2021, Randall S. Becker via curl-library wrote:


My latest configure command for OpenSSL 3.0 is:

CFLAGS="-c99" CPPFLAGS="-D_XOPEN_SOURCE_EXTENDED=1 -Wlp64 -WIEEE_float 
-I/usr/local-ssl3.0/openssl/include" LDFLAGS="-L/usr/local-ssl3.0/lib" 
conf_script_floss_cc --with-ssl=/usr/local-ssl3.0 --disable-pthreads 
--disable-threaded-resolver --enable-ipv6'


On 64 bit systems, OpenSSL doesn't make a 'lib' anymore. It uses lib64 only 
(and no, I don't know why they decided this). My custom OpenSSL 3 install dir:


$ ls ~/build-openssl/
bin  include  lib64  share  ssl

This breaks old curl configure scripts but the current one is adapted and I 
build with OpenSSL 3 fine and we have a CI job doing it as well.


My configure line for this needs nothing special:

  ./configure --with-ssl=$HOME/build-openssl

... even if I also usually set LDFLAGS="-Wl,-rpath,$HOME/build-openssl/lib64" 
to make it co-exist with other OpenSSL installs better.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: CURLE_COULDNT_RESOLVE_HOST from CURLU API on https://www.google.com

[Daniel Stenberg via curl-library]

On Fri, 20 Aug 2021, Samuel Marks via curl-library wrote:


curl_easy_setopt(curl, CURLOPT_URL, urlp);


From the CURLOPT_URL man page:

  Pass in a pointer to the URL to work with. The parameter should be a char *
  to a null-terminated string which must be URL-encoded

You seem to have intended to use CURLOPT_CURLU?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl.a linker errors on XCode

[Daniel Stenberg via curl-library]

On Thu, 19 Aug 2021, Anmol Mishra wrote:

Using these flags allows for a build - -L/usr/local/Cellar/curl/7.78.0/lib 
-lcurl


The problem is that -lcurl is probably bringing in the default system lib. 
And that is not something I want.


When you build your application you use a linker. If that linker doesn't do 
what you want it to do, I don't see how that is a curl problem. That's a 
linker problem.


-L and -l should point out the directory and the lib to use. Maybe your Xcode 
needs some extra help to understand it needs to pick the static build and not 
use the shared system one?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl.a linker errors on XCode

[Daniel Stenberg via curl-library]

On Thu, 19 Aug 2021, Anmol Mishra wrote:

Intuitively, these extra dependencies seems wrong. If I build a static 
version of a lib, it should not depend on a shared lib.


You build a static version of libcurl. That's static.

You then need to make sure to use whatever you want from the dependencies. 
They can be shared or static, that's not up to the curl build to decide.


You can build your own stuff to use what you think it should use.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl.a linker errors on XCode

[Daniel Stenberg via curl-library]

On Thu, 19 Aug 2021, Anmol Mishra via curl-library wrote:


That makes sense. Is there a list of dependencies somewhere ?


It depends entirely on your specific build.

"sh curl-config --libs" should list them after a build.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl.a linker errors on XCode

[Daniel Stenberg via curl-library]

On Thu, 19 Aug 2021, Anmol Mishra via curl-library wrote:

I built a homebrew libcurl in a custom dir. The project is linking against 
this custom lib. I am getting 200+ linker errors. As far as I know, I 
followed the FAQ. I'm missing something very basic.


It looks like you're building and using libcurl as a static library. Then you 
need to provide all the dependency libraries manually at link time. OpenSSL, 
libssh2, nghttp2 etc by the looks of it.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

7.79.0 feature freeze starts now

[Daniel Stenberg via curl-library]

Hi friends!

Starting now, we no longer merge new features into the master branch until we 
lift the freeze again, which hopefully happens the Monday following the next 
release.


The pending release is planned to ship on September 15th, 2021.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: No curl_*_strerror() for CURLUcode

[Daniel Stenberg via curl-library]

On Fri, 13 Aug 2021, Gleb Ivanovsky wrote:

It is my first time contributing to libcurl. Are there additional steps 
required or do I just create a pull request?


Nothing special, no. A pull-request is perfect and we take it from there.

Perhaps eye through this first: https://curl.se/dev/contribute.html

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: No curl_*_strerror() for CURLUcode

[Daniel Stenberg via curl-library]

On Fri, 13 Aug 2021, Gleb Ivanovsky via curl-library wrote:


Would be nice to have something like:
const char *curl_url_strerror(CURLUcode errornum);


I agree. It would be consistent with how the API works generally.

Do you want to work on it and make it happen?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: External WS library?

[Daniel Stenberg via curl-library]

On Thu, 12 Aug 2021, Felipe Gasper via curl-library wrote:

Regarding the plan to add WebSocket to curl, would it be useful to consider, 
instead of writing a new implementation, incorporating an existing library?


Yes, I think that's a very sensible way to imlement the discussed API and 
matches how we have implemented many other protocols in curl. Standing on the 
shoulders of giants etc.


	I’ve worked a bit with libwebsockets, which seems quite complete. 
Until recently it didn’t integrate into a custom external event loop--as 
would be needed for integration into something like curl--but the upstream 
code now can.


That looks like a very good contender indeed. License wise it should be fine 
and they seem to have a lot of testing and CI going on and it seems like a 
mature library. Claims to be "lightweight" and with a small footprint too.


On the con side, without having actually used or worked with it, seems to be 
that the docs is *generated* and that uninevitably means it will be poor. 
Might not be a big problem though, and is very common.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Question on CURLINFO_SPEED_DOWN/UPLOAD when paused/resumed

[Daniel Stenberg via curl-library]

On Fri, 6 Aug 2021, Joshua Harlow via curl-library wrote:


When pausing and resuming I was noticing that CURLINFO_SPEED_UPLOAD and
CURLINFO_SPEED_DOWNLOAD and its counters/timers don't seem to reset (which
may just be how it is) and from looking at the progress.c code I think this
is what is happening. So if a long pause happens, then the speed
counters/timers may not actually take into account the fact that the
transfer was paused for quite some time which then would produce a not
exactly valid upload or download speed.

Is this the expected behavior when pause/resume is used?


Yes, it is expected. There's no logic that deducts the paused time, so you get 
the transfer size divided by the total time used, including all the paused 
time.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

RE: more WebSockets

[Daniel Stenberg via curl-library]

On Thu, 12 Aug 2021, Daniel Stenberg via curl-library wrote:

Should we just impose our own maximum size and have applications raise it 
when needed?


Answering myself. =)

Ok, I'm convinced we should make the API able to provide full messages. I'll 
adjust acccording.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: more WebSockets

[Daniel Stenberg via curl-library]

On Thu, 12 Aug 2021, Weston Schmidt wrote:


I'd like to add a flag to CURLOPT_WS_OPTIONS that tells curl if it
should negotiate compression or not for easy & multi.



I like the automatic response to pings & pongs by default.  Perhaps
another CURLOPT_WS_OPTIONS flag might disable the automatic response
behavior in the cases where an app doesn't want to respond (or delay
the response, etc).


Thanks, vert good remarks and I've added some text about it now.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

RE: more WebSockets

[Daniel Stenberg via curl-library]

On Wed, 11 Aug 2021, Dmitry Karpov via curl-library wrote:

Thanks for the feedback, this is very helpful!

From a brief look at the document, it looks like Curl will provide only 
WebSocket frame level of communication, so the client will have to implement 
full message assembling itself.


If you by "assembling" mean concatenating multiple frames until the FIN frame, 
then my thinking was yes, so that we wouldn't have to buffer up potentially a 
large amount of data before passing it on.


How do other client implementations work and how do they handle the unlimited 
message size? Should we just impose our own maximum size and have applications 
raise it when needed?


If my understanding is correct, then it seems like a good initial approach 
to me - it handles the most critical WS steps: WS handshake, frame 
sending/receiving and error reporting, even though it leaves WS message 
layer communication (sending/receiving full message) to the clients.


That's my current thinking, yes.

I want the API to be "good enough" to get sufficiently advanced websockets 
communication going against "most" server-side websockets implementations.


When we think it is, we can start working on code to make it real and then see 
how it actually works with some early test client applications. The feature 
will be marked EXPERIMENTAL until we deem it ready anyway so there will be 
wiggle room to change things around all the way through until we decide it is 
fine enough to carve in stone (and ship enabled by default in a release).


As it was mentioned in some of our WS-related discussions, the WS message 
layer is more complex than the framing layer


Can you elaborate on this? Aren't ws messages just the payload from N frames 
concatenated and delivered?


I know there can be control frames injected in the middle of stream of data 
frames, but the only standard such frames are close, ping and pong and I 
imagine libcurl would handle them and thus what is passed on to the client 
would be an unbroken stream of data frames.


libcurl should provide a way to the client to handle incoming message data. 
This means that besides Frame-based "send/receive" callbacks, as described 
in the document, there should be message-based callbacks on top of the 
Frame-layer, which would allow clients to work with WS messages, rather than 
with WS frames.


There's no way then for libcurl to avoid having to buffer the entire ws 
message, right?



A small note about iflags:

" iflags is a bitmask featuring the following (incoming) flags:
   CURLWS_TEXT - this is text data
   CURLWS_BINARY - this is binary data
   CURLWS_FIN - this is also the final fragment of a message
   CURLWS_CLOSE - this transfer is now closed"

The "Text" and "Binary" are special WS frame/message opcodes, so it is 
probably better to distinguish frame flags and the frame opcodes instead of 
mixing them together.


This has been mentioned before but I don't understand why.

Why does it matter to an application exactly how the information arrived? The 
application doesn't see the websocket protocol and it doesn't have to know 
much about it using this API.


If client is supposed to handle WS frames it gets from libcurl, then it 
needs to know the precise opcode along with the frame flags, so it can 
properly handle cases when "control" and "data" frames from different 
messages are intermixed (i.e. one large "data" messages intermixed with many 
"control" messages) and when some "custom" opcodes are used for some 
proprietary WS communications.


To me, that sounds like an argument for providing all the opcodes through to 
the application. I didn't understand that they are actually used like that, 
especially not within the same message.


Can opcode 3-7 and b-f be used "at will" by implementations to signal 
something without negotiating an extension?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: more WebSockets

[Daniel Stenberg via curl-library]

On Wed, 11 Aug 2021, Felipe Gasper wrote:


When a single frame can be 61 bits large?


(Of course I meant 63...)

And thanks for this. As you know I'm a WebSockets rookie so I need and 
appricate pointers like this!


I believe most implementations enforce a maximum message length. Mojolicious 
(Perl), for example, stipulates 256 KiB by default. 
(https://metacpan.org/pod/Mojo::Transaction::WebSocket#max_websocket_size) I 
think Firefox is 2 GiB.


It could of course work to have a maximum message size set, but this makes me 
curious. Surely a client will run into problems if you use 256KB max size 
against a server-side websocket thing that assumes much larger?


Using up to 2 gigabytes buffer for a single message is still several 
magnitudes larger than I would want libcurl to do.


Other websockets implementations are doing that then I presume?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: more WebSockets

[Daniel Stenberg via curl-library]

On Wed, 11 Aug 2021, Felipe Gasper wrote:

Why frame by frame? JS’s API only does full messages, and I think the RFC 
actually stipulates that.


When a single frame can be 61 bits large?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

more WebSockets

[Daniel Stenberg via curl-library]

Hi,

I've refreshed the wiki page a bit using input from the discussion so far. See 
https://github.com/curl/curl/wiki/WebSockets


A few things I realized and tried to reflect in the page:

A single fragment can be 61 bits large and a message consists of multiple such 
fragments: we must have an API that provides data piece by piece to the 
applicaiton and signal the FIN when it arrives.


We need to provide a callback-based approach (as well) to allow for many 
concurrent websocket transfers - especially for applications that want to mix 
those up with a few "regular protocol" transfers as well. I've tried to 
describe how it could work. Not sure it is flexible enough.


I added a few questions marked "TBD" in there that I don't think we have 
answered yet.


I think we can design an API that can work. What's the biggest omissions or 
mistakes in the current draft?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://curl.se/support.html
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: [PATCH] docs: correct symbols-in-versions & doc mismatch for 7.16.4

[Daniel Stenberg via curl-library]

On Fri, 30 Jul 2021, Ævar Arnfjörð Bjarmason via curl-library wrote:

These options introduced in 7.17.0 were documented in 9f44a9552 (Renamed 
several libcurl error codes[...], 2007-08-30) as being introduced in version 
7.16.4, but as looking at the 7.16.4 tag and tarball[1] reveals they weren't 
added until 7.17.0.


This makes me wonder if we're not mostly having a wording issue and I'm not 
sure I agree with your suggested updates.


Let's take the CURLOPT_FTPAPPEND option as an example:


.SH AVAILABILITY
-This option was known as CURLOPT_FTPAPPEND up to 7.16.4
+This option was known as CURLOPT_FTPAPPEND up to 7.17.0


The option was called CURLOPT_FTPAPPEND up to *and including* 7.16.4 from 
what I can tell.


In 7.17.0, this option had been renamed to CURLOPT_APPEND. I thus think the 
former wording is correct. In 7.16.4 it was CURLOPT_FTPAPPEND. Since 7.17.0 we 
call it CURLOPT_APPEND.


Is this wrong?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: How to handle _wcsdup() memory allocation

[Daniel Stenberg via curl-library]

On Thu, 29 Jul 2021, Jeff Mears via curl-library wrote:

Another answer would be for libcurl to implement its own wcsdup, because 
it's just wcslen + Curl_cmalloc + memcpy.


Not being a Windows guy at all, this seems like the most reasonable way that 
doesn't introduce a new API and make things work transparently. Is there a 
downside with this?


Feel like making a PR?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Feature window OPEN

[Daniel Stenberg via curl-library]

Hi,

We've opened the feature window again. It will remain open until August 18th.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: 7.78.0: configure: tweak nghttp2 library name fix

[Daniel Stenberg via curl-library]

On Wed, 21 Jul 2021, Christian Weisgerber via curl-library wrote:

Similar variable manipulation elsewhere in configure.ac uses sed, so I 
suggest to use the same idiom here:


Thanks, I made a PR out of your patch: https://github.com/curl/curl/pull/7472

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[SECURITY ADVISORY] curl: CURLOPT_SSLCERT mixup with Secure Transport

[Daniel Stenberg via curl-library]

CURLOPT_SSLCERT mixup with Secure Transport
===

Project curl Security Advisory, July 21st 2021 -
[Permalink](https://curl.se/docs/CVE-2021-22926.html)

VULNERABILITY
-

libcurl-using applications can ask for a specific client certificate to be
used in a transfer. This is done with the `CURLOPT_SSLCERT` option (`--cert`
with the command line tool).

When libcurl is built to use the macOS native TLS library Secure Transport, an
application can ask for the client certificate by name or with a file name -
using the same option. If the name exists as a file, it will be used instead
of by name.

If the appliction runs with a current working directory that is writable by
other users (like `/tmp`), a malicious user can create a file name with the
same name as the app wants to use by name, and thereby trick the application
to use the file based cert instead of the one referred to by name making
libcurl send the wrong client certificate in the TLS connection handshake.

We are not aware of any exploit of this flaw.

INFO


This flaw has existed in curl since commit
[d2fe616e7e](https://github.com/curl/curl/commit/d2fe616e7e) in libcurl
7.33.0, released on October 14, 2013.

The fixed libcurl version will now instead first check for a certificate in
the key chain using the specified name and only if one does not exist, it will
check for a file name.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2021-22926 to this issue.

CWE-295: Improper Certificate Validation

Severity: Medium

AFFECTED VERSIONS
-

Using libcurl on macOS built to use Secure Transport.

- Affected versions: curl 7.33.0 to and including 7.77.0
- Not affected versions: curl < 7.33.0 and curl >= 7.78.0

Also note that libcurl is used by many applications, and not always advertised
as such.

THE SOLUTION


File names used in this option must contain at least one slash.

A [fix for 
CVE-2021-22926](https://github.com/curl/curl/commit/fd9b40bf8dfd43edcbc0d254d613d95a11061c05)

RECOMMENDATIONS
--

 A - Upgrade curl to version 7.78.0

 B - Apply the patch to your local version

 C - Do now run your application in directories where other users can inject
 files.

TIMELINE


This issue was reported to the curl project on June 15, 2021.

This advisory was posted on July 21, 2021.

CREDITS
---

This issue was reported by Harry Sintonen. Patched by Daniel Stenberg.

Thanks a lot!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[SECURITY ADVISORY] curl: TELNET stack contents disclosure again

[Daniel Stenberg via curl-library]

TELNET stack contents disclosure again
==

Project curl Security Advisory, July 21st 2021 -
[Permalink](https://curl.se/docs/CVE-2021-22925.html)

VULNERABILITY
-

curl supports the `-t` command line option, known as `CURLOPT_TELNETOPTIONS`
in libcurl. This rarely used option is used to send variable=content pairs to
TELNET servers.

Due to flaw in the option parser for sending `NEW_ENV` variables, libcurl
could be made to pass on uninitialized data from a stack based buffer to the
server. Therefore potentially revealing sensitive internal information to the
server using a clear-text network protocol.

This could happen because curl did not call and use sscanf() correctly when
parsing the string provided by the application.

The previous curl security vulnerability
[CVE-2021-22898](https://curl.se/docs/CVE-2021-22898.html) is almost identical
to this one but the fix was insufficient so this security vulnerability
remained.

We are not aware of any exploit of this flaw.

INFO


This flaw has existed in curl since commit
[a1d6ad2610](https://github.com/curl/curl/commit/a1d6ad2610) in libcurl 7.7,
released on March 22, 2001. There was a previous attempt to fix this issue in
curl 7.77.0 but it was not done proper.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2021-22925 to this issue.

CWE-457: Use of Uninitialized Variable

Severity: Medium

AFFECTED VERSIONS
-

- Affected versions: curl 7.7 to and including 7.77.0
- Not affected versions: curl < 7.7 and curl >= 7.78.0

Also note that libcurl is used by many applications, and not always advertised
as such.

THE SOLUTION


Use sscanf() properly and only use properly filled-in buffers.

A [fix for 
CVE-2021-22925](https://github.com/curl/curl/commit/894f6ec730597eb243618d33cc84d71add8d6a8a)

RECOMMENDATIONS
--

 A - Upgrade curl to version 7.78.0

 B - Apply the patch to your local version

 C - Avoid using `CURLOPT_TELNETOPTIONS`

TIMELINE


This issue was reported to the curl project on June 11, 2021.

This advisory was posted on July 21, 2021.

CREDITS
---

This issue was reported and patched by Red Hat Product Security.

Thanks a lot!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[SECURITY ADVISORY] curl: Bad connection reuse due to flawed path name checks

[Daniel Stenberg via curl-library]

Bad connection reuse due to flawed path name checks
===

Project curl Security Advisory, July 21st 2021 -
[Permalink](https://curl.se/docs/CVE-2021-22924.html)

VULNERABILITY
-

libcurl keeps previously used connections in a connection pool for subsequent
transfers to reuse, if one of them matches the setup.

Due to errors in the logic, the config matching function did not take 'issuer
cert' into account and it compared the involved paths *case insensitively*,
which could lead to libcurl reusing wrong connections.

File paths are, or can be, case sensitive on many systems but not all, and can
even vary depending on used file systems.

The comparison also didn't include the 'issuer cert' which a transfer can set
to qualify how to verify the server certificate.

We are not aware of any exploit of this flaw.

INFO


This flaw has existed in curl since commit
[89721ff04af70f](https://github.com/curl/curl/commit/89721ff04af70f) in
libcurl 7.10.4, released on April 2, 2003.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2021-22924 to this issue.

CWE-295: Improper Certificate Validation

Severity: Medium

AFFECTED VERSIONS
-

- Affected versions: curl 7.10.4 to and including 7.77.0
- Not affected versions: curl < 7.10.4 and curl >= 7.78.0

Also note that libcurl is used by many applications, and not always advertised
as such.

THE SOLUTION


The SSL configs are compared appropriately.

A [fix for 
CVE-2021-22924](https://github.com/curl/curl/commit/5ea3145850ebff1dc2b13d17440300a01ca38161)

RECOMMENDATIONS
--

 A - Upgrade curl to version 7.78.0

 B - Apply the patch to your local version

TIMELINE


This issue was reported to the curl project on June 11, 2021.

This advisory was posted on July 21, 2021.

CREDITS
---

This issue was reported by Harry Sintonen. Patched by Daniel Stenberg.

Thanks a lot!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[SECURITY ADVISORY] curl: Metalink download sends credentials

[Daniel Stenberg via curl-library]

Metalink download sends credentials
===

Project curl Security Advisory, July 21th 2021 -
[Permalink](https://curl.se/docs/CVE-2021-22923.html)

VULNERABILITY
-

When curl is instructed to get content using the metalink feature, and a user
name and password are used to download the metalink XML file, those same
credentials are then subsequently passed on to each of the servers from which
curl will download or try to download the contents from. Often contrary to the
user's expectations and intentions and without telling the user it happened.

We are not aware of any exploit of this flaw.

INFO


This flaw exists only in the curl tool. libcurl is not affected.

This flaw has existed in curl since commit
[b5fdbe848bc3d](https://github.com/curl/curl/commit/b5fdbe848bc3d) in curl
7.27.0, released on July 27, 2012.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2021-22923 to this issue.

CWE-522: Insufficiently Protected Credentials

Severity: Medium

AFFECTED VERSIONS
-

- Affected versions: curl 7.27.0 to and including 7.77.0
- Not affected versions: curl < 7.27.0 and curl >= 7.78.0

THE SOLUTION


curl has completely removed the metalink feature as of 7.78.0. No fix for this
flaw will be produced by the curl project.

The fix for earlier versions is to rebuild curl with the metalink support
switched off!

RECOMMENDATIONS
--

 A - Upgrade curl to version 7.78.0

 B - Make sure you do not use metalink with curl

 C - Disable metalink in your build

TIMELINE


This issue was reported to the curl project on May 30, 2021.

This advisory was posted on Jul 21, 2021.

CREDITS
---

This issue was reported by Harry Sintonen. Patched by Daniel Stenberg.

Thanks a lot!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[SECURITY ADVISORY] curl: Wrong content via metalink not discarded

[Daniel Stenberg via curl-library]

Wrong content via metalink not discarded


Project curl Security Advisory, July 21th 2021 -
[Permalink](https://curl.se/docs/CVE-2021-22922.html)

VULNERABILITY
-

When curl is instructed to download content using the metalink feature, the
contents is verified against a hash provided in the metalink XML file.

The metalink XML file points out to the client how to get the same content
from a set of different URLs, potentially hosted by different servers and the
client can then download the file from one or several of them. In a serial or
parallel manner.

If one of the servers hosting the contents has been breached and the contents
of the specific file on that server is replaced with a modified payload, curl
should detect this when the hash of the file mismatches after a completed
download. It should remove the contents and instead try getting the contents
from another URL. This is not done, and instead such a hash mismatch is only
mentioned in text and the potentially malicious content is kept in the file on
disk.

There's a risk the user doesn't notice the message and instead assumes the
file is fine.

We are not aware of any exploit of this flaw.

INFO


This flaw exists only in the curl tool. libcurl is not affected.

This flaw has existed in curl since commit
[b5fdbe848bc3d](https://github.com/curl/curl/commit/b5fdbe848bc3d) in curl
7.27.0, released on July 27, 2012.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2021-22922 to this issue.

CWE-20: Improper Input Validation

Severity: Medium

AFFECTED VERSIONS
-

- Affected versions: curl 7.27.0 to and including 7.77.0
- Not affected versions: curl < 7.27.0 and curl >= 7.78.0

THE SOLUTION


curl has completely removed the metalink feature as of 7.78.0. No fix for this
flaw will be produced by the curl project.

The fix for earlier versions is to rebuild curl with the metalink support
switched off!

RECOMMENDATIONS
--

 A - Upgrade curl to version 7.78.0

 B - Make sure you do not use metalink with curl

 C - Disable metalink in your build

TIMELINE


This issue was reported to the curl project on May 30, 2021.

This advisory was posted on Jul 21, 2021.

CREDITS
---

This issue was reported by Harry Sintonen. Patched by Daniel Stenberg.

Thanks a lot!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[RELEASE] curl 7.78.0

[Daniel Stenberg via curl-library]

ove two variable assigns that are never read [90]
 o url: (void)-prefix a curl_url_get() call [138]
 o url: bad CURLOPT_CONNECT_TO syntax now returns error [31]
 o version: turn version number functions into returning void [135]
 o vtls: exit addsessionid if no cache is inited [7]
 o vtls: fix connection reuse checks for issuer cert and case sensitivity [165]
 o vtls: only store TIMER_APPCONNECT for non-proxy connect [93]
 o vtls: use free() not curl_free() [140]
 o warnless: simplify type size handling [30]
 o Win32: fix build with Watt-32
 o winbuild/README: VC should be set to 6 'or larger' [64]
 o winbuild: support alternate nghttp2 static lib name [174]
 o wolfssl: failing to set a session id is not reason to error out [151]
 o write-out.d: clarify urlnum is not unique for de-globbed URLs [145]
 o zuul: use the new rustls directory name [141]

This release includes the following known bugs:

 o see docs/KNOWN_BUGS (https://curl.se/docs/knownbugs.html)

This release would not have looked like this without help, code, reports and
advice from friends like these:

  Albin Vass, Aleksander Mazur, Alexis Vachette, Alex Xu, Andrea Pappacoda,
  Andrei Rybak, Bachue Zhou, Bastian Krause, Bin Lan, Bin Meng,
  Christian Weisgerber, Christoph M. Becker, civodul on github, Dan Fandrich,
  Daniel Gustafsson, Daniel Stenberg, David Hu, dEajL3kA on github,
  Dmitry Karpov, Dmitry Kostjuchenko, Douglas R. Reno, Ebe Janchivdorj,
  Fawad Mirza, Francisco Munoz, Gabriel Simmer, Gealber Morales, Gergely Nagy,
  Gerrit Renker, Gisle Vanem, Gregor Jasny, Gregory Muchka, Harry Sintonen,
  Hugh Macdonald, Jacob Hoffman-Andrews, Jishan Shaikh, Joel Depooter,
  Jonathan Wernberg, Jon Rumsey, Josh Soref, Josie Huddleston, Jun-ya Kato,
  Kevin Burke, Laurent Dufresne, Li Xinwei, MAntoniak on github, Marcel Raad,
  Marc Hörsken, Mark Swaanenburg, Martin Howarth, Max Zettlmeißl,
  Michael Forney, Michael Kaufmann, Mohammed Naser, nian6324 on github,
  Nikos Mavrogiannopoulos, Paul Groke, Peter Körner, Phil E. Taylor,
  Pierre Yager, Randolf J, Ray Satiro, Red Hat Product Security,
  Richard Marion, Richard Whitehouse, Sergey Markelov, Shikha Sharma,
  shithappens2016 on github, sylgal on github, Timur Artikov, Tobias Nyholm,
  Tommy Chiang, User Sg, Vadim Grinshpun, Valentín Gutiérrez, Viktor Szakats,
  William Desportes, Wyatt OʼDay, Xiang Xiao, Yongkang Huang, Younes El-karama,
  Zhang Xiuhua, Борис Верховский, Коваленко Анатолий Викторович,
  (83 contributors)

References to bug reports and discussions on issues:

 [1] = https://curl.se/bug/?i=7133
 [2] = https://curl.se/bug/?i=7167
 [3] = https://curl.se/bug/?i=7152
 [4] = https://curl.se/bug/?i=7154
 [5] = https://curl.se/bug/?i=7159
 [6] = https://curl.se/bug/?i=7162
 [7] = https://curl.se/bug/?i=7165
 [8] = https://curl.se/bug/?i=7164
 [9] = https://curl.se/bug/?i=7095
 [10] = https://curl.se/bug/?i=7157
 [11] = https://curl.se/bug/?i=7157
 [12] = https://curl.se/bug/?i=7158
 [13] = https://curl.se/bug/?i=7153
 [14] = https://curl.se/bug/?i=7148
 [15] = https://curl.se/bug/?i=7151
 [16] = https://curl.se/bug/?i=7039
 [17] = https://curl.se/bug/?i=7146
 [18] = https://curl.se/mail/lib-2021-06/0001.html
 [19] = https://curl.se/bug/?i=7143
 [20] = https://curl.se/mail/lib-2021-06/0003.html
 [21] = https://curl.se/bug/?i=7141
 [22] = https://curl.se/bug/?i=7141
 [23] = https://curl.se/bug/?i=7139
 [24] = https://curl.se/bug/?i=7138
 [25] = https://curl.se/bug/?i=7172
 [26] = https://curl.se/bug/?i=7122
 [27] = https://curl.se/bug/?i=7134
 [28] = https://curl.se/bug/?i=7130
 [29] = https://curl.se/bug/?i=7204
 [30] = https://curl.se/bug/?i=7181
 [31] = https://curl.se/bug/?i=7183
 [32] = https://curl.se/bug/?i=7204
 [33] = https://curl.se/bug/?i=7184
 [34] = https://curl.se/bug/?i=7185
 [35] = https://curl.se/bug/?i=7170
 [36] = https://curl.se/bug/?i=7205
 [37] = https://curl.se/bug/?i=7144
 [38] = https://curl.se/bug/?i=7177
 [39] = https://curl.se/bug/?i=7179
 [40] = https://curl.se/bug/?i=7175
 [41] = https://curl.se/bug/?i=7169
 [42] = https://curl.se/bug/?i=7173
 [43] = https://curl.se/bug/?i=7166
 [44] = https://curl.se/bug/?i=7028
 [45] = https://curl.se/bug/?i=7028
 [46] = https://curl.se/bug/?i=7205
 [47] = https://curl.se/bug/?i=7205
 [48] = https://curl.se/bug/?i=7205
 [49] = https://curl.se/bug/?i=7205
 [50] = https://curl.se/bug/?i=7205
 [51] = https://curl.se/bug/?i=7205
 [52] = https://curl.se/bug/?i=7206
 [53] = https://curl.se/bug/?i=7203
 [54] = https://curl.se/bug/?i=7176
 [55] = https://curl.se/mail/lib-2021-06/0008.html
 [56] = https://curl.se/bug/?i=7222
 [57] = https://curl.se/bug/?i=7218
 [58] = https://curl.se/bug/?i=7257
 [59] = https://curl.se/bug/?i=7236
 [60] = https://curl.se/mail/lib-2021-06/0018.html
 [61] = https://curl.se/bug/?i=7240
 [62] = https://curl.se/bug/?i=7227
 [63] = https://github.com/curl/curl/discussions/7255
 [64] = https://curl.se/bug/?i=7253
 [65] = https://curl.se/bug/?i=7250
 [66] = https://curl.se/bug/?i=7209
 [67] = https://curl

RE: How to authenticate when using “CURLOPT_CONNECT_ONLY” so I can use non-blocking curl_easy_send() and curl_easy_recv()

[Daniel Stenberg via curl-library]

On Mon, 19 Jul 2021, Venkat Maddisetti via curl-library wrote:

Before coming here I realized curl_easy_setopt has no effect setting 
username and password


CURLOPT_USERPWD, CURLOPT_USERNAME and CURLOPT_PASSWORD included ?


"POST /emp/ HTTP/1.1


If you want HTTP, then using CONNECT_ONLY + send/recv is *extremely* rarely 
the right answer.


You seem to want a plain POST with Basic auth and libcurl does that pretty 
well and simply with the regular easy API.


First, get the http-post.c example
Then set CURLOPT_USERNAME and CURLOPT_PASSWORD
Then set the right Content-type and make sure the right request body is sent.

Doesn't that cover it?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Can I use curl_easy_setopt during curl_easy_perform.

[Daniel Stenberg via curl-library]

On Mon, 12 Jul 2021, Wu Yuanshou via curl-library wrote:

Shall I call curl_easy_setopt() during curl_easy_perform() ? It seems like 
we should not do this according to the document I can find.Instead , I shall 
quit from the curl_easy_perform() function , apply the setting via 
curl_easy_setopt() and call curl_easy_perform() again.


Correct, that's the way the documentation encourages and that we guarantee.

There are a few options that users often rather want to modify at run-time, 
like sometimes CURLOPT_MAX_RECV_SPEED_LARGE, and while there's no guarantee in 
the docs or code, this typically works fine.


I would like us to at some point explictly mark and document which options 
that are fine to update during a transfer.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: [PATCH] Revert "ftp: Expression 'ftpc->wait_data_conn' is always false"

[Daniel Stenberg via curl-library]

On Wed, 7 Jul 2021, Jonathan Wernberg via curl-library wrote:

I could not make a GitHub pull request at this point in time, I hope it is 
acceptable to send the patch this way instead.


Thanks!

I made a PR out of it: https://github.com/curl/curl/pull/7362

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl websockets

[Daniel Stenberg via curl-library]

On Fri, 2 Jul 2021, Felipe Gasper wrote:


Or perhaps with the type and flags just made as a single bitmask.


This will make it impossible to specify a continuation frame rather than a 
data frame. (Context: when a message is fragmented, the message’s first 
frame is text/binary, and the latter ones are continuation.)


"CURL_WS_MORE" is my currently suggested flag to say to libcurl that there's 
more data coming that belongs to the same data packet. Doesn't that work?


And if doesn't work, can't we just invent flags to make a way that works?

Also, if WS were to add a 3rd data type -- e.g., via some extension -- then 
it would be possible to specify CURL_WS_TEXT | CURL_WS_FANCY_NEW_EXT_TYPE, 
which means curl would need to detect that.


We would have some saved bits for potential future use for sure, just as we 
would have saved types, so a future curl version could get support for it. I 
don't think we can imagine a way that can add support for such new features 
without an update.



Alternatively, have separate curl_ws_send_text and curl_ws_send_binary 
functions?


That could certainly work and would save users sending text from having to set 
a "this is text" bit. My personal preference is to rather keep it to one 
function with the bits for type, but I'm here to listen in what peeps want and 
I will not push my own preferences if I'm in a minority.


curl_ws_recv() could just be made to not return until the entire packet can 
be returned. Or with an option to return a partial one.


I assume you mean something like EAGAIN rather than actually blocking?


Yes, sorry I didn't even think about that. I'm positive users will 
immediately come up with use cases where they want 22 parallel websocket 
connections going on so they cannot block. Or perhaps we can offer some kind 
of blocking abilitites, but non-blocking is a must-support feature.


Another thing that struck me is that we probably want to offer a decent way to 
do non-blocking ws-connects as well for the 22-connections user who won't 
accept that the application will block once for each setup...


Having the caller call curl_ws_recv() directly would mean that that same 
caller would need to know when the socket has data to read. I don’t know if 
that’s always a safe assumption.


Yeah, that's one of my outstanding questions. How do we do that?

For CONNECT_ONLY, libcurl allows the application to extract the socket for the 
transfer so that it can select() on it, but I have a feeling that's a little 
too raw here. We might need to provide a function for this purpose. 
curl_ws_poll() maybe. It needs some further thinking.


Also, WS includes ping/pong, so even if the socket has data there might not 
be any application-level data incoming. (Ping/pong isn’t usually exposed to 
the application.)


Yes, that certainly needs to be taken into account and be handled correctly.

An area for consideration is how to deal with the buffer (size). I mean, if 
it wants to wait for a full packet to arrive I suppose it can be large and 
it might not fit in the user-provided buffer, so the application would need 
to call it multiple times do drain the data.


Would this argue in favour of a callback, then? Curl could just allocate as 
it knows it needs. The caller, then, could either memcpy or assume ownership 
of the buffer (by, e.g., returning CURL_WS_NOFREE from the callback).


We wouldn't need a callback for that, the recv-function could just as well 
return a pointer to an allocated buffer of a given size that has the received 
packet if we would like that kind.


That said, providing allocated memory blocks to the application is a rather 
quirky (and unusual) way to deliver data and it doesn't sit very well with me.


We could also just return a bit (CURL_WS_FRAG) in the 'recvflags' which means 
it was only a fragment and that it needs to call the recv function again to 
get the next piece of the data.



 https://github.com/curl/curl/wiki/WebSockets#api


I feel funny asking this, but is there a comment feature in this wiki 
system? I don’t see such, but it would seem useful.


Yeah, it would be super useful but there is no such thing. I suppose you can 
edit the wiki and insert comments with some markup (color?) that stands out.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl websockets

[Daniel Stenberg via curl-library]

On Fri, 2 Jul 2021, Weston Schmidt wrote:

I'm not sure who might be using that & yes it's 8 years old in a draft 
state.  I mentioned it because it's one of the more extreme looking use 
cases.  If we can support that type of extension via a planned interface 
then we've probably got a good API.


Isn't websockets multiplexing done with HTTP/2 these days, RFC 8441 style?

https://datatracker.ietf.org/doc/html/rfc8441

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl websockets

[Daniel Stenberg via curl-library]

On Fri, 2 Jul 2021, Felipe Gasper wrote:

Thanks!


The send interface could look much like curl_easy_send():

curl_ws_send( easy, buffer, buflen, , enum curl_ws_message_type type, 
enum curl_ws_flags flags )


Or perhaps with the type and flags just made as a single bitmask.

Most WS libraries that I’ve seen implement reception as a callback, with a 
separate control to pause reception of incoming messages.


I won't say that's wrong, but I don't see how a callback helps with this API.

curl_ws_recv() could just be made to not return until the entire packet can be 
returned. Or with an option to return a partial one.


An area for consideration is how to deal with the buffer (size). I mean, if it 
wants to wait for a full packet to arrive I suppose it can be large and it 
might not fit in the user-provided buffer, so the application would need to 
call it multiple times do drain the data.


I took the liberty of jotting down some of these API thoughts in the wiki 
page. Still incomplete and not really functional, but I figured it could help 
to stir up our collective minds..


  https://github.com/curl/curl/wiki/WebSockets#api

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl websockets

[Daniel Stenberg via curl-library]

On Thu, 1 Jul 2021, Weston Schmidt wrote:


I like the simplicity of the CURLOPT_CONNECT_ONLY=2L + ws:// or wss:// plan.

If libcurl stops at the _The WebSocket Connection is Established_ point 
(https://datatracker.ietf.org/doc/html/rfc6455#page-20) when 
CURLOPT_CONNECT_ONLY=2L, then by definition that is only after the HTTP 
response 101 Upgrade stuff has happened.  This means that redirects and 
other HTTP magic happens as normal.  Libcurl then does not need to support 
or be altered for extensions or subprotocols.


I think that's a decent plan for the initial bootstrap-into-websockets step. I 
could even imagine that CURLOPT_CONNECT_ONLY=2L could be implied by the 
explicit use of ws:// or wss:// in the URL.


What's a little unclearer to me is what we need for the next phase, once the 
connection is done and setup. curl_easy_send/recv seem inadequate for 
websockets, which might imply that we need to provide curl_ws_send/recv. If 
so: A) how should they work and B) what else do we need?


If we would write an entire imaginary very basic "hello world" websockets 
program using libcurl, what would it look like?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

curl download mirrors no more

[Daniel Stenberg via curl-library]

Hello friends!

When curl started to become popular, we setup a system to provide curl 
downloads via a system of mirrors run and hosted by volunteers. It really 
helped us to deliver curl everywhere at high speeds and with great stability.


Today however, with the website infrastructure being handled by a 
globe-spanning powerful CDN like Fastly, we actually save ourselves some 
trouble and maintenance by providing the downloads ourselves. They're still 
fast and reliably provided.


Since we transitioned to the new website infra, months ago, I haven't setup 
the rsync for mirrors so they've not been able to stay in synch, and I've also 
recently removed the mirror-links from the download page.


As of today we're officially stopping the mirrored download service.

I'd like to express a HUGE and sinsere THANK YOU to all of you awesome people 
who ran and have run these lovely mirrors for us for so long just for the good 
of the project and its users. It is friends like you who make curl possible.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl websockets

[Daniel Stenberg via curl-library]

On Wed, 30 Jun 2021, Weston Schmidt wrote:

Maybe this is a misunderstanding on my part, but I think (and my testing 
seems to confirm) that CURLOPT_CONNECT_ONLY will stop at the first 
connection to the first server encountered.  If you have some other http 
redirector in front of the ultimate server hosting the websocket, or the 
websocket server returns 429 to indicate it's busy, try again later, the app 
must then handle all this work (I think ... I certainly could be wrong).


No, that's a correct understanding. CURLOPT_CONNECT_ONLY, as the name implies 
only does the connect part and then stops.


The similarity it would share with websockets-in-curl is how it doesn't 
actually do any transfer, but will just do a controlled set of setting up 
protocol things and then return.


In my use cases, I'm generally asking libcurl to use HTTP to do redirection 
and traffic reduction things with libcurl, then after the "exciting bits" 
are done, I'd like to take ownership of the socket.


And that's not a feature we have in current libcurl... but I don't think it 
would be too complicated to add if we end up deciding that's what we need.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl websockets

[Daniel Stenberg via curl-library]

On Thu, 24 Jun 2021, Weston Schmidt via curl-library wrote:

This has been good feedback, thank you all.  I'm going to work on writing up 
what has been covered into a proposal here: 
https://github.com/schmidtw/curl-websocket-proposal


I created a "home" for the WebSockets brain-storming in curl's wiki:

  https://github.com/curl/curl/wiki/WebSockets

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Access gmail with pycurl.

[Daniel Stenberg via curl-library]

On Sat, 26 Jun 2021, Hongyi Zhao via curl-library wrote:

I noticed that libcurl, hence pycurl supports IMAP and IMAPS, so I want to 
know if I can use it to retrieve/remove emails from my Gmail account?


Yes you can.

Jay once wrote ShowGmailFolders.c for example:

  https://gist.github.com/jay/a43724c98220de85d020

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Multi Threaded Resolver Hanging on Application Shutdown

[Daniel Stenberg via curl-library]

On Thu, 24 Jun 2021, Tanner, Matt via curl-library wrote:

We have a multi-threaded embedded linux application that uses curl 7.76.1 
for https client requests with easy handles and the multi threaded resolver 
(AsynchDNS feature).  We're seeing intermittent hangs in curl during 
application shut-down. Its in the resolver. I was wondering if any obvious 
mis-uses of the library jumped out for anyone, or known bugs.


This is a known bug or issue 
(https://curl.se/docs/knownbugs.html#Multi_perform_hangs_waiting_for). It is 
complicated. Once discussed somewhat over here:


 https://github.com/curl/curl/issues/2975

The problem is that one of the name resolving threads is slow and curl awaits 
them all to die before it can continue, to avoid risking to leak memory. In 
earlier curl history, it instead leaked memory for that situation but then it 
didn't do this "hang".


In many cases, the leaked memory might not be a problem and would instead 
rather be the preferred behavior if the choice existed...


There are legacy areas of our app that do not use curl and are calling 
getaddrinfo directly in other threads. Could that be a problem?


I don't think that's related.

Its possible we could try the c-ares backend, but I need to try and get a 
feel for the root cause before starting the process of adding another 
third-party library.


The c-ares backend does not have this problem as a slow resolve done by c-ares 
can be cancelled at once.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl websockets

[Daniel Stenberg via curl-library]

On Wed, 23 Jun 2021, Weston Schmidt via curl-library wrote:

You bring up an interesting point that I don't have a good answer for - how 
should the easy interface work?  I struggled with it when working on the 
curlws code & decided to avoid it in favor of the multi api.  If there is a 
good example of something similar in curl as a pattern, that might help 
inform how to do it.


The easy interface is for full data transfers in either or both directions. It 
is not a good fit for websockets (generic custom bidirectional connectivity), 
and that's why I thought about mimicing the model we support for CONNECT_ONLY 
plus curl_easy_recv/curl_easy_send.


Even if I having been educated in this thread have realized that the 
curl_easy_recv and curl_easy_send calls won't work for this as-is, so we 
should therefore probably consider adding special-purpose websocket read/send 
API calls.


The CONNECT_ONLY style of connect-only could perhaps be implied when as 
ws(s):// scheme is used.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl websockets

[Daniel Stenberg via curl-library]

On Tue, 22 Jun 2021, Weston Schmidt via curl-library wrote:

I'm interested in feedback or suggestions for what/how to define a good 
websocket API that complements libcurl.


My very simple idea on how it could be added to the API:

1. We add another value to set CURLOPT_CONNECT_ONLY to (2). If this is used, 
and the URL is "http(s)://" then libcurl could "bootstrap" into a websocket 
connection.


2. Once it is completed, we could have the application extract the socket (to 
have something to wait on) and then use curl_easy_recv() and curl_easy_send() 
to receive and send websockets data over that connection.


What would this be missing?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl hangs in curl_multi_cleanup

[Daniel Stenberg via curl-library]

On Wed, 23 Jun 2021, firefo...@interia.pl wrote:


So, since 215db086e0 test660 ensures wrong behavior of libcurl.


Since we try to have tests for *everything* curl can do, so the fact that we 
verify bugs is what is *supposed* to happen. Until we realize it is a bug, it 
isn't a bug and then it works as it should...


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

feature freeze for 7.78.0

[Daniel Stenberg via curl-library]

Hello friends!

The door is now officially closed for new features for the next release. Now 
we just merge bug-fixes until July 21 when we intend to ship the next curl 
release.


Happy hunting!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl hangs in curl_multi_cleanup

[Daniel Stenberg via curl-library]

On Tue, 22 Jun 2021, FirefoxOS via curl-library wrote:


Thanks for creating a test case.


I've now made a PR that I believe fixes this issue:

 https://github.com/curl/curl/pull/7288

... and the test 677 is included there and reproduces the bug before this fix.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

"curl: 25 years and 200 releases later"

[Daniel Stenberg via curl-library]

Hiya,

I visited the GitHub ReadME podcast and talked curl a while ago and today it 
was published: https://github.com/readme/curl-25-years


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Feature request: new callback to report http chunks boundaries

[Daniel Stenberg via curl-library]

On Sun, 20 Jun 2021, Daniel Stenberg via curl-library wrote:


I think that would encourage really bad applications.


This all said, you can disable decoding it with CURLOPT_HTTP_TRANSFER_DECODING 
and then you get to decode the chunked encoding yourself, which then allows 
you to find the boundaries.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Feature request: new callback to report http chunks boundaries

[Daniel Stenberg via curl-library]

On Sun, 20 Jun 2021, Daniel F via curl-library wrote:

I think of two possible callbacks: one called before first part of a chunk 
is delivered to the app (it may have chunk size as a param), and another 
called after last part of chunk is delivered to the app. Any of them would 
be fine for me.


I think that would encourage really bad applications.

1. The server is not obliged to provide chunks in the response just because it 
did it once or usually does it. You cannot rely on them.


2. If you access the resource over a HTTP proxy it may *change* the chunks
mid-stream.

3. If you get the data over HTTP/2 or HTTP/3, chunked encoding doesn't
exist...


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl hangs in curl_multi_cleanup

[Daniel Stenberg via curl-library]

On Fri, 18 Jun 2021, Firefox OS via curl-library wrote:


Do you mean imap://...? Yes, it does. And the callstack looks the same.


I cannot reproduce. I converted "curl_imap_teardown_issue.c" into test case 
677 in this commit: 
https://github.com/curl/curl/commit/4401aede3133d1db9e3cc907b3c6640eda06c57a


You can check out this branch:
  https://github.com/curl/curl/tree/bagder/multi_cleanup-imap-test

And if you build the code and the tests run 677 like:

 cd tests && make -sj7 && ./runtests.pl 677

Can you figure out what we need to change to make lib677.c hang?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: How to separate connection from request

[Daniel Stenberg via curl-library]

On Thu, 17 Jun 2021, 6dsovh35--- via curl-library wrote:

I'm trying to build a transport service that will maintain open connections 
to various services and send requests off as I receive them. I would like to 
build a curl session and connect it (including openssl handling) in my 
parent process and reuse that connection in my child process after it has 
been forked to actually send the request payload to the destination. I don't 
know what the request will be when I'm trying to open the connection so 
would like to keep from making a random curl_easy_perform that is likely to 
return an error. How can I accomplish this? I did stumble across a 
connect_only config that seemed like it would tell curl to only connect, but 
if I understand it correctly it then disables any reuse on that socket which 
is not what I'm after. Is what I'm seeking to do possible?


To me, this sounds like you want to do what I'd call "pre-connects". That is, 
connect to the server before you know if you want to issue a request to it, in 
order to have the request done faster later on if it indeed needs to be done.


libcurl doesn't have any (current) API that allows such connects without a 
request. The CONNECT_ONLY requests stops after the connect and that connection 
can't be reused again.


I have two suggestions on how to move forward:

1 - Issue a basic HEAD request and ignore the response. That's very little 
traffic so it almost achieves the same thing, just with some data back and 
forth as well.


2 - Introduce new functionality to libcurl that is a CONNECT_ONLY flavor that 
makes a normal connection left in the connection cache, possible to get reused 
in subsequent requests. Most of the logic should be the same as CONNECT_ONLY.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl hangs in curl_multi_cleanup

[Daniel Stenberg via curl-library]

On Thu, 17 Jun 2021, FirefoxOS via curl-library wrote:


Can you help me understand what does test660 actually check?



The issue is easily reproducible using curl_imap_teardown_issue.c


Then clearly test660 doesn't run the same sequence.

Does the curl_imap_teardown_issue.c code also cause a problem if run against 
an IMAP server without using TLS?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

webinar: common libcurl mistakes

[Daniel Stenberg via curl-library]

Hello friends!

This is just a quick mention that I'll do a free-to-attend webinar this 
Tursday titled as the subject says: common libcurl mistakes. With a following 
Q of course.


Sign up here:

  https://www.wolfssl.com/upcoming-wolfssl-webinar-common-libcurl-mistakes/

See you Thursday!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: building curl on macos

[Daniel Stenberg via curl-library]

On Thu, 10 Jun 2021, Dave via curl-library wrote:


Am I right in assuming that $PREFIX_OPENSSL would be set by this?
./configure --with-ssl=$openssldir/builds/osx/release


Yes. And that should be the prefix to where you installed OpenSSL.

And note that we prefer --with-openssl now instead of--with-ssl to make it 
more explicit what you're asking for. The latter one will be removed at some 
point in time.



The path I am passing (see my original email) does indeed have a ssl.h at
$openssldir/builds/osx/release/include/openssl/ssl.h

The error though is not about the ../osx/release folder, but a
.../osx/debug folder next to it, which doesn't exist.
Nor am I telling any script to look for a debug folder as far as I am aware.


I can't explain that. Here's the configure code that checks for the header:

https://github.com/curl/curl/blob/23eef2394cb810294a669d4bb4862bbdb2f5ced6/m4/curl-openssl.m4#L333-L335

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl hangs in curl_multi_cleanup

[Daniel Stenberg via curl-library]

On Thu, 10 Jun 2021, Firefox OS wrote:


The story is the same as 2 years ago


You perceive it as the same, I do not. The previous issue was reproduced with 
and then verified with test 660 and that test case still runs fine today.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: 7.77.0: tests/server/sws overflows stack

[Daniel Stenberg via curl-library]

On Tue, 8 Jun 2021, Christian Weisgerber via curl-library wrote:

Maybe such oversized allocations should use malloc() instead of local 
variables?  Here's a patch to this effect:


Thanks! (proves we should have openbsd CI jobs...)

Converted to a pull-request: https://github.com/curl/curl/pull/7235

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: building curl on macos

[Daniel Stenberg via curl-library]

On Thu, 10 Jun 2021, Dave via curl-library wrote:


configure: error: [redacted, $openssldir]/builds/osx/debug is a bad
--with-openssl prefix!


This means $PREFIX_OPENSSL/include/openssl/ssl.h is missing.

If I create a debug library for OpenSSL and place it in that folder it does 
succeed.


My question is whether I can avoid creating and depending on openssl debug 
libs with the right options for the curl configure script?


The header file being there or not should not be dependent on a debug version.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl hangs in curl_multi_cleanup

[Daniel Stenberg via curl-library]

On Thu, 10 Jun 2021, Firefox OS via curl-library wrote:


Unfortunately the bug is there again.
At least in 7.76.1 and 7.70.0.
Particularly, curl_multi_cleanup() hangs for exactly 2 minutes.


Please tell us the whole story.

What do you do and when does it hang? We want to be able to reproduce.

What curl version with what components?

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl options for --create-dirs -O --output-dir

[Daniel Stenberg via curl-library]

On Tue, 8 Jun 2021, Anmol Mishra via curl-library wrote:

However, I cannot see the equivalent options inside libcurl. Is the 
--output-dir option only available in the curl CLI ?


Yes, since libcurl doesn't really save content to files/directories. It 
delivers the content to a callback.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

[Daniel Stenberg via curl-library]

On Mon, 7 Jun 2021, Jeroen Ooms wrote:

I'm curious to know if this was followed up on? R users have been reporting 
random HTTP2 framing layer errors for multiplexed connections to google 
servers/apis for a long time, but I have never succeeded to narrow down the 
exact problem.


https://github.com/curl/curl/pull/7192

Merged two days ago as commit a0709f99513

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

RE: Connection takes too long in dual-stack if IPv6 starting family is not available or blocked and has multiple addresses.

[Daniel Stenberg via curl-library]

On Fri, 4 Jun 2021, Dmitry Karpov wrote:

(I've put the CC back to the libcurl list to allow everyone to participate)

Initially, I also had the same idea to use happy eyeballs timeout to cover 
cases of connection socket errors, but with host resolutions having multiple 
IPv6 addresses, it was really a waste of time to try next addresses of the 
failed family knowing that they would definitely fail. I think that in the 
current state, when Curl tries family addresses sequentially, happy eyeball 
timeout should be applied only for "good" connections that don't issue 
socket errors, and socket errors should be probably treated as a "family 
failure".


Hm, what if we also consider the first *error* on IPv6 as a signal to kick off 
the IPv4 connect in addition to the timeout? See updated patch below.


If Curl tried parallel connections for all addresses inside the family 
instead of doing it sequentially, then the idea to apply happy eyeballs 
timeout to the socket errors would work very well. Parallel connections 
would allow to fail the family much faster if something was wrong with it, 
and this would also be beneficial because the client would get the fastest 
connection inside the family - which is nice.


Yes, and I'm not against trying this out - this idea has been mentioned before 
in HTTP implementor "circles". I would however imagine that it would need to 
be capped at some value because some names will return MANY addresses so this 
could be rather excessive, which could become a problem for apps who fire up 
many transfers at once.


--- patch proposal v2

diff --git a/lib/connect.c b/lib/connect.c
index d9317f378..76e9d50dd 100644
--- a/lib/connect.c
+++ b/lib/connect.c
@@ -931,18 +931,10 @@ CURLcode Curl_is_connected(struct Curl_easy *data,
  conn->timeoutms_per_addr[i]) {
 infof(data, "After %" CURL_FORMAT_TIMEDIFF_T
   "ms connect time, move on!\n", conn->timeoutms_per_addr[i]);
 error = ETIMEDOUT;
   }
-
-  /* should we try another protocol family? */
-  if(i == 0 && !conn->bits.parallel_connect &&
- (Curl_timediff(now, conn->connecttime) >=
-  data->set.happy_eyeballs_timeout)) {
-conn->bits.parallel_connect = TRUE; /* starting now */
-trynextip(data, conn, sockindex, 1);
-  }
 }
 else if(rc == CURL_CSELECT_OUT || conn->bits.tcp_fastopen) {
   if(verifyconnect(conn->tempsock[i], )) {
 /* we are connected with TCP, awesome! */

@@ -1002,10 +994,20 @@ CURLcode Curl_is_connected(struct Curl_easy *data,
conn->tempsock[other] == CURL_SOCKET_BAD)
   /* the last attempt failed and no other sockets remain open */
   result = status;
   }
 }
+
+if(!rc || error) {
+  /* should we try another protocol family? */
+  if(i == 0 && !conn->bits.parallel_connect &&
+ (error || (Curl_timediff(now, conn->connecttime) >=
+data->set.happy_eyeballs_timeout))) {
+conn->bits.parallel_connect = TRUE; /* starting now */
+trynextip(data, conn, sockindex, 1);
+  }
+}
   }

   if(result &&
  (conn->tempsock[0] == CURL_SOCKET_BAD) &&
  (conn->tempsock[1] == CURL_SOCKET_BAD)) {


--

 / daniel.haxx.se
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: configure script does not handle --with-darwinssl correctly?

[Daniel Stenberg via curl-library]

On Sun, 6 Jun 2021, Vadim Grinshpun wrote:

Shouldn't the configure script fail if --with-darwinssl is provided? Or does 
that option have a different meaning?


It should! I've amended the PR just now with that fix as well.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: configure script does not handle --with-darwinssl correctly?

[Daniel Stenberg via curl-library]

On Sun, 6 Jun 2021, Vadim Grinshpun via curl-library wrote:

The docs  seem to indicate that 
--with-darwinssl and --with-secure-transport should be equivalent, however 
the above message disagrees.


Correct, --with-secure-transport is the only name of the flag since a while 
back.


I've filed a PR to fix the docs: https://github.com/curl/curl/pull/7200

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Connection takes too long in dual-stack if IPv6 starting family is not available or blocked and has multiple addresses.

[Daniel Stenberg via curl-library]

On Thu, 3 Jun 2021, Dmitry Karpov via curl-library wrote:

I am not sure if this is the right fix, but it helped to improve the 
connection phase time for such cases very dramatically and make it close to 
pure IPv4 connection times. My tests also didn't reveal any bad side effects 
of this change.


Thanks for this!

From my understanding of this issue, this problem happens because 
Curl_is_connected() gets called after a while but not because of a timeout on 
the socket but because of an error.


This makes 'rc' not contain zero so the conditional check if it should start 
the other family for connecting doesn't run but instead it kicks off another 
connect attempt to the next IPv6 address.


If this is correct, shouldn't we then be able to fix the issue by making sure 
that we check the happy-eyeballs-timeout both for timeouts and for errors? To 
me, that seems like a simpler take than your proposal.


I mean like this:

diff --git a/lib/connect.c b/lib/connect.c
index d9317f378..2c209fa56 100644
--- a/lib/connect.c
+++ b/lib/connect.c
@@ -931,18 +931,10 @@ CURLcode Curl_is_connected(struct Curl_easy *data,
  conn->timeoutms_per_addr[i]) {
 infof(data, "After %" CURL_FORMAT_TIMEDIFF_T
   "ms connect time, move on!\n", conn->timeoutms_per_addr[i]);
 error = ETIMEDOUT;
   }
-
-  /* should we try another protocol family? */
-  if(i == 0 && !conn->bits.parallel_connect &&
- (Curl_timediff(now, conn->connecttime) >=
-  data->set.happy_eyeballs_timeout)) {
-conn->bits.parallel_connect = TRUE; /* starting now */
-trynextip(data, conn, sockindex, 1);
-  }
 }
 else if(rc == CURL_CSELECT_OUT || conn->bits.tcp_fastopen) {
   if(verifyconnect(conn->tempsock[i], )) {
 /* we are connected with TCP, awesome! */

@@ -1002,10 +994,20 @@ CURLcode Curl_is_connected(struct Curl_easy *data,
conn->tempsock[other] == CURL_SOCKET_BAD)
   /* the last attempt failed and no other sockets remain open */
   result = status;
   }
 }
+
+if(!rc || error) {
+  /* should we try another protocol family? */
+  if(i == 0 && !conn->bits.parallel_connect &&
+ (Curl_timediff(now, conn->connecttime) >=
+  data->set.happy_eyeballs_timeout)) {
+conn->bits.parallel_connect = TRUE; /* starting now */
+trynextip(data, conn, sockindex, 1);
+  }
+}
   }

   if(result &&
  (conn->tempsock[0] == CURL_SOCKET_BAD) &&
  (conn->tempsock[1] == CURL_SOCKET_BAD)) {


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Happy Eyeballs mechanism may start from IPv4 family if c-ares resolver reply for IPv6 comes first.

[Daniel Stenberg via curl-library]

On Thu, 3 Jun 2021, Dmitry Karpov via curl-library wrote:


Here is the proposed fix for c-ares resolver that helped to mitigate the 
problem:


Thanks!

Any chance you can make a pull-request out of this and submit it at 
https://github.com/curl/curl/pulls ?


If not, I can do it for you.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

[Daniel Stenberg via curl-library]

On Thu, 3 Jun 2021, Shikha Sharma wrote:


Proposed solution:


Ah, yes that looks interesting - especially since it fixes the problem for 
you! Are you able to make a pull-request out of that and submit, or would you 
like me to do it for you? Thanks!


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

removing metalink support for security

[Daniel Stenberg via curl-library]

Hi all.

In the curl security-team we agree that the only sensible way out of a very 
troublesome security situation is to completely remove support for metalink 
from curl already in the next release.


See details here:

  https://github.com/curl/curl/pull/7176

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

The curl user survey 2021 (fwd)

[Daniel Stenberg via curl-library]

Hi friends,

Just a quick reminder that the survey is still up in case you haven't filled 
it in yet!


-- Forwarded message --
I've just launched the curl user survey 2021. Please consider donating some of
your precious time and head over there and fill in your views on curl:

  https://forms.gle/Z1ugr7tAGoiPyNTJ9

This survey will be up for 14 days. Please tell all your curl-using friends
about it and ask them to participate.

(Yeah, I will remind you a few times as well.)

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/

---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Feture window: OPEN

[Daniel Stenberg via curl-library]

Hi team,

We've had no alarming bug reports or regressions filed since the previous 
release which means we open the feature window today:


  https://curl.se/dev/feature-window.html

It will be open until June 23.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Travis isn't our friend anymore?

[Daniel Stenberg via curl-library]

Hi all,

Travis is shutting down their .org domain and pushes everyone over to their 
services on their .com domain. There's a new pricing model there. As an open 
source project we can apply to get into the free OSS tier.


However, the requirements to get accepted include for example that we cannot 
have anyone in the project (curl) getting paid to work on the project (curl).


Obviously, feeding your family is not open source enough for travis.

We either need to start paying for travis, or move the 33 jobs over to other 
services...


I filed an issue and collect details over here:

  https://github.com/curl/curl/issues/7150

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

[Daniel Stenberg via curl-library]

On Thu, 27 May 2021, Shikha Sharma via curl-library wrote:

HTTP2 connection with prior knowledge initiated by curl, No TLS. Server sends 
GoAway frame after processing each request.


I suspect the problem/bug here is that the request that reads the trailing 
GOAWAY isn't properly considered as a "dead connection" to get retried on a 
new connection.


Have you figured out a way to reproduce this issue against public URL with 
a small code snippet that you can share with us?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: WIN - Command Prompt - Curl: (77) error setting certificate verify locations

[Daniel Stenberg via curl-library]

On Thu, 27 May 2021, Arantxa Esparza via curl-library wrote:

Please show error messages as text, not as images. Some of us still read 
emails using text only.



I'm getting this error once and again and I don't know how to deal with it.

Curl: (77) error setting certificate verify locations: ..


Your curl image was built or is instructed to read a ca-cert file that causes 
OpenSSL to return error. Does it really exist and is readable there?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

[Daniel Stenberg via curl-library]

On Thu, 27 May 2021, Shikha Sharma via curl-library wrote:

I am facing an issue where curls responds with error "'Error in the HTTP2 
framing layer' (16)" to the application.


Curl version: libcurl-7.74.0

Scenario: Sending multiple outgoing HTTP2 requests to an external HTTP 
server.


I'm afraid this descripion lacks details to allow us to draw many conclusions. 
You also didn't mention if you use h2 over TLS or not and you didn't state 
which nghttp2 version you're using.


So it looks as if its a race condition where the client starts processing 
the new request and creates a stream on the existing connection before the 
goaway frame from the previous connection is processed. The request is 
ultimately dropped and application using libcurl receives the framing error.


If that's true then it sounds like a bug to me. We've fixed several h2 related 
bugs since 7.74.0 so I'd recommand you start out by trying the latest curl 
version and see if anything gets better.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl with custom malloc and SSL

[Daniel Stenberg via curl-library]

On Thu, 27 May 2021, Ville Likitalo wrote:


it was a build/linking related one instead.


Great, glad you found it and that it wasn't a curl bug! =)

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[SECURITY ADVISORY] curl: TLS session caching disaster

[Daniel Stenberg via curl-library]

TLS session caching disaster


Project curl Security Advisory, May 26th 2021 -
[Permalink](https://curl.se/docs/CVE-2021-22901.html)

VULNERABILITY
-

libcurl can be tricked into using already freed memory when a new TLS session
is negotiated or a client certificate is requested on an existing connection.
For example, this can happen when a TLS server requests a client certificate
on a connection that was established without one. A malicious server can use
this in rare unfortunate circumstances to potentially reach remote code
execution in the client.

OpenSSL can declare a "new session" for different reasons, including the
initial TLS handshake completion, TLS 1.2 (or earlier) renegotiation, or TLS
1.3 client certificate requests. When libcurl at run-time sets up support for
session ID caching on a connection using OpenSSL, it stores pointers to the
transfer in-memory object for later retrieval when OpenSSL considers a new
session to be established.

However, if the connection is used by multiple transfers (like with a reused
HTTP/1.1 connection or multiplexed HTTP/2 connection) that first transfer
object might be freed before the new session is established on that connection
and then the function will access a memory buffer that might be freed. When
using that memory, libcurl might even call a function pointer in the object,
making it possible for a remote code execution if the server could somehow
manage to get crafted memory content into the correct place in memory.

We are not aware of any exploit of this flaw.

INFO


The flaw can only happen in libcurl built to use OpenSSL (or one of its forks).

This flaw has existed in curl since commit
[a304051620b92](https://github.com/curl/curl/commit/a304051620b92) in libcurl
7.75.0, released on February 3, 2021.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2021-22901 to this issue.

CWE-416: Use After Free

Severity: High

## Steps to remote code execution

1. libcurl built to use OpenSSL (BoringSSL and libressl work the same)

2. A multi interface using application

3. One of the following:

 - create and use a first easy handle to do HTTP/1.1 over TLS to a malicious
   server

 - free that easy handle with `curl_easy_cleanup()`

 - create and use a second easy handle to do HTTP/1.1 over TLS with to the
   same server such that the TLS connection is reused

or

 - more than one concurrent easy handle created that do HTTP/2 over a TLS
   connection to a malicious server,

 - the *first* easy handle to use the connection must be freed with
   `curl_easy_cleanup()`

 - at least one easy handle remaining in use of the same connection

4. The attacking server needs to figure out heap address details in order to
know what payload contents to provide

5. The necessary exact memory address in the heap gets populated by memory
contents controlled by the server

6. The attacker starts a new handshake (on TLS 1.2 or earlier), or sends a TLS
1.3 client certificate request, or otherwise triggers OpenSSL to consider a
new session to be established

For a remote code execution, the client needs to perform (potentially many)
more transfers (and thus have more easy handles) to allow the server to place
crafted contents into heap memory.  Instead of remote code execution, the
client could instead crash or otherwise experience undefined behaviour.

AFFECTED VERSIONS
-

- Affected versions: curl 7.75.0 to and including 7.76.1
- Not affected versions: curl < 7.75.0 and curl >= 7.77.0

Also note that libcurl is used by many applications, and not always advertised
as such.

THE SOLUTION


When the transfer is detached from the connection, it clears the association
to it from the session ID cache logic.

A [fix for 
CVE-2021-22901](https://github.com/curl/curl/commit/7f4a9a9b2a49547eae24d2e19bc5c346e9026479)

RECOMMENDATIONS
--

 A - Upgrade curl to version 7.77.0

 B - Apply the patch to your local version

 C - Build libcurl to use another TLS backend

TIMELINE


This issue was reported to the curl project on April 29, 2021.

This advisory was posted on May 26, 2021.

CREDITS
---

This issue was reported by Harry Sintonen. Patched by Harry Sintonen and
Daniel Stenberg. Help and research by Brad Spencer.

Thanks a lot!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[SECURITY ADVISORY] curl: TELNET stack contents disclosure

[Daniel Stenberg via curl-library]

TELNET stack contents disclosure


Project curl Security Advisory, May 26th 2021 -
[Permalink](https://curl.se/docs/CVE-2021-22898.html)

VULNERABILITY
-

curl supports the `-t` command line option, known as `CURLOPT_TELNETOPTIONS`
in libcurl. This rarely used option is used to send variable=content pairs to
TELNET servers.

Due to flaw in the option parser for sending `NEW_ENV` variables, libcurl
could be made to pass on uninitialized data from a stack based buffer to the
server. Therefore potentially revealing sensitive internal information to the
server using a clear-text network protocol.

This could happen because curl did not check the return code from a
`sscanf(command, "%127[^,],%127s")` function invoke correctly, and would leave
the piece of the send buffer uninitialized for the value part if it was
provided longer than 127 bytes. The buffer used for this is 2048 bytes big and
the *variable* part of the *variable=content* pairs would be stored correctly
in the send buffer, making curl sending "interleaved" bytes sequences of stack
contents. A single curl TELNET handshake could then be made to send off a
total of around 1800 bytes of (non-contiguous) stack contents in this style:

[control byte]name[control byte]
stack contents
[control byte]name[control byte]
stack contents
...

An easy proof of concept command line looks like this:

curl telnet://example.com -tNEW_ENV=a,bb (256 'b's)

We are not aware of any exploit of this flaw.

INFO


This flaw has existed in curl since commit
[a1d6ad2610](https://github.com/curl/curl/commit/a1d6ad2610) in libcurl 7.7,
released on March 22, 2001.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2021-22898 to this issue.

CWE-457: Use of Uninitialized Variable

Severity: Medium

AFFECTED VERSIONS
-

- Affected versions: curl 7.7 to and including 7.76.1
- Not affected versions: curl < 7.7 and curl >= 7.77.0

Also note that libcurl is used by many applications, and not always advertised
as such.

THE SOLUTION


Use sscanf() properly and only use properly filled-in buffers.

A [fix for 
CVE-2021-22898](https://github.com/curl/curl/commit/39ce47f219b09c380b81f89fe54ac586c8db6bde)

RECOMMENDATIONS
--

 A - Upgrade curl to version 7.77.0

 B - Apply the patch to your local version

 C - Avoid using `CURLOPT_TELNETOPTIONS`

TIMELINE


This issue was reported to the curl project on April 27, 2021.

This advisory was posted on May 26, 2021.

CREDITS
---

This issue was reported and patched by Harry Sintonen.

Thanks a lot!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[SECURITY ADVISORY] curl: schannel cipher selection surprise

[Daniel Stenberg via curl-library]

schannel cipher selection surprise
==

Project curl Security Advisory, May 26th 2021 -
[Permalink](https://curl.se/docs/CVE-2021-22897.html)

VULNERABILITY
-

libcurl lets applictions specify which specific TLS ciphers to use in
transfers, using the option called `CURLOPT_SSL_CIPHER_LIST`. The cipher
selection is used for the TLS negotation when a transfer is done involving any
of the TLS based transfer protocols libcurl supports, such as HTTPS, FTPS,
IMAPS, POP3S, SMTPS etc.

Due to a mistake in the code, the selected cipher set was stored in a single
"static" variable in the library, which has the surprising side-effect that if
an application sets up multiple concurrent transfers, the last one that sets
the ciphers will accidentally control the set used by all transfers. In a
worst-case scenario, this weakens transport security significantly.

We are not aware of any exploit of this flaw.

INFO


This flaw has existed in libcurl since commit
[9aefbff30d280c60fc](https://github.com/curl/curl/commit/9aefbff30d280c60fc)
in libcurl 7.61.0, released on July 11, 2018.

It can only trigger when Schannel is used, which is the native TLS library in
Microsoft Windows.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2021-22897 to this issue.

CWE-488: Exposure of Data Element to Wrong Session

Severity: Low

AFFECTED VERSIONS
-

This issue only exists when libcurl is built to use Schannel.

- Affected versions: libcurl 7.61.0 to and including 7.76.1
- Not affected versions: libcurl < 7.61.0 and libcurl >= 7.77.0

Also note that libcurl is used by many applications, and not always advertised
as such.

THE SOLUTION


Store the cipher selection in data associated with the connection.

A [fix for 
CVE-2021-22897](https://github.com/curl/curl/commit/bbb71507b7bab52002f9b1e0880bed6a32834511)

RECOMMENDATIONS
--

If you're using an Schannel based libcurl, We suggest you take one of the
following actions immediately, in order of preference:

 A - Upgrade libcurl to version 7.77.0

 B - Apply the patch to your local version

 C - Avoid using `CURLOPT_SSL_CIPHER_LIST`

TIMELINE


This issue was reported to the curl project on April 23, 2021.

This advisory was posted on May 26, 2021.

CREDITS
---

This issue was reported by Harry Sintonen. Patch by Daniel Stenberg.

Thanks a lot!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

[RELEASE] curl 7.77.0

[Daniel Stenberg via curl-library]

 the unused 'overhead' function [35]
 o lib/hostip6.c: make NAT64 address synthesis on macOS work [135]
 o lib1564.c: enable last wakeup test part on Windows [26]
 o lib: fix 0-length Curl_client_write calls [60]
 o lib: fix some misuse of curlx_convert_UTF8_to_tchar [64]
 o libcurl-security.3: be careful of setuid [66]
 o libcurl-security.3: don't try to filter IPv4 hosts based on the URL [71]
 o libcurl.3: mention the URL API [76]
 o libssh2: fix Value stored to 'sshp' is never read [13]
 o libssh2: ignore timeout during disconnect [45]
 o libssh: fix "empty expression statement has no effect" warnings [7]
 o libtest: remove lib530.c [88]
 o m4: add security frameworks on Mac when compiling rustls [31]
 o multi: don't close connection HTTP_1_1_REQUIRED
 o multi: fix slow write/upload performance on Windows [27]
 o multi: reduce Win32 API calls to improve performance [28]
 o ngtcp2: fix the cb_acked_stream_data_offset proto [46]
 o NSS: add ciphers to map [30]
 o NSS: make colons, commas and spaces valid separators in cipher list [106]
 o nss_set_blocking: avoid static for sock_opt [72]
 o ntlm: precaution against super huge type2 offsets [65]
 o openldap: protect SSL-specific code with proper #ifdef [12]
 o openldap: replace ldap_ prefix on private functions [84]
 o openssl: fix build error with OpenSSL < 1.0.2 [4]
 o openssl: remove unneeded cast for CertOpenSystemStore() [93]
 o os400: additional support for options metadata [24]
 o progress: fix scan-build-11 warnings [92]
 o progress: reset limit_size variables at transfer start [114]
 o progress: when possible, calculate transfer speeds with microseconds [48]
 o README.md: delete Codacy UTM parameters [5]
 o Revert "Revert 'multi: implement wait using winsock events'" [26]
 o rustls: only return CURLE_AGAIN when TLS session is fully drained [2]
 o rustls: use ALPN [56]
 o sasl: use 'unsigned short' to store mechanism [112]
 o schannel: Disable auto credentials; add an option to enable it [18]
 o schannel: Support strong crypto option [44]
 o sectransp: allow cipher name to be specified [29]
 o sectransp: fix EXC_BAD_ACCESS caused by uninitialized buffer [136]
 o sigpipe: ignore SIGPIPE when using wolfSSL as well [70]
 o sockfilt: avoid getting stuck waiting for writable socket [80]
 o sockfilt: fix invalid increment of handles index variable nfd [79]
 o sws: #ifdef S_IFSOCK use [32]
 o sws: allow HTTP requests up to 2MB in size [100]
 o test server: take care of siginterrupt() deprecation [25]
 o test2100: make it run with and require IPv6 [127]
 o tests/disable-scan.pl: also scan all m4 files [17]
 o tests/getpart: generate output URL encoded for better diffs [128]
 o tests: ignore case of chunked hex numbers in tests [86]
 o tls: add USE_HTTP2 define [59]
 o tool_getparam: handle failure of curlx_convert_tchar_to_UTF8() [78]
 o tool_getparam: replace (in-place) '%20' by '+' according to RFC1866 [14]
 o tool_operate: don't discard failed parallel transfer result [16]
 o tool_writeout: fix the HTTP_CODE json output [11]
 o travis: disable the failing libssh build [94]
 o URL-SYNTAX: update IDNA section for WHATWG spec changes [74]
 o urlapi: "normalize" numerical IPv4 host names [6]
 o vauth: factor base64 conversions out of authentication procedures [22]
 o version: add gsasl_version to curl_version_info_data [43]
 o version: add OpenLDAP version in the output [110]
 o vtls: deduplicate some DISABLE_PROXY ifdefs [19]
 o vtls: reset ssl use flag upon negotiation failure [42]
 o wolfssl: handle SSL_write() returns 0 for error [68]
 o wolfssl: remove SSLv3 support leftovers [115]

This release includes the following known bugs:

 o see docs/KNOWN_BUGS (https://curl.se/docs/knownbugs.html)

This release would not have looked like this without help, code, reports and
advice from friends like these:

  3eka on github, Alessandro Ghedini, Andrew Barnert, Ayushman Singh Chauhan,
  Benjamin Riefenstahl, Blake Burkhart, Brad Spencer, Calvin Buckley,
  Cameron Cawley, Dan Fandrich, Daniel Carpenter, Daniel Gustafsson,
  Daniel Stenberg, David Cook, Denis Goleshchikhin, Dmitry Karpov,
  Dmitry Kostjuchenko, ebejan on github, Emil Engler, Georeth Zhou,
  Gergely Nagy, Gilles Vollant, Harry Sintonen, Howard Chu, Ikko Ashimine,
  Illarion Taev, Jacob Hoffman-Andrews, Jakub Zakrzewski, Javier Blazquez,
  J. Bromley, Jeroen Ooms, Joel Depooter, Joel Jakobsson, Johann150 on github,
  Jon Rumsey, Kamil Dudka, Kevin Burke, Kevin R. Bulgrien, Koichi Shiraishi,
  Lucas Clemente Vella, Lucas Servén Marín, MAntoniak on github, Marc Aldorasi,
  Marcel Raad, Marc Hörsken, Martin Dorey, Martin Halle, Matias N. Goldberg,
  Max Dymond, Michael Kolechkin, Michael O'Farrell, Michał Antoniak,
  Michal Rus, Morten Minde Neergaard, Oliver Urbann, Orgad Shaneh,
  Patrick Monnerat, Paweł Wegner, Peng-Yu Chen, Pontus Lundkvist, Radek Zajic,
  Ralph Langendam, Ray Satiro, rcombs on github, Rich FitzJohn,
  Ryan Beck-Buysse, Sergey Markelov, sergio-nsk on githu

Re: Valgrind Suppress error, in test cases. Warning on new testcases.

[Daniel Stenberg via curl-library]

On Tue, 25 May 2021, Gealber Morales via curl-library wrote:


Here I attached the file *valgrind1196 *which correspond to this test case.
I think the problem is with the *Curl_rand_hex *function, that Valgrind
goes crazy with it.
Maybe I'm not doing it correctly, so any correction is good to know. The
files *valgrind.pm  *and *valgrind.supp*, are in the
*tests* folder.


That looks like a set of false positives in OpenSSL. We don't see those in the 
CI and I don't see them in my dev machines.


It looks like errors we should suppress in Valgrind.


When I add new test cases, for example *test3018*, I'm getting this warning:
*Warning: test3018 not present in tests/data/Makefile.inc*
I checked this file *tests/data/Makefile.inc* is true that the test is not
present in the list of tests, but should I just add it to the list?


Yes. That's the canonical list of all test cases we have so you should add 
your new test files in that list.



I did it but that doesn't seems to be the solution.


You need to rerun autoreconf again so that tests/Makefile gets regenenerated 
and then it detects the file correctly.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: libcurl with custom malloc and SSL

[Daniel Stenberg via curl-library]

On Tue, 25 May 2021, Ville Likitalo via curl-library wrote:


The only interesting bit in the stack trace is that the pointer
which gets passed to libc free(void*) is one that has been
allocated with the private malloc libcurl has been set up with.

Now, I could not quickly find any existing open or closed bug
that would describe this but would this already be fixed in the
later versions?


Without debug symbols nor any way to reproduce this problem, I don't think we 
can tell for sure what the problem is or if it has been fixed already.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

The curl user survey 2021

[Daniel Stenberg via curl-library]

Hi friends!

I've just launched the curl user survey 2021. Please consider donating some of 
your precious time and head over there and fill in your views on curl:


  https://forms.gle/Z1ugr7tAGoiPyNTJ9

This survey will be up for 14 days. Please tell all your curl-using friends 
about it and ask them to participate.


(Yeah, I will remind you a few times as well.)

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: PR #7117 failing, help

[Daniel Stenberg via curl-library]

On Sun, 23 May 2021, Gealber Morales via curl-library wrote:


mqtt.c:200:3: error: implicit declaration of function 'bzero'
[-Wimplicit-function-declaration]

 200 |   bzero(packet, packetlen);


You musn't use bzero() because it isn't portable enough. Use memset() instead 
and there won't be any problems.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

#curl switches IRC network to libera.chat

[Daniel Stenberg via curl-library]

Hi team!

Due to turbulence on Freenode [1][2], we follow our friends and we are setting 
up our new camp on the https://libera.chat/ network.


The same channel name (#curl), same rules, mostly the same people.

Follow along, talk curl and related topics in a relaxed atmosphere. As much as 
you want. Or just idle and chill.


The docs is also updated accordingly: https://curl.se/docs/irc.html

[1] = 
https://www.vice.com/en/article/m7ev8y/freenode-open-source-korea-crown-prince-takeover
[2] = 
https://boingboing.net/2021/05/19/freenode-irc-staff-quit-after-new-owner-seizes-control.html


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl error messages in mail

[Daniel Stenberg via curl-library]

On Mon, 17 May 2021, R C via curl-library wrote:


I receive messages in  the root mail account (a deamon running as root) like:

curl_easy_perform() failed: Failure when receiving data from the peer
curl_easy_perform() failed: Failure when receiving data from the peer

or other error/warning messages. these are probably because something is 
being printed on stdout/stderr.  Is there a way, in curl, to prevent these 
message form being 'displayed' ?


Those message don't look like they're made by libcurl. They look like someone 
using libcurl makes them.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl w/OpenSSL - OCSP_CERTID hash choice

[Daniel Stenberg via curl-library]

On Sun, 16 May 2021, igorr+curl--- via curl-library wrote:


Am I missing something here?

If not, imvho, the "fix" in this particular case is somewhat involved -- for 
every OCSP_CERTID (#1) available in the stapled response, curl should 
construct its own OCSP_CERTID (#2) corresponding to the peer certificate 
based on the hash of #1 and use OCSP_resp_find_status() to locate the 
OCSP_CERTID in the response. And only after trying all of OCSP_CERTIDs in 
this fashion unsuccessfully should one reply with:


I'm not really updated with how OCSP stapling should be implemented so I'll 
just take your word for that this is a sound way to do it.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: curl_multi_remove_handle callbacks - debug mode?

[Daniel Stenberg via curl-library]

On Wed, 12 May 2021, Felipe Gasper via curl-library wrote:

	On a couple of occasions now I’ve chased down segfaults that it turned 
out were happening because a callback was freeing the multi.


	Is there any kind of “debug mode” in libcurl available that would 
warn, assert, or otherwise complain if something like that happens? I 
_think_ I have a fix in place that will prevent the issue moving forward, 
but that’s what I thought last time I fixed this problem. :-P


	If there’s not such a mode available, would that be a reasonable 
feature to add?


curl_multi_remove_handle doesn't "free the multi" though, so it sounds like 
you're mixing something here.


curl_multi_cleanup() will already not accept getting called from within a 
callback:


https://github.com/curl/curl/blob/dae382a1a1481a94b708c82d5aa9fa7253084160/lib/multi.c#L2555-L2557

curl_multi_remove_handle also tries to detect and avoid getting called from 
within a callback:


https://github.com/curl/curl/blob/dae382a1a1481a94b708c82d5aa9fa7253084160/lib/multi.c#L740-L741

... but since you write this, I assume something makes these conditional 
checks not be good enough. Can you figure out why? And if so, how would such 
an additional "debug check" you speak of work that isn't already checked for?


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: localhost to be truly local?

[Daniel Stenberg via curl-library]

On Mon, 10 May 2021, Geoff Beier wrote:

If this is the main goal, it seems useful to test all resolved addresses to 
see if they're loopback addresses, and flag them as a "secure context" if 
they are. That would both make sure the address returned when localhost is 
resolved is really local and let other aliases for loopback addresses be 
recognized that way.


It is at least *a* goal, not sure if it is the main one.

I have three separate reasons why I don't think we should flag secure context 
in run-time like that:


1. It opens up for trickery where the owner of the DNS decides whether
   a name is secure context. Once the user has used the name for a few years
   and thinks it will remain secure forever, it changes and bad things happen.

2. A huge point of my change is that you know by looking at the host name/URL
   if it is secure or not.

3. curl knows immediately if the context is secure without having to resolve
   the host name. There's no moment of not knowing. It makes things a lot
   easier to not have to rely on resolver responses for this.

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

localhost to be truly local?

[Daniel Stenberg via curl-library]

Hi,

I've created PR #7039 that makes "localhost" resolve to 127.0.0.1 and ::1 
without using the resolver [1].


The point of this is to make sure localhost is the local host for sure. With 
this, we should be able to consider transfers from localhost to be using a 
"secure context" as per web standards and for example allow 'secure' cookies 
even for 'http://localhost' [5].


Firefox already does this [2].

Chrome has a page [3] tracking its and others work on this and it says Edge 
already does this.


In Chrome's bug entry for this task [4], it sounds as if 'localhost' is 
already at least partially special-cased in Chrome code.


I've tried to find conclusive documentation on exactly how Windows deals with 
this. They started to resolve 'localhost' without it being present in their 
hosts file several years ago, but I've not found reliable source for this. I 
believe you can still put it in there and have it acknowledged.


curl's --resolve option and its libcurl counterpart still allows a user to 
make localhost URL's connect to another IP address, just like for any other 
name.


Your feedback and thoughts on this are most welcome!

[1] = https://github.com/curl/curl/pull/7039
[2] = https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts
[3] = https://www.chromestatus.com/feature/6269417340010496#details
[4] = https://bugs.chromium.org/p/chromium/issues/detail?id=589141
[5] = https://github.com/curl/curl/issues/6733

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: scan-build-11 warnings

[Daniel Stenberg via curl-library]

On Sat, 8 May 2021, Patrick Monnerat via curl-library wrote:


IMHO, this is not only a false positive, but a scan-build bug.

Mabe you can silent it by rephrasing the code. In example:


I tried a whole bunch of variations like that without success. Until I moved 
the entire speed calc logic into a separate function and then voila it is no 
longer "bad"! =)


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: scan-build-11 warnings

[Daniel Stenberg via curl-library]

On Sat, 8 May 2021, Daniel Stenberg via curl-library wrote:

For all I can tell, these are false positives. Can *you* find an actual 
problem and fix?


The warnings go away when I move the speed logic into a sub function, which 
also makes the code a little nicer:


 https://github.com/curl/curl/pull/7035

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

scan-build-11 warnings

[Daniel Stenberg via curl-library]

Hi friends!

I ran scan-build-11 against the current git master curl code and it generates 
two warnings that I just cannot understand and at the same time I cannot 
easily silence.


We use scan-build in the CI (travis), but that runs an older version that 
doesn't seem to warn for these issues. No other code analyzer has warned on 
these ones either.


For all I can tell, these are false positives. Can *you* find an actual 
problem and fix?


  https://daniel.haxx.se/scan-build-2021-05-08-115719/

scan-build offers no (good) way to individually switch off false positives!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Re: Crash libCurl 7.76.1 32-bit build in H3 transfers

[Daniel Stenberg via curl-library]

On Thu, 6 May 2021, Dmitry Karpov via curl-library wrote:

So "size_t datalen" in libCurl is different from "uint64_t datalen" in the 
ngtcp2 library.


Fix pending: https://github.com/curl/curl/pull/7027

The crash was so obvious, so it was a bit surprising how it was able to 
escape pre-release testing. Maybe H3, as experimental feature, wasn't tested 
enough on 32-builds?


We don't test h3 at all yet on any platform, I'm sorry to say! :-O

We still lack a test server/proxy for this purpose and we have at least three 
rather nasty-sounding HTTP/3 related bugs in KNOWN_BUGS: 
https://curl.se/docs/knownbugs.html#If_the_HTTP_3_server_closes_conn


I would prefer doing h3 tests with a h3 <=> h1 proxy front, a little like 
stunnel but for h3, so that we could still serve the core stuff with our 
existing h1 test server. But it's not a hard requirement.


I would love to get this going better and further but I've not had time for it 
lately and no one has stepped up wanting to sponsor my work on h3.


32-bit builds are still widely used on embedded devices, so it will help a 
lot if you guys look at this issue and look for other potential pitfalls 
like this when switching from 64-bit builds to 32-bit builds.


Once we get a few h3-tests landed, I think it would make sense and be good to 
have at least one set of them running on a 32 bit system, yes. The availablity 
of convenient CI builds on 32 bit systems might be a bit limited though.


--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Reminder: curl up 2021 on May 9th

[Daniel Stenberg via curl-library]

Hello!

This is just a quick reminder that we meet on Sunday May 9th at 13:00 CEST 
(11:00 UTC) for curl up 2021 [*]


We will run the meeting over Zoom, and you generally get the best experience 
by using a dedicated installed client but it is also possible to join with 
just a web browser.


There is right now 5 presentations for curl up that is already in the agenda 
and I suggest you go ahead and watch the ones there that you're intersted in, 
and then bring your questions and ideas to the meeting. (I might still add 
another presentation if I manage to record it before the weekend.)


If you have anything curl or network releated to present or discuss on Sunday, 
there's still time and room for you as well!


I will update the curl up wiki page with meeting details shortly before the 
meeting starts. And announce it in IRC.


Summary:

 Date:  May 9, 2021
 Time:  11:00 UTC
 Where: zoom meeting, details will be avaiable

[*] = https://github.com/curl/curl-up/wiki/2021

See you!

--

 / daniel.haxx.se
 | Commercial curl support up to 24x7 is available!
 | Private help, bug fixes, support, ports, new features
 | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html