Re: Guidance on monitoring connection open/close with CURLM API

2024-05-31 Thread Stefan Eissing via curl-library
> Am 31.05.2024 um 17:30 schrieb Stefan Eissing via curl-library > : > > > >> Am 31.05.2024 um 16:57 schrieb Cao Duc Quan : >> >> Hi Stefan, >> Thanks for the explanation. >> To conclude: >> - If the easy handle is not used with CU

Re: Guidance on monitoring connection open/close with CURLM API

2024-05-31 Thread Stefan Eissing via curl-library
> Am 31.05.2024 um 16:57 schrieb Cao Duc Quan : > > Hi Stefan, > Thanks for the explanation. > To conclude: > - If the easy handle is not used with CURLM, the connection will be closed > when we call curl_easy_cleanup() so I could say connection lifetime is the > same as easy handle lifetime.

Re: Guidance on monitoring connection open/close with CURLM API

2024-05-31 Thread Stefan Eissing via curl-library
> Am 31.05.2024 um 06:30 schrieb Cao Duc Quan via curl-library > : > > Hi, > > Here is the main flow in my application with CURLM > > struct Request { > CURL *easy; > // other data > } > > int on_close_socket(void* clientp, curl_socket_t sock) { > struct Request *request = (struct

Re: Increase in CPU usage in 8.7.1 vs 8.6.0 for rate-limited downloads

2024-05-15 Thread Stefan Eissing via curl-library
> Am 15.05.2024 um 08:56 schrieb David Pfitzner via curl-library > : > > Hi, > > I've noticed that from curl 8.6.0 to 8.7.1, the CPU usage when doing > rate-limited downloads with libcurl has increased significantly. This is most > noticeable when making multiple (rate-limited) downloads

Re: Setting up the user survey 2024

2024-05-07 Thread Stefan Eissing via curl-library
> Am 07.05.2024 um 10:27 schrieb Daniel Stenberg via curl-library > : > > Hello, > > I have started to work on the 2024 version of the user survey. If things go > well, we can run it two weeks later in May. > > As per usual, I copied the version from last year and I have edited it >

Re: HTTP/2 and server push related questions

2024-04-23 Thread Stefan Eissing via curl-library
> Am 23.04.2024 um 09:26 schrieb Abhinav Singhal via curl-library > : > > Hi, > I need to confirm two things: > > 1. If I build libcurl with the '--without-nghttp2' flag (and with > '--with-openssl'), will the http2 component be disabled in its entirety? Yes. > 2. Is

Re: Subtle H2 performance degradation in 8.7.1 vs 8.6.0

2024-04-01 Thread Stefan Eissing via curl-library
Hi Dmitry, we were discussing this in the project when I introduced the change. The advantage of writing HTTP/2 data frames directly is that curl does not buffering and less memory (way less in some cases). The disadvantage, as you noticed, is that applications are called much more often. We

Re: 8.7.1 : client read function EOF fail, only x/y of needed bytes read

2024-03-28 Thread Stefan Eissing via curl-library
> Am 28.03.2024 um 09:56 schrieb Matt Barnett : > > After many hours I found the issue and it's not a bug - just something that > we need to be aware of and change our code when using 8.7.1 vs 8.6.0 > > We had been using hd_src = fopen(LOCAL_FILE, "r") but it needs to be hd_src = >

Re: SIGSEGV in curl-8.6.0/lib/http2.c:288 (curl_multi_perform->extract_if_dead->http2_data_done)

2024-03-27 Thread Stefan Eissing via curl-library
> Am 26.03.2024 um 18:04 schrieb Stefan Eissing via curl-library > : > > > >> Am 26.03.2024 um 17:09 schrieb Aleksander Mazur via curl-library >> : >> >> Sorry for garbled gdb output, I hope this time it will look better: >> >> #0 0x000

Re: 8.7.1 : client read function EOF fail, only x/y of needed bytes read

2024-03-27 Thread Stefan Eissing via curl-library
> Am 27.03.2024 um 12:05 schrieb Matt Barnett via curl-library > : > > Hi All > > I have noticed a possible issue in 8.7.1... > > I am using libcurl within Visual Studio 2019 C++ on Win x64 to upload files > via SFTP > > With 8.6.0 all works great but when I compile the same project using

Re: SIGSEGV in curl-8.6.0/lib/http2.c:288 (curl_multi_perform->extract_if_dead->http2_data_done)

2024-03-26 Thread Stefan Eissing via curl-library
> Am 26.03.2024 um 17:09 schrieb Aleksander Mazur via curl-library > : > > Sorry for garbled gdb output, I hope this time it will look better: > > #0 0x77e4a6ba in http2_data_done (cf=0x794698, data=0x490878, > premature=true) at http2.c:288 > #1 0x77e4f1e9 in cf_h2_cntrl

Re: SIGSEGV in curl-8.6.0/lib/http2.c:288 (curl_multi_perform->extract_if_dead->http2_data_done)

2024-03-26 Thread Stefan Eissing via curl-library
> Am 26.03.2024 um 16:57 schrieb Aleksander Mazur via curl-library > : > > Dnia 2024-03-25, o godz. 22:33:13 Jeffrey Walton > napisał(a): > >> You have to build your application and cURL with -O0 or -O1. Valgrind >> becomes inaccurate at -O2 and above, and sometimes incorrectly reports >>

Re: SIGSEGV in curl-8.6.0/lib/http2.c:288 (curl_multi_perform->extract_if_dead->http2_data_done)

2024-03-21 Thread Stefan Eissing via curl-library
> Am 21.03.2024 um 09:28 schrieb Daniel Stenberg via curl-library > : > > On Wed, 20 Mar 2024, Aleksander Mazur via curl-library wrote: > >> Recently I observe my program crashing (sometimes) during system startup. >> (During system startup internet connection is down so a few first >>

Re: Speed throttling precision issues in the latest libcurl

2024-03-05 Thread Stefan Eissing via curl-library
HI Dmitry, > Am 05.03.2024 um 02:04 schrieb Dmitry Karpov via curl-library > : > > Hi All, > After running some speed throttling tests on the latest libcurl, I noticed > that the speed throttling precision got lost again. > > I reported this issue for 8.5.0, and it was fixed for 8.6.0 as

Re: are ALPN values desirable command line options?

2023-10-24 Thread Stefan Eissing via curl-library
> Am 24.10.2023 um 00:32 schrieb Stephen Farrell via curl-library > : > > > Hiya, > > On 23/10/2023 07:51, Daniel Stenberg wrote: >> Are there actually cases where sending in a custom ALPN is likely to >> be the right thing for a user to get a transfer going? I have not >> heard of any such

Re: are ALPN values desirable command line options?

2023-10-23 Thread Stefan Eissing via curl-library
> Am 22.10.2023 um 01:47 schrieb Stephen Farrell via curl-library > : > > > Hiya, > > This is a little long-winded before I get to the question. Sorry > about that;-) > > One of the things we'd like to see for ECH is that "smaller" web > sites and hosters can relatively easily make use of

Re: Getting a list of easy handles in a multi handle - possible?

2023-08-28 Thread Stefan Eissing via curl-library
> Am 28.08.2023 um 13:06 schrieb Patrick Monnerat via curl-library > : > > > On 8/28/23 12:51, Stefan Eissing via curl-library wrote: >> >>> Am 28.08.2023 um 12:41 schrieb Daniel Stenberg : >>> >>> On Mon, 28 Aug 2023, Stefan Eissing via

Re: Getting a list of easy handles in a multi handle - possible?

2023-08-28 Thread Stefan Eissing via curl-library
> Am 28.08.2023 um 12:41 schrieb Daniel Stenberg : > > On Mon, 28 Aug 2023, Stefan Eissing via curl-library wrote: > >>>>> CURL **handles = curl_multi_get_handles(multi); >> >> The tricky part is to handle iterations when easy handles are removed an

Re: Getting a list of easy handles in a multi handle - possible?

2023-08-28 Thread Stefan Eissing via curl-library
> Am 27.08.2023 um 20:35 schrieb Patrick Monnerat via curl-library > : > > > On 8/27/23 18:01, Daniel Stenberg via curl-library wrote: >> On Sun, 27 Aug 2023, Henrik Holst via curl-library wrote: >> >>> CURL **handles = curl_multi_get_handles(multi); >>> >>> where the last position in

Re: Curl segfault in curl_multi_perform

2023-07-31 Thread Stefan Eissing via curl-library
Thanks, can confirm that this test program crashes here as well. Investigating... > Am 31.07.2023 um 13:54 schrieb Richard W.M. Jones : > > > Could you try the attached program? This crashes 100% for me, same > stack trace as before. > > You will still need to set up an h2c Apache server

Re: Curl segfault in curl_multi_perform

2023-07-31 Thread Stefan Eissing via curl-library
Rich, thanks for the testing and adding more debug output. I added test cases here to try to reproduce your scenario in curl parallel transfers with an Apache, but everything works fine here so far. What would be helpful in analysing is the Connection and Transfer ID we introduced in curl

Re: Curl (or server) offers HTTP/2 but falls back to HTTP/1.1

2023-07-28 Thread Stefan Eissing via curl-library
Hi Rich, you see the ALPN handling in curl's log output. Specifically: >> * Connected to gemmei.ftp.acc.umu.se (2001:6b0:19::137) port 443 (#0) >> * ALPN: offers h2,http/1.1 This means that curl offers the server to talk 'h2' (preferred) or 'http/1.1'. This is as it is supposed to be. Then:

Re: Curl (or server) offers HTTP/2 but falls back to HTTP/1.1

2023-07-28 Thread Stefan Eissing via curl-library
Looks the same for me. It seems the server, Apache httpd, is not configured to use HTTP/2. > Am 28.07.2023 um 09:47 schrieb Richard W.M. Jones via curl-library > : > > curl-8.1.2-1.fc39.x86_64 > > I'm trying to force HTTP/2 to a particular server in my program (to > test multiplexing). For

Re: HTTP/2 deferring DATA frame to another TCP packet

2023-06-21 Thread Stefan Eissing via curl-library
> Am 21.06.2023 um 08:04 schrieb Pontakorn Prasertsuk via curl-library > : > > Hi, > > I am currently testing HTTP/2 connection to my local server using Curl i.e. > running: > > SSLKEYLOGFILE=/tmp/sslkeylog.log src/curl --location "https://localhost:8000; > --insecure --http2 --data

Re: CURLINFO_APPCONNECT_TIME_T in HTTP 2

2023-05-27 Thread Stefan Eissing via curl-library
Addendum: I speak of a recent curl version. If you use an older libcurl, please state the version. > Am 27.05.2023 um 12:00 schrieb Stefan Eissing via curl-library > : > > > >> Am 26.05.2023 um 23:22 schrieb Babacar Ndiaye via curl-library >> : >> >&

Re: CURLINFO_APPCONNECT_TIME_T in HTTP 2

2023-05-27 Thread Stefan Eissing via curl-library
> Am 26.05.2023 um 23:22 schrieb Babacar Ndiaye via curl-library > : > > Hi: > > The setup we have is TLS in TLS HTTPS with proxy client authentication. As > suggested in the docs, we create one multi handler, keep it around to > leverage the connection caching, and keep adding/removing

Re: Parallel curl testing project

2023-03-23 Thread Stefan Eissing via curl-library
Dan, very happy that you will work on this. I like to run the test suite locally before a large push and the time it takes on my machine is around 10 minutes. I'd very much appreciate that to go down! Kind Regards, Stefan > Am 22.03.2023 um 18:54 schrieb Dan Fandrich via curl-library > : >

Re: On more stable curl releases

2023-03-22 Thread Stefan Eissing via curl-library
> Am 22.03.2023 um 00:10 schrieb Daniel Stenberg via curl-library > : > > On Tue, 21 Mar 2023, Dan Fandrich via curl-library wrote: > >> Some examples of regressions would be the crash that prompted yesterday's >> 8.0.1 point release, the noproxy host matching bug #9842 (fixed in 7.86.0)

Re: Read idle time out

2023-03-17 Thread Stefan Eissing via curl-library
> Am 17.03.2023 um 11:25 schrieb Nithin Das via curl-library > : > > Hi, > > I am running into an issue where a curl client is timing out while > transferring a file to the server that uses lighttpd as the web server. It > looks like the timeout is because of the setting on the lighttpd

Re: Critique our use of curl; and HTTP/2, /3, multiplexing & pipelining questions

2023-02-22 Thread Stefan Eissing via curl-library
> Am 22.02.2023 um 17:01 schrieb Daniel Stenberg via curl-library > : > > On Tue, 21 Feb 2023, Richard W.M. Jones via curl-library wrote: > >> Firstly, I don't understand if the multi interface would actually help us >> here. Because nbdkit gives us lots of threads and expects an NBD

Re: performance tests

2023-02-10 Thread Stefan Eissing via curl-library
> Am 09.02.2023 um 23:37 schrieb Daniel Stenberg via curl-library > : > > On Thu, 9 Feb 2023, Fabian Keil via curl-library wrote: > >> In the mean time I'll keep an eye on the curl commits to see if anyone beats >> me to it ... > > Here's some brainstorming. > > We could start out

Re: changed behavior of CURLOPT_SSL_VERIFYPEER + CURLOPT_CAINFO_BLOB (libcurl 7.87.0)

2023-01-27 Thread Stefan Eissing via curl-library
Hi Paul, I may have accidentally killed that while refactoring the TLS backend IO handling. Sorry about that. I think we need a test case for that edge case. Kind Regards, Stefan > Am 27.01.2023 um 10:13 schrieb Paul Groke via curl-library > : > > Hi, > > we noticed a change in behavior

Re: curl_easy_upkeep : support sending HTTP/2 PING frame via multi interface

2023-01-16 Thread Stefan Eissing via curl-library
Hi Kalin, it is listed in the TODO as https://github.com/curl/curl/blob/6113dec2a829d4ab766428ccca9535b7a5efd012/docs/TODO#L482 but there are no plans yet when we get around to implement that. Kind Regards, Stefan > Am 16.01.2023 um 10:15 schrieb Kalin via curl-library > : > > Hello, > > I

Re: HTTP/3 options

2023-01-04 Thread Stefan Eissing via curl-library
> Am 04.01.2023 um 10:50 schrieb Daniel Stenberg : > > On Wed, 4 Jan 2023, Stefan Eissing wrote: > >> And then there are the cases where someone really wants to nail it down. >> That is what we are talking here, I guess. Ultimately, what is given on the >> command line is converted by curl

Re: HTTP/3 options

2023-01-04 Thread Stefan Eissing via curl-library
> Am 04.01.2023 um 09:08 schrieb Daniel Stenberg via curl-library > : > > On Tue, 3 Jan 2023, Timothe Litt via curl-library wrote: > > Thanks, this is certainly an interesting idea. > >> --http=3 - use http3, fall back to 2, 1.1, 1.0, 0.9, ... > > We don't use '='-separators in our option

Re: HTTPS records

2022-12-08 Thread Stefan Eissing via curl-library
> Am 08.12.2022 um 23:28 schrieb Daniel Stenberg via curl-library > : > > On Thu, 8 Dec 2022, Daniel Stenberg via curl-library wrote: > >> Sure. That should be fairly even even! The "struct ares_addrinfo" contains >> TTL data. > > Oops, I meant to say "fairly easy even". We require at

Re: feature window open

2022-11-01 Thread Stefan Eissing via curl-library
Nice! > Am 01.11.2022 um 13:15 schrieb Daniel Stenberg via curl-library > : > > Hello, > > The feature window is hereby open. > > -- > > / daniel.haxx.se > | Commercial curl support up to 24x7 is available! > | Private help, bug fixes, support, ports, new features > |

Re: credentials in memory

2022-09-30 Thread Stefan Eissing via curl-library
> Am 30.09.2022 um 13:49 schrieb Stefan Eissing via curl-library > : > > > >> Am 30.09.2022 um 13:41 schrieb Daniel Stenberg : >> >> On Fri, 30 Sep 2022, Stefan Eissing wrote: >> >>> I know of threee patterns to solve this probl

Re: credentials in memory

2022-09-30 Thread Stefan Eissing via curl-library
> Am 30.09.2022 um 13:41 schrieb Daniel Stenberg : > > On Fri, 30 Sep 2022, Stefan Eissing wrote: > >> I know of threee patterns to solve this problem (and increase usability as a >> side effect): > > Those methods transfer the data to another process, and that is certainly > even more

Re: credentials in memory

2022-09-30 Thread Stefan Eissing via curl-library
> Am 30.09.2022 um 09:43 schrieb Daniel Stenberg via curl-library > : > > Hi, > > I bring back an old discussion as I think it might be worth having it again. > > libcurl hold credentials (passwords for servers and proxies) in memory in > clear text, potentially for a long time. If

Re: a strlen update

2022-02-14 Thread Stefan Eissing via curl-library
Just a rough estimate: do you free() 2.5 times on average? > Am 14.02.2022 um 09:33 schrieb Daniel Stenberg via curl-library > : > > Other fun observations running this same command line: > > The top-25 lib calls done: > > 2239 free >635 malloc >545 memcpy >300 memchr >245

Re: drop TPF support?

2022-02-03 Thread Stefan Eissing via curl-library
> Am 03.02.2022 um 10:55 schrieb Daniel Stenberg via curl-library > : > > Hello team, > > Is anyone building or using a modern (lib)curl on TPF? > > Back in September 2010 we did the last "confirmed working" change to curl > that builds on TPF (commit 7e1a45e224e57). > > I did find the

Re: THe mailing lists glitched

2021-12-01 Thread Stefan Eissing via curl-library
\o/ > Am 01.12.2021 um 10:03 schrieb Daniel Stenberg via curl-library > : > > Hey all, > > I messed up the other day, and this mistake killed all the mailing lists on > the lists.haxx.se server. See https://github.com/curl/curl/issues/8073 > > If you see this mail, the service is at least

Re: relative performance

2021-08-24 Thread Stefan Eissing via curl-library
> Am 24.08.2021 um 17:20 schrieb Daniel Stenberg via curl-library > : > > Hi all, > > For a long time I've wanted to get something done that allows us to compare > curl's relative performance. Ideally something we can run every once in a > while to compare that nothing major has turned sour

Re: more WebSockets

2021-08-12 Thread Stefan Eissing via curl-library
> Am 12.08.2021 um 09:33 schrieb Stefan Eissing via curl-library > : > > One thing from rfc6455, ch. 5.4: > > "An intermediary MUST NOT change the fragmentation of a message if > any reserved bit values are used and the meaning of these values > is

Re: more WebSockets

2021-08-12 Thread Stefan Eissing via curl-library
One thing from rfc6455, ch. 5.4: "An intermediary MUST NOT change the fragmentation of a message if any reserved bit values are used and the meaning of these values is not known to the intermediary." which I read as: if you want to use libcurl as an intermediary, it needs to expose

Re: curl up 2021

2021-02-17 Thread Stefan Eissing via curl-library
\o/ > Am 17.02.2021 um 09:26 schrieb Daniel Stenberg via curl-library > : > > Hi team! > > curl up 2021 will of course again be online-only. I think it worked pretty > nicely last year so I propose we do it this: > > 1. We agree on a time for the event. > > 2. We come up with presentations

Re: On memory-leaks as security problems

2021-01-07 Thread Stefan Eissing via curl-library
Daniel, I'd appreciate if there were a definition, since in my work at apache security, we come across issues of categorisations often as well. D: is very clear: if someone mis-uses the API, it may trigger an update in documentation, but it is not a security issue. Same with a

Re: Thoughts on HSTS

2020-09-02 Thread Stefan Eissing via curl-library
Nice work, Daniel. In Apache I can preload certain servers, because it is pretty safe to assume that ACME servers do not migrated to http: But the persistence seems not usable. I cannot specify a file to libcurl, as process privileges will change during the lifetime of the server and also

Re: "curl_multi_unblock"

2019-10-10 Thread Stefan Eissing via curl-library
Thinking about how I might use the curl_multi_unblock(). It seems unreliable in the sense that a thread calling this cannot be sure that another thread is actually blocked. The other one might be about to enter a blocking state and the unblock is just a bit too early... So, threads who want to

Re: curl up 2020: May 9-10 in Berlin, Germany

2019-08-15 Thread Stefan Eissing via curl-library
Great Stuff! Certainly want to be there! > Am 14.08.2019 um 23:40 schrieb Daniel Stenberg : > > Hi friends, > > I'm happy to say that we have a country, a city, a date and a venue all set > for curl up 2020! > > Country: Germany > City: Berlin > Date: May 9-10, 2020 > Venue:

Re: Async thoughts

2019-07-16 Thread Stefan Eissing via curl-library
Feedback as one of the more recent multi-users: - once I read the documentation, it was quite easy to figure out X-) - I find the way of stuffing several easy things into one multi quite intuitive. - there are several use scenarios that can be described as "async" * mine was a pretty simple one

Re: connection pooling with http/2 when multiplexing to remote http1.1/http2 server

2019-05-29 Thread Stefan Eissing via curl-library
> Am 29.05.2019 um 02:21 schrieb Sangamkar, Dheeraj via curl-library > : > > I am using a libcurl client to GET content using http/2 over tls from an > http1.1/2 server. > I am using the multi interface. > CURLPIPE_WAIT is not set on the easy handles. > CURLMOPT_PIPELINING=MULTIPLEX on the

Re: Limit connection reuse to the first 120 seconds

2019-04-16 Thread Stefan Eissing via curl-library
Same behaviour for HTTP/2 connections? > Am 15.04.2019 um 22:37 schrieb Daniel Stenberg via curl-library > : > > Hi, > > I propose we change the connection reuse logic in curl to only ever consider > connections that have been kept in the connection pool for shorter than 120 > seconds.

Re: libcurl reconnect

2019-02-27 Thread Stefan Eissing via curl-library
Hei, I did some tests with the tool. Always nice to see someone digging into benchmarks. First of all, I can confirm that the --noreuse option makes things very slow. But, as I found out, it means you use a new connection for each request. This *should* be very slow as compared to reusing the

Re: Does curl validate the ":authority" header of HTTP/2 PUSH_PROMISE frames?

2019-02-14 Thread Stefan Eissing via curl-library
> Am 14.02.2019 um 15:26 schrieb Daniel Stenberg via curl-library > : > > Then the question remains if nghttp2 does the check for us, but browsed > around in that code for a while and I can't say I'm entirely sure of my > findings but I couldn't see that it checked for this. I'm afraid this

Re: only http2 mode?

2018-08-30 Thread Stefan Eissing via curl-library
g : > > On Wed, 29 Aug 2018, Stefan Eissing via curl-library wrote: > >> In Tests I'd like to make a curl command line requests that should only >> succeed for HTTP/2. Is that possible? '--no-http1.1' does not seem to have >> an effect. Or maybe I am holding it

only http2 mode?

2018-08-29 Thread Stefan Eissing via curl-library
In Tests I'd like to make a curl command line requests that should only succeed for HTTP/2. Is that possible? '--no-http1.1' does not seem to have an effect. Or maybe I am holding it wrong? Thanks, Stefan --- Unsubscribe: