Re: how to make requests of http2 using curl

[Ray Satiro via curl-library]

On 12/17/2019 2:26 AM, Daniel Stenberg via curl-library wrote:

On Tue, 17 Dec 2019, peng xu via curl-library wrote:

I have wriiten the code below to make a http2 request, but the debug 
info showed me that it's http 1.1, would you please tell me why and 
how should I amend it:


Does your libcurl support HTTP/2 to begin with?


?? {
?? curl_version_info_data *info = curl_version_info(CURLVERSION_NOW);
?? printf("HTTP2 is %s\n", ((info->features & CURL_VERSION_HTTP2) ? 
"supported" : "NOT supported"));

?? printf("%s\n", curl_version());
?? }

---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Re: how to make requests of http2 using curl

[Daniel Stenberg via curl-library]

On Tue, 17 Dec 2019, peng xu via curl-library wrote:


curl_multi_setopt(easy_handle, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);


curl_multi_setopt() needs a *multi* handle in its first argument, not an easy 
handle. This is therefore a very bad line of code.


You can only do multiplexing with the multi interface in curl, since the easy 
interface makes a single transfer synchronously all the way until completed 
it can't multiplex.


--

 / daniel.haxx.se | Get the best commercial curl support there is - from me
  | Private help, bug fixes, support, ports, new features
  | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Re: how to make requests of http2 using curl

[Daniel Stenberg via curl-library]

On Tue, 17 Dec 2019, peng xu via curl-library wrote:

I have wriiten the code below to make a http2 request, but the debug info 
showed me that it's http 1.1, would you please tell me why and how should I 
amend it:


Does your libcurl support HTTP/2 to begin with?

--

 / daniel.haxx.se | Get the best commercial curl support there is - from me
  | Private help, bug fixes, support, ports, new features
  | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

how to make requests of http2 using curl

[peng xu via curl-library]

I have wriiten the code below to make a http2 request, but the debug info 
showed me that it's http 1.1, would you please tell me why and how should I 
amend it:
curl_global_init(CURL_GLOBAL_ALL);
CURL* easy_handle = curl_easy_init();
struct curl_slist* head = NULL;
char *form_data = "Content-Disposition: form-data; name=\"metadata\"";
head = curl_slist_append(head, form_data);
head = curl_slist_append(head, "Content-Type:application/json;charset=UTF-8");
curl_easy_setopt(easy_handle, CURLOPT_HTTPHEADER, head);
//curl_easy_setopt(easy_handle, CURLOPT_URL, "ai-test.xfyousheng.com:91");
curl_easy_setopt(easy_handle, CURLOPT_URL, "https://nghttp2.org/;);
curl_multi_setopt(easy_handle, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);
curl_easy_setopt(easy_handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_2_0);


curl_easy_setopt(easy_handle, CURLOPT_POSTFIELDS, request_json->data);
curl_easy_setopt(easy_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
data_chunk_t chunk;
chunk.data = malloc(1);  /* will be grown as needed by the realloc above */
chunk.size = 0;/* no data at this point */
curl_easy_setopt(easy_handle, CURLOPT_WRITEDATA, );
curl_easy_setopt(easy_handle, CURLOPT_SSL_VERIFYPEER, 0L);//忽略证书检查
curl_easy_setopt(easy_handle, CURLOPT_SSL_VERIFYHOST, 0L);
/* please be verbose */
curl_easy_setopt(easy_handle, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(easy_handle, CURLOPT_DEBUGFUNCTION, my_trace);
CURLcode code;
code = curl_easy_perform(easy_handle);
long retcode = 0;
code = curl_easy_getinfo(easy_handle, CURLINFO_RESPONSE_CODE, );---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Re: Connection re-use w.r.t HTTP/1.1 and HTTP/2.0

[Kunal Ekawde via curl-library]

On Mon, Dec 16, 2019 at 6:17 PM Daniel Stenberg  wrote:
>
> On Mon, 16 Dec 2019, Kunal Ekawde via curl-library wrote:
>
> > Connection re-use has been discussed earlier in some context but I wanted to
> > know more with respect to recent modifications.
>
> I don't think there have been any recent modifications that changed connection
> reuse.
>
> > curl_multi_setopt(multi_handle_m, CURLMOPT_MAXCONNECTS, maxConnects);
>
> This is the size of the multi handle's connection cache. Basically that is the
> max number of connections that can be kept alive after a request has completed
> - but it should be noted that also running connections are part of the cache
> so if MAXCONNECTS is smaller than the number of active transfers, there will
> be no idle connections left in the cache after a transfer (thus effectively
> killing connection reuse).
>
> > curl_multi_setopt(multi_handle_m, CURLMOPT_MAX_HOST_CONNECTIONS,
> > maxConnectsPerHost);
>
> This limits the number of connections this handle will use to the same host
> name.
>
> > curl_multi_setopt(multi_handle_m, CURLMOPT_MAX_TOTAL_CONNECTIONS,
> > maxParallelConnects);
>
> This limits the total number of *active* connections the multi handle will
> use. Attempting to create more, with additional transfers, will put those
> transfer in an internal waiting queue, waiting for a connection "slot" to
> become available.
>
> > curl_multi_setopt(multi_handle_m, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);
>
> This is the default since 7.62.0, it enables multiplexing if curl finds that
> it can. Like when doing HTTP/2 to the same host with two concurrent requests.
>
> > For each transfer:
> > curl_easy_setopt(easy_handle, CURLOPT_PIPEWAIT, 1L);
>
> This makes transfers prefer to (wait and) multiplex on an existing connection
> rather than immediately start a separate new connection, if that situation
> arises.
>
> > Desired behavior in HTTP/2 for me would be to re-use the same connection
> > until some limit on requests on a connection,
>
> Why would you limit this to something else than the underlying maximum number
> of streams allowed? The limit we provide for that is the one *you* implemented
> and is provided since 7.67.0: CURLMOPT_MAX_CONCURRENT_STREAMS.
>
> > something like may be max concurrent streams and only then create new
> > connection based on 'CURLMOPT_MAX_HOST_CONNECTIONS'.
>
> Right: CURLMOPT_MAX_CONCURRENT_STREAMS.
>

Yes, this is desired behavior, basically that the connection is fully
utilized to its multiplexing limit
before new one is created towards that host, but I don't see it,
rather, it creates new connection to
host for a new stream if 'CURLMOPT_MAX_HOST_CONNECTIONS' limit not
reached yet. e.g.

Given that: CURLMOPT_MAX_HOST_CONNECTIONS = 100
Request 1 -> Stream1 -- New connection #0
Request 2 -> Stream2 -- New connection #1

Desired:
Request 1 -> Stream1 -- New connection #0
Request 2 -> Stream2 -- Re-use connection #0

> > Current observation is that each new request/transfer starts a new
> > connection upto 'CURLMOPT_MAX_HOST_CONNECTIONS' and then re-uses
> > those.
>
> Yes exactly. Isn't that also what the documentation says?
>
> > For HTTP/1.1 desired would be to pipe the requests upto certain limits and
> > then create another connection.
>
> libcurl doesn't support Pipelining (anymore) so it will never pipeline, but it
> will reuse connections whenever it can.
>
> --
>
>   / daniel.haxx.se | Get the best commercial curl support there is - from me
>| Private help, bug fixes, support, ports, new features
>| https://www.wolfssl.com/contact/



-- 
~Kunal
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Re: [poll comment] "grpc"

[Daniel Stenberg via curl-library]

On Mon, 16 Dec 2019, Rémy Léone wrote:

gRPC got a URL system. For instance if you have the method myMethod in the 
foo.bar package the URL would be:


https://endpoint/foo.bar/myMethod


That's an HTTPS URL, not a gRPC URL. To me, that's a sign gRPC should be done 
*using* libcurl, not by libcurl itself.


--

 / daniel.haxx.se | Get the best commercial curl support there is - from me
  | Private help, bug fixes, support, ports, new features
  | https://www.wolfssl.com/contact/---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Re: [poll comment] "grpc"

[Rémy Léone via curl-library]

Hello,

gRPC got a URL system. For instance if you have the method myMethod in the 
foo.bar package the URL would be:

https://endpoint/foo.bar/myMethod

From a protocol perspective, all gRPC calls are HTTP2 POST on a specific URL 
such as the one previously shown

https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md 

.
The content of the body is a binary encoded protocol buffer.

Best regards

Rémy 

> No dia 27/08/2019, às 16:06, Daniel Stenberg via curl-library 
>  escreveu:
> 
> Hi,
> 
> Someone mentioned "grpc" as a suggested feature to work on in curl going 
> forward.
> 
> I don't see the fit. I don't think grpc even has its own URL syntax. If 
> someone wants to argue for this, please help me understand.
> 
> -- 
> 
> / daniel.haxx.se | Get the best commercial curl support there is - from me
>  | Private help, bug fixes, support, ports, new features
>  | https://www.wolfssl.com/contact/
> ---
> Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
> Etiquette:   https://curl.haxx.se/mail/etiquette.html

---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Re: Connection re-use w.r.t HTTP/1.1 and HTTP/2.0

[Daniel Stenberg via curl-library]

On Mon, 16 Dec 2019, Kunal Ekawde via curl-library wrote:

Connection re-use has been discussed earlier in some context but I wanted to 
know more with respect to recent modifications.


I don't think there have been any recent modifications that changed connection 
reuse.



curl_multi_setopt(multi_handle_m, CURLMOPT_MAXCONNECTS, maxConnects);


This is the size of the multi handle's connection cache. Basically that is the 
max number of connections that can be kept alive after a request has completed 
- but it should be noted that also running connections are part of the cache 
so if MAXCONNECTS is smaller than the number of active transfers, there will 
be no idle connections left in the cache after a transfer (thus effectively 
killing connection reuse).



curl_multi_setopt(multi_handle_m, CURLMOPT_MAX_HOST_CONNECTIONS,
maxConnectsPerHost);


This limits the number of connections this handle will use to the same host 
name.



curl_multi_setopt(multi_handle_m, CURLMOPT_MAX_TOTAL_CONNECTIONS,
maxParallelConnects);


This limits the total number of *active* connections the multi handle will 
use. Attempting to create more, with additional transfers, will put those 
transfer in an internal waiting queue, waiting for a connection "slot" to 
become available.



curl_multi_setopt(multi_handle_m, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);


This is the default since 7.62.0, it enables multiplexing if curl finds that 
it can. Like when doing HTTP/2 to the same host with two concurrent requests.



For each transfer:
curl_easy_setopt(easy_handle, CURLOPT_PIPEWAIT, 1L);


This makes transfers prefer to (wait and) multiplex on an existing connection 
rather than immediately start a separate new connection, if that situation 
arises.


Desired behavior in HTTP/2 for me would be to re-use the same connection 
until some limit on requests on a connection,


Why would you limit this to something else than the underlying maximum number 
of streams allowed? The limit we provide for that is the one *you* implemented 
and is provided since 7.67.0: CURLMOPT_MAX_CONCURRENT_STREAMS.



something like may be max concurrent streams and only then create new
connection based on 'CURLMOPT_MAX_HOST_CONNECTIONS'.


Right: CURLMOPT_MAX_CONCURRENT_STREAMS.


Current observation is that each new request/transfer starts a new
connection upto 'CURLMOPT_MAX_HOST_CONNECTIONS' and then re-uses
those.


Yes exactly. Isn't that also what the documentation says?

For HTTP/1.1 desired would be to pipe the requests upto certain limits and 
then create another connection.


libcurl doesn't support Pipelining (anymore) so it will never pipeline, but it 
will reuse connections whenever it can.


--

 / daniel.haxx.se | Get the best commercial curl support there is - from me
  | Private help, bug fixes, support, ports, new features
  | https://www.wolfssl.com/contact/
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Connection re-use w.r.t HTTP/1.1 and HTTP/2.0

[Kunal Ekawde via curl-library]

Hi All,

Connection re-use has been discussed earlier in some context but I
wanted to know more with respect to recent modifications.

I've use case of both HTTP/1.1 and HTTP/2 transfers and the settings
are same to both as it applies to same multi handle.
curl_multi_setopt(multi_handle_m, CURLMOPT_MAXCONNECTS, maxConnects);
curl_multi_setopt(multi_handle_m, CURLMOPT_MAX_HOST_CONNECTIONS,
maxConnectsPerHost);
curl_multi_setopt(multi_handle_m, CURLMOPT_MAX_TOTAL_CONNECTIONS,
maxParallelConnects);
curl_multi_setopt(multi_handle_m, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);
For each transfer:
curl_easy_setopt(easy_handle, CURLOPT_PIPEWAIT, 1L);

Desired behavior in HTTP/2 for me would be to re-use the same
connection until some limit on requests on a connection,
something like may be max concurrent streams and only then create new
connection based on 'CURLMOPT_MAX_HOST_CONNECTIONS'.
Current observation is that each new request/transfer starts a new
connection upto 'CURLMOPT_MAX_HOST_CONNECTIONS' and then re-uses
those.

For HTTP/1.1 desired would be to pipe the requests upto certain limits
and then create another connection.

I'm currently using 7.66.0 version, request to please shed some light on this.

Thanks,
~Kunal
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html