Re: better handling of websocket protocol upgrades
On 27/07/15 15:49, Gisle Vanem wrote: Sorry I haven't followed this thread. So I don't know really what you're trying to accomplish. Is your intention to handle all the nitty-gritty details of WebSocket (RFC6455) yourself? So in the end make this possible? curl ws://echo.websocket.org/ curl wss://echo.websocket.org/ (secure WS) The idea was, to enable libcurl to conduct the websocket protocol upgrade (the HTTP part) and then yield the control to whatever comes next. So the established connection is nothing more than a bidirectional byte-transport layer (TCP/SSL). Nothing more. To generate/send and receive/interpret the Websocket-Frames, one has to use a library (libwebsocket?) or programm the framing by himself. My thoughts did not extend to the 'curl' command line tool, but maybe it would be possible to feed it the bytestream by stdin/stdout. I had a test of: https://libwebsockets.org/trac/libwebsocket Thanks allot for the hint. Since I do my http stuff with libcurl it was natural to try to adopt the websocket handling on top of libcurl. But maybe this is not the right approach at all. One problem with a library like libwebsocket could be that it might does not offer to set the HTTP parameters of the upgrade request on a low enough level for my purposes. But I will for sure have a look at the library. cheers, Frank --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: better handling of websocket protocol upgrades
Frank Meier wrote: consider yet to check the correctness of the Sec-WebSocket-Accept header. But of course there will be a minimum of checks that have to be performed before libcurl switches to websocket-mode. Sorry I haven't followed this thread. So I don't know really what you're trying to accomplish. Is your intention to handle all the nitty-gritty details of WebSocket (RFC6455) yourself? So in the end make this possible? curl ws://echo.websocket.org/ curl wss://echo.websocket.org/ (secure WS) I had a test of: https://libwebsockets.org/trac/libwebsockets some time ago. And I was truly amazed of how easy it was to build and run on Windows. An alternative? -- --gv --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: TLS1.2 from TLS1.0
On Mon, Jul 27, 2015 at 11:51 AM, Daniel Stenberg dan...@haxx.se wrote: On Mon, 27 Jul 2015, Praveen Pvs wrote: What version of TLS it would be using when i set SSL version to *CURL_SSLVERSION_TLSv1* Will it use only TLS1.0?? will it not auto-negotiates to highest i,e, 1.2 Yes it should, but it should even negotiate to TLS 1.2 by default even without that option set. The option is more for setting the lowest acceptable level. When server allowing only TLS1.2 and not allowing TLS1.0, the terminal is not able to connect to the server. Here is the trace provided by server team: How can we debug this? In your earlier mail, you have mentioned Upgrade to at least 7.34.0, then ask for CURL_SSLVERSION_TLSv1_2. And make sure you have a TLS lib (version) that speaks 1.2. Should we have TLS lib on the terminal that speaks to 1.2 with the current CURL library version which we are using?? Could you please help *Trace of failing handshake - when we don't allow TLS 1.0:* Notice how the client (Terminal) presents itself with ClientHello Version 3.1 = TLS1.0 14:29:03: New TCP connection #2: gbibp9ph1--blueice4n2.emea.ibm.com(33494) - 192.168.162.193(64443) 2 1 0.6400 (0.6400) CSV3.1(227) Handshake ClientHello Version 3.1 random[32]= 55 77 4a dd 14 85 79 55 04 b3 4c c3 2e 82 96 5e 85 db d8 e1 30 05 f8 7a c8 1d dd 18 d5 ff 23 4f cipher suites Unknown value 0xc014 Unknown value 0xc00a TLS_DHE_RSA_WITH_AES_256_CBC_SHA TLS_DHE_DSS_WITH_AES_256_CBC_SHA TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA Unknown value 0xc00f Unknown value 0xc005 TLS_RSA_WITH_AES_256_CBC_SHA TLS_RSA_WITH_CAMELLIA_256_CBC_SHA Unknown value 0xc012 Unknown value 0xc008 TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA Unknown value 0xc00d Unknown value 0xc003 TLS_RSA_WITH_3DES_EDE_CBC_SHA Unknown value 0xc013 Unknown value 0xc009 TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA Unknown value 0x9a Unknown value 0x99 Unknown value 0x45 Unknown value 0x44 Unknown value 0xc00e Unknown value 0xc004 TLS_RSA_WITH_AES_128_CBC_SHA Unknown value 0x96 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 Unknown value 0xc011 Unknown value 0xc007 Unknown value 0xc00c Unknown value 0xc002 TLS_RSA_WITH_RC4_128_SHA TLS_RSA_WITH_RC4_128_MD5 TLS_DHE_RSA_WITH_DES_CBC_SHA TLS_DHE_DSS_WITH_DES_CBC_SHA TLS_RSA_WITH_DES_CBC_SHA TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA TLS_RSA_EXPORT_WITH_DES40_CBC_SHA TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5 TLS_RSA_EXPORT_WITH_RC4_40_MD5 Unknown value 0xff compression methods unknown value NULL 2 2 0.6400 (0.) SCV3.1(2) Alert level fatal value handshake_failure 20.6401 (0.) SC TCP FIN 20.7433 (0.1032) CS TCP FIN *Trace of OK handshake when we allow TLS 1.0:* New TCP connection #1: gbibp9ph1--blueice4n2.emea.ibm.com(47957) - 192.168.162.193(64443) 1 1 0.8569 (0.8569) CSV3.1(227) Handshake ClientHello Version 3.1 random[32]= 55 77 48 87 e9 42 d3 1d 58 4a af 28 61 5b 49 02 0d 24 b1 60 1f fe 2c 22 e4 18 79 16 c9 ba 0b 81 cipher suites Unknown value 0xc014 Unknown value 0xc00a TLS_DHE_RSA_WITH_AES_256_CBC_SHA TLS_DHE_DSS_WITH_AES_256_CBC_SHA TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA Unknown value 0xc00f Unknown value 0xc005 TLS_RSA_WITH_AES_256_CBC_SHA TLS_RSA_WITH_CAMELLIA_256_CBC_SHA Unknown value 0xc012 Unknown value 0xc008 TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA Unknown value 0xc00d Unknown value 0xc003 TLS_RSA_WITH_3DES_EDE_CBC_SHA Unknown value 0xc013 Unknown value 0xc009 TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA Unknown value 0x9a Unknown value 0x99 Unknown value 0x45 Unknown value 0x44 Unknown value 0xc00e Unknown value 0xc004 TLS_RSA_WITH_AES_128_CBC_SHA Unknown value 0x96 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 Unknown value 0xc011 Unknown value 0xc007 Unknown value 0xc00c Unknown value 0xc002 TLS_RSA_WITH_RC4_128_SHA TLS_RSA_WITH_RC4_128_MD5 TLS_DHE_RSA_WITH_DES_CBC_SHA TLS_DHE_DSS_WITH_DES_CBC_SHA TLS_RSA_WITH_DES_CBC_SHA TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA
Re: TLS1.2 from TLS1.0
On Monday, July 27, 2015 22:15:44 Praveen Pvs wrote: not able to connect to the server. Here is the trace provided by server team: How can we debug this? In your earlier mail, you have mentioned Upgrade to at least 7.34.0, then ask for CURL_SSLVERSION_TLSv1_2. And make sure you have a TLS lib (version) that speaks 1.2. Should we have TLS lib on the terminal that speaks to 1.2 with the current CURL library version which we are using?? Could you please help Which version of libcurl did you upgrade to? Which TLS library are you using? Kamil --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: TLS1.2 from TLS1.0
On Tuesday, July 28, 2015 02:22:06 Praveen Pvs wrote: On Tue, Jul 28, 2015 at 1:36 AM, Kamil Dudka kdu...@redhat.com wrote: On Monday, July 27, 2015 22:15:44 Praveen Pvs wrote: not able to connect to the server. Here is the trace provided by server team: How can we debug this? In your earlier mail, you have mentioned Upgrade to at least 7.34.0, then ask for CURL_SSLVERSION_TLSv1_2. And make sure you have a TLS lib (version) that speaks 1.2. Should we have TLS lib on the terminal that speaks to 1.2 with the current CURL library version which we are using?? Could you please help Which version of libcurl did you upgrade to? Which TLS library are you using? We are using 7.21 version of libcurl. Not sure of TLS library, should we use specific version of TLS library? libcurl-7.21.0 is not aware of TLS 1.0. Unless your TLS library uses TLS 1.2 by default (which is apparently not the case), you need to upgrade libcurl as Daniel already told you in the first reply. Kamil --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: TLS1.2 from TLS1.0
On Tue, Jul 28, 2015 at 1:36 AM, Kamil Dudka kdu...@redhat.com wrote: On Monday, July 27, 2015 22:15:44 Praveen Pvs wrote: not able to connect to the server. Here is the trace provided by server team: How can we debug this? In your earlier mail, you have mentioned Upgrade to at least 7.34.0, then ask for CURL_SSLVERSION_TLSv1_2. And make sure you have a TLS lib (version) that speaks 1.2. Should we have TLS lib on the terminal that speaks to 1.2 with the current CURL library version which we are using?? Could you please help Which version of libcurl did you upgrade to? Which TLS library are you using? We are using 7.21 version of libcurl. Not sure of TLS library, should we use specific version of TLS library? Kamil --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: better handling of websocket protocol upgrades
On 25/07/15 01:03, Daniel Stenberg wrote:
If you insist on using libcurl for this, then I think that is at least
a way you _can_ do it.
Yes, the benefit of using curl in this case, only to establish the
connection, is really small. That's why I tried to find/implement a
solution were the functionality of libcurl could be used for more than that.
I haven't fully looked through the code path for this approach but it
seems reasonable. And even if you made it work, it's not really a
documented way to do it and we don't test for it so it would be a
rather fragile method that could break in a future release.
I agree, this would be more a hack than a nice solution. If we go this
direction I think there should be a documented way to detach the
easy-handle from the multi-handle, without severing the underling
connection. After that the easy-handle could be freely used with
curl_easy_send() und _recv().
An Idea that comes to my mind would be to add a curl_easy_option that
let curl do the HTTP header send/receive part and then yields the
control to the application, similarly to CURLOPT_CONNECT_ONLY.
It'd need to be more than that (like for example we always need to
wait for the socket to be BOTH readable and writable since we don't
know the protocol and that seems like a hard nut to crack or you need
to signal that somehow)
I don't understand. Isn't it good enough to have the the KEEP_RECV/SEND
flags set? The sending is controlled by the READFUNCTION that could
return READFUNC_PAUSE if no data has to be sent. And the WRITEFUNCTION
is just called if there is data available from the server. This is
really low level, all the websocket protocol and framing stuff has to be
implemented in the application itself.
and I'm also interested in how you imagine the API for
enabling/controlling this. I'm slightly scared that allowing this is
the first step on a slippery slope and that we will soon find out that
it isn't enough to do good websockets anyway unless we also do
something else that is more intrusive.
In this first approach I thought about it to be controlled by the
availability of the Upgrade: websocket header in the request (see new
patch). But maybe it would be better to enable it explicitly with a
config option. So far I also didn't consider yet to check the
correctness of the Sec-WebSocket-Accept header. But of course there
will be a minimum of checks that have to be performed before libcurl
switches to websocket-mode.
After some more tinkering I also think after the protocol switch is
performed, the conn-handler should be switched to something simpler
that only receives and sends data back and forth on the socket (similiar
to telnet). This way the impact on existing code should be very limited.
I'm standing at crossroads. I'd be willing to put effort into this, but
I need guidance to know what could be done, and what would be accepted.
We are talking about quite different approaches. The first is more a
workaround to takeover the connection, whereas the other aims at a
naturally integrated solution.
Also my knowhow of the internals of libcurl might not be sufficient to
work out the solution in the needed quality by myself.
And I'll again mention that we need test cases if we are to consider
doing this.
I completely agree. I'd be willing to provide those too as far I can
manage, but might need some help with this too.
cheers, Frank
diff --git a/lib/http.c b/lib/http.c
index a1eef81..db2d91c 100644
--- a/lib/http.c
+++ b/lib/http.c
@@ -2350,6 +2350,11 @@ CURLcode Curl_http(struct connectdata *conn, bool *done)
if(result)
return result;
}
+ else if((ptr=Curl_checkheaders(conn, Upgrade:))) {
+if(Curl_compareheader(ptr, Upgrade:, websocket)) {
+ data-req.upgr101 = UPGR101_WS_REQUESTED;
+}
+ }
#if !defined(CURL_DISABLE_COOKIES)
if(data-cookies || addcookies) {
@@ -2942,6 +2947,7 @@ CURLcode Curl_http_readwrite_headers(struct SessionHandle *data,
{
CURLcode result;
struct SingleRequest *k = data-req;
+ const char *ptr;
/* header line within buffer loop */
do {
@@ -3035,16 +3041,6 @@ CURLcode Curl_http_readwrite_headers(struct SessionHandle *data,
#endif /* CURL_DOES_CONVERSIONS */
if(100 = k-httpcode 199 = k-httpcode) {
-/*
- * We have made a HTTP PUT or POST and this is 1.1-lingo
- * that tells us that the server is OK with this and ready
- * to receive the data.
- * However, we'll get more headers now so we must get
- * back into the header-parsing state!
- */
-k-header = TRUE;
-k-headerline = 0; /* restart the header line counter */
-
/* A user agent MAY ignore unexpected 1xx status responses. */
switch(k-httpcode) {
case 100:
@@ -3056,10 +3052,9 @@ CURLcode Curl_http_readwrite_headers(struct SessionHandle *data,
break;
case 101:
/* Switching Protocols */
- if(k-upgr101
Re: TLS1.2 from TLS1.0
On Jul 27, 2015, at 2:52 PM, Praveen Pvs meetpraveen...@gmail.com wrote: should we use specific version of TLS library? Yes, you will indeed need a recent enough version to support TLSv1.2. If you are using a *nix system, you are probably using OpenSSL or LibreSSL as your TLS library. If this is the case, the command openssl version should display the exact version. In OpenSSL, support for TLSv1.2 was added in version 1.0.1, so any version prior to that will not be able to provide TLSv1.2 facilities for curl. In fact, you should be able to simply upgrade your OpenSSL library without any other changes to your curl version or configuration, and curl should then be able to connect to your TLSv1.2-only server. Since you say the server is already enforcing the TLSv1.2 requirement, all you really need is for the SSL/TSL library used by curl to support TLSv1.2 - there is no particular requirement to upgrade and configure curl so as to force TLSv1.2 from its end. -Aaron--- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
Re: TLS1.2 from TLS1.0
On Mon, 27 Jul 2015, Praveen Pvs wrote: What version of TLS it would be using when i set SSL version to *CURL_SSLVERSION_TLSv1* Will it use only TLS1.0?? will it not auto-negotiates to highest i,e, 1.2 Yes it should, but it should even negotiate to TLS 1.2 by default even without that option set. The option is more for setting the lowest acceptable level. -- / daniel.haxx.se --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
