Ian Hickson wrote:
On Wed, 5 Aug 2009, Amos Jeffries wrote:
On Wed, 5 Aug 2009 00:57:18 +0000 (UTC), Ian Hickson <i...@hixie.ch> wrote:
On Thu, 30 Jul 2009, Robert Collins wrote:
On Wed, 2009-07-29 at 23:48 +0000, Ian Hickson wrote:
Surely once the HTTP server has decided that it can Upgrade, it
doesn't actually need to worry about sending back something
HTTP-valid at all, since we can just say the entire connection was
always using the WebSocket protocol, and was never HTTP.
10.1.2 101 Switching Protocols
The server understands and is willing to comply with the client's
request, via the Upgrade message header field (section 14.42), for a
change in the application protocol being used on this connection. The
server will switch protocols to those defined by the response's
Upgrade header field immediately after the empty line which
terminates the 101 response.
So the server has to send a response before switching.
The above text has no normative conformance criteria ("the server
will" is an informative statement that is unsupported as far as I can
tell).
Point to note: if webserver fails to switch the WebSockets handshake to
verify said switch actually occurred will fail. Detecting such bad servers
and acting properly.
If web server successfully switched then WebSockets client needs to handle
the HTTP response correctly and begin to do a WebSockets native handshake
and verify the link is safe _after_ the 101 message is received.
I don't understand what you are trying to say here.
In any case, the WebSocket protocol handshake does do a complete HTTP
Upgrade including the empty line, it just requires that the handshake
be done in a particular way. It's "profiling" the HTTP spec.
No, our whole point repeated over and over in different ways is that the
specd description for the upgrade request is _incomplete_. And does not
account for an enormous number of byte-level variations. All of which
can be safely discarded without affecting security.
Being sensitive to whether the server replies "101 Blah" versus "101
blah" absolutely cripples WebSockets. We want to help you fix this
problem.
I still do not understand why anything gets crippled. Maybe you could show
an example of how you expect this problem to occur?
Your protocol definition used byte-level. At the byte-level 'b' (0x97
IIRC) does not equal 'B' (0x65 IIRC). Thus the response is a different
byte pattern and a failed WebSocket connection.
One very real example of this would be the web server or an fully
WebSocket capable intermediary sending back bytes
Your spec section 3.1 sub 12 says:
"
12. Read the first 85 bytes from the server. If the connection
closes before 85 bytes are received, or if the first 85 bytes
aren't exactly equal to the following bytes, then fail the Web
"
[ note the words _exactly equal_ ]
"
Socket connection and abort these steps.
48 54 54 50 2f 31 2e 31 20 31 30 31 20 57 65 62
20 53 6f 63 6b 65 74 20 50 72 6f 74 6f 63 6f 6c
20 48 61 6e 64 73 68 61 6b 65 0d 0a 55 70 67 72
61 64 65 3a 20 57 65 62 53 6f 63 6b 65 74 0d 0a
43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 55 70 67 72
61 64 65 0d 0a
"
example #1 suppose there was an intermediary translating
websockets-over-http to websockets-port-81 which used HTTP to format
said headers of confirmation.
In all other ways it is fully WebSockets compliant. But sends byte 18 as
73 (s) instead of 53 (S).
Boom! The entire application is not WebSockets compliant and will fail
every single transaction that goes through it.
example #2 is where the traffic is processed by an HTTP-only
intermediary which sees the 'Upgrade:' header and flags the connection
for transparent pass-thru (This by the way is the desirable method of
making Squid support WebSockets).
Being a good HTTP relay it accepts these bytes:
HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
It violates HTTP by omitting the Via and other headers your spec omits
to handle. And passes these on:
HTTP/1.1 101 Web Socket Protocol Handshake
Connection: Upgrade
Upgrade: WebSocket
then moves to tunnel mode for you.
Bang. Another whole network isolated from WebSockets.
This is because if it doesn't, how can it say 'I won't upgrade'.
It can just not upgrade. Returning anything but the correct handshake
will be treated as a failed connection by the WebSocket client.
Correct.
So why would it need to say "I won't upgrade"?
To inform MITM that the upgrade is not going to happen and the links
they have open maybe used for other HTTP things without wasting network
resources tearing them down and rebuilding.
On Thu, 30 Jul 2009, Robert Collins wrote:
Suppose we had no handshake at all, and that there was no data
framing, so that as soon as we connected to a port, we could send
arbitrary data down.
A Web page, say evil.example.net, could open a Web Socket
connection to http://www.corp.example.com/, send it a GET request
for /secret-plans, and then forward the contents of the file to a
remote host. If they could then trick someone on example.com's
intranet to look at this file, and assuming www.corp.example.com
did nothing more than rely on connectivitity for authentication
(pretty common in small intranets), then evil.example.net could
steal the company's secret plans.
Can't the web page just send an ajax request to corp.example.com
anyway?
It can't read the response, no.
Can you explain this please?
AFAIK, Sending a request then closing the connection immediately is the
only way said response might be unreadable. I don't understand what you
mean.
A script on a Web page can cause a GET request to be sent to an arbitrary
URL, but it has no way to obtain the contents of the result of that
request unless that server opts in (using CORS) to allowing the script to
see the contents of the response.
So in other words:
the remote web server being queried has to accept the TCP connection,
perform the correct HTTP-level handshakes, then correct CORS handshakes
in order for you to use those links?
as compared to:
the remote web server being queried has to accept the TCP connection,
perform the correct HTTP-level handshakes, then correct WebSockets
handshakes in order for you to use those links?
(note the single word difference).
Dumping the ability to look like HTTP altogether is your best bet
IMO: even on port 443. For port 443, someone running a websocket
server should just not run an HTTPS server there. Or define port 815
as websocket/s.
Port 815 is the preferred port, but in certain situations, only ports
80 and 443 are allowed, and port 80 is mitm'ed, so that only leaves
443.
You don't always have the option of using a different host, that's why
we need to share the port sometimes. (There have also been statements
to the effect that just sending a new protocol over ports 80 or 443
without using the Upgrade mechanism would be a violation of the
semantics of those ports.)
Understood. However strange protocols (torrent is one example) on port 80
can be detected and dealt with easily.
Something that appears to be perfect HTTP cannot be detected as non-HTTP.
When HTTP operations are done to the traffic it will break badly and in
nasty invisible ways.
Could you explain how a WebSocket connection can break in an invisible
way? An example would be very helpful here.
see every single example so far posted. Particularly example #2 this post.
If its not, it should bail. That seems reasonable :). We can
definitely (and have been :)) advising on how to do that with HTTP,
but it means *having a decent HTTP stack*. The response to a HTTP
request is always an HTTP response. Upgrade: changes the way the wire
behaves after that.
It seems that just having a WebSocket "stack" is adequate also.
When all layers of the stack confirm to the appropriate protocols, yes
agreed.
Point of this discussion is that the WebSockets layer _when sitting on top
of HTTP_ needs to be using HTTP layer correctly.
WebSockets, even when initiating the connection by talking to an HTTP
server and then switching to WebSockets, isn't layered on HTTP. It's just
doing the bare minimum required to allow the HTTP server to understand the
request and get out of the way. Once the handshake is complete, there is
no HTTP anywhere on the stack at all.
Its not doing the bare minimum.
The bare minimum would be to accept the valid HTTP transforms which the
Internet _will_ perform on the handshake. Discard those useless
transforms and validate the handshake status line.
Yes, but the client is a WebSocket client, not an HTTP client, so
why would it send anything but the WebSocket handshake?
Because its asking an HTTP server to upgrade.
How can it know that ahead of time?
When using port 80 is must assume to be doing so.
a) because it _is_
b) because any WebSockets server on port 80 is expecting that HTTP upgrade
request of yours to come in.
Therefore, doing proper HTTP Upgrade and discarding the junk headers is
no loss, remains secure, and a net gain when MITM are present doing
perfectly normal HTTP things.
I disagree that it remains secure, but leaving that aside, could you
explain how it is a net gain when MITM are present? Under what conditions
would a MITM allow a WebSocket Upgrade to actually succeed end-to-end,
even if we do everything according to HTTP's rules instead of the
simplified WebSocket rules?
see example #2 outlined above.
That makes it an HTTP client and a WebSocket client... as in fact you
note below ..
It's both. It's not an HTTP client in that it doesn't do anything but
what the WebSocket spec says it should do. It's an HTTP client in that
it speaks something that can be interpreted as HTTP.
Your spec says otherwise. Simply by defining the byte-level content of
the headers the server and client are now speaking non-HTTP. Were they
spec'd as speaking HTTP this entire argument would not be ocurring.
They're not speaking HTTP, they're speaking something that can be
interpreted as HTTP. I think the distinction is quite important.
Correct the distinction _is_ important. Add to it the port they are
speaking said 'something' through.
It becomes a critical point of failure if said 'something' should be
sent over port 80, port 3128.
It becomes an annoyance to network admin, but not critical if said
'something' is sent over port 81, port 88, port 8080, and port 8000.
Also a range of other ports with even lesser impact.
Ian, are you absolutely certain that everywhere you use "the
internet", there is no "man in the middle" between you and the
server you're speaking to?
In the case of TLS connections, yes.
In corporate networking, TLS MITM is a 'feature': company signed
certificates are used to sign the TLS connection to the corporate
firewall, and the firewall validate the SSL connection to the outside
world. I haven't personally used this, so can't really say much more
about it. Squid's ssl-bump feature can be used for this, but I
believe browser config is needed (again on a corporate basis) to tell
it that this is expected.
I can't think of any way of getting a WebSocket connection through
such a situation, so there's not much point doing anything to handle
this case. If you can think of a way of handling it that _would_ allow
WebSocket through, let me know, as then it would make sense to support
it.
Don't confuse TLS MITM with regular MITM. Regular MITM are still very
common and can be tunneled over without trouble just by accepting
regular HTTP changes during the upgrade request and reply headers.
Could you show me a proof of concept of this? I've not been able to get
any sort of two-way connection to a remote host when there's a MITM proxy
in the middle of the connection, regardless of what packets I send. Any
help you could provide here would be much appreciated.
It's not the sending that matters so much. It's the receiving.
Write your code as I spec'd out for you earlier.
* Send what you want.
* Receive whatever HTTP arrives and discard/skip/ignore all the
useless headers that come back before the first pair of 'CRLFCRLF' (two
sequential CRLF).
* The bytes that follow after the 101 reply (for Upgrade:) or 200 (for
CONNECT) are your two-way TCP link.
What MITM are you testing with?
Amos
--
Please be using
Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18
Current Beta Squid 3.1.0.13