Re: [whatwg] TCPConnection feedback
(Sorry if this counts as thread necromancy. The discussion just didn't seem to have come to an end yet.) On Fri, Jun 20, 2008 at 3:55 PM, Frode Børli [EMAIL PROTECTED] wrote: I would manage, but i do not like the implementation (it is much more complex than it needs to be). I would basically hate doing extra work because the implementation wasnt good enough. It is worth spending months improving the implementation here, if it saves only one minute of work for each of the millions of web developers out there, in the future. Alright, point taken. You're of course absolutely right with that :) I agree, it would be very convenient to basically set up and control a web app in a single connection. However, I think there are valid use cases for just the opposite set up as well. So, if we use a HTTP handshake, we should provide two ways. If it is impossible to use HTML, then it is absolutely required that Session ID is added as a standard header - since that will be the only way of linking the generated HTML with the separate persistent connection. You can't assume that an application server or the web server will be able to parse the cookie, since the cookie format is different for each programming language/application. This depends on the layer where the session management takes place. For example, PHP's existing session handling system already uses cookies. So, a hypothetical future PPHP version of PHP could extend the session system to support this. This feature couldn't be implemented in the afromentioned few lines of perl though. If this feature is still wanted in a standardized way, I think the idea to have the server advertise that it wants to change the protocol but to let the client do the actual switch would be the best way. The HTTP 101 Switching Protocols can be sent by the server, without the client asking for a protocol change. The only requirement is that the server sends 426 Upgrade Required first, then specifies which protocol to switch to. The protocol switched to could possibly be the one proposed in the beginning of this thread. I don't see how we could use 426 as a notification that the client should open a WebSocket connection. 426 is still an error code, so if you send it as the reply to the initial GET request, you can't be sure the HTML file you pushed gets interpreted the correct way. While this would probably work, it would be semantically unclean at best. However, the TLS -over-HTTP spec states As specified in HTTP/1.1, the server MAY include an Upgrade header in any response other than 101 or 426 to indicate a willingness to switch to any (combination) of the protocols listed. http://www.ietf.org/rfc/rfc2817.txt I can't seem to find any mention about this in the HTTP spec though. Is this implemented or at least widely known? Maybe you could reasoneably assume that a proxy keeps a connection open if an upgrade seems imminent? If this works, we could extend Michael's original algorithm as follows (this would be in addition to the new WebSocket() interface and would not replace it) PROPOSAL: Turning an existing HTTP connection into a WebSocket connection: If the server sends a Connection: Upgrade header and an Upgrade header with a WebSocket token as part of a normal response and if the resource fetched established a browsing contest, the client must not issue any other requests on that connection and must initiate a protocol switch. After the switch has finished, the client would expose the connection to the application via a DefaultWebSocket property or something similar. An exchange could look like this: C: GET /uri HTTP/1.1 C: Host: example.com C: [ ... usual headers ... ] C: S: HTTP/1.1 200 OK S: Content-Type: text/html S: [ ... usual headers ... ] S: Upgrade: WebSocket/1.0 S: Connection: Upgrade S: S: [ ... body ... ] C: OPTIONS /uri HTTP/1.1 C: Host: example.com C: Upgrade: WebSocket/1.0 C: Connection: Upgrade C: S: HTTP/1.1 101 Switching Protocols S: Upgrade: WebSocket/1.0 S: Connection: Upgrade S: C/S: [ ... application specific data ... ] Because the connection would be same-origin pretty much per definition, no access checks would be needed in that situation. Would something like this be doable and wanted? On Fri, Jun 20, 2008 at 1:52 PM, Frode Børli [EMAIL PROTECTED] wrote: I have a proposal for a cross domain security framework that i think should be implemented in browsers, java applets, flash applets and more. If we use any kind of verification that happens outside the connections, we should at least a hint inside the connection, which host the browser wants to connect though. I think raw TCP connections would cause major security holes with shared hosts, even if cross-domain connections need a DNS record. Yes, if we are using the WebSocket - but not if using the TCP connection. Nobody will be allowed to run a separate application that listens on a port on a shared server anyway. Consider the following scenario:
Re: [whatwg] TCPConnection feedback
It is worth spending months improving the implementation here, if it saves only one minute of work for each of the millions of web developers out there, in the future. Alright, point taken. You're of course absolutely right with that :) I agree, it would be very convenient to basically set up and control a web app in a single connection. However, I think there are valid use cases for just the opposite set up as well. So, if we use a HTTP handshake, we should provide two ways. Agree If it is impossible to use HTML, then it is absolutely required that Session ID is added as a standard header - since that will be the only way of linking the generated HTML with the separate persistent connection. You can't assume that an application server or the web server will be able to parse the cookie, since the cookie format is different for each programming language/application. This depends on the layer where the session management takes place. For example, PHP's existing session handling system already uses cookies. So, a hypothetical future PPHP version of PHP could extend The PHP *script* decides how to encode session identifiers, not the PHP-engine. PHP has a default cookie variable called PHPSESSID that many php-scripts use but many PHP-applications implement their own session handling. If the Session ID was implemented trough headers, it would have the following benefits: 1. Many more links in the chain between the browser and the server side script can utilize session id; for example a load balancer will easily see which internal server to pass requests to. 2. Session ID is not available for client side scripts - and makes session hijacking much more difficult. (Today they are accessible trough document.cookie) 3. Server side logic can be implemented in many more layers for connecting requests to an actual user session. 4. Statistics tools can much more easily identify unique visitors, as Session ID potentially could be logged in log files. I am sure there are more advantages. the session system to support this. This feature couldn't be implemented in the afromentioned few lines of perl though. SessionID header only need to be implemented on the client side for HTML5 browsers. Server side scripting languages can immediately read headers and set headers - but it would be an advantage if PHP (and others) was updated to use SessionID-header as default for request from browsers supporting HTML5. The HTTP 101 Switching Protocols can be sent by the server, without the client asking for a protocol change. The only requirement is that the server sends 426 Upgrade Required first, then specifies which protocol to switch to. The protocol switched to could possibly be the one proposed in the beginning of this thread. I don't see how we could use 426 as a notification that the client should open a WebSocket connection. 426 is still an error code, so if you send it as the reply to the initial GET request, you can't be sure the HTML file you pushed gets interpreted the correct way. While this would probably work, it would be semantically unclean at best. I am not a HTTP protocol wizard, but I have read that something similar is done for starting HTTPS-communications, and I believe the same procedure can be used for WebSocket. PROPOSAL: Turning an existing HTTP connection into a WebSocket connection: If the server sends a Connection: Upgrade header and an Upgrade header with a WebSocket token as part of a normal response and if the resource fetched established a browsing contest, the client must not issue any other requests on that connection and must initiate a protocol switch. After the switch has finished, the client would expose the connection to the application via a DefaultWebSocket property or something similar. An exchange could look like this: C: GET /uri HTTP/1.1 C: Host: example.com C: [ ... usual headers ... ] C: S: HTTP/1.1 200 OK S: Content-Type: text/html S: [ ... usual headers ... ] S: Upgrade: WebSocket/1.0 S: Connection: Upgrade S: S: [ ... body ... ] C: OPTIONS /uri HTTP/1.1 C: Host: example.com C: Upgrade: WebSocket/1.0 C: Connection: Upgrade C: S: HTTP/1.1 101 Switching Protocols S: Upgrade: WebSocket/1.0 S: Connection: Upgrade S: C/S: [ ... application specific data ... ] Because the connection would be same-origin pretty much per definition, no access checks would be needed in that situation. Would something like this be doable and wanted? This is exactly what I was trying to describe :) Consider the following scenario: Bob and Eve have bought space on a run-of-the-mill XAMPP web hoster. They have different domains but happen to be on the same IP. Now Eve wants do bruteforce Bob's password-protected web application. So she adds a script to her relatively popular site that does the following: So Bob will DDOS his own server? And my proposals allows using hostnames OR ip-addresses in the DNS TXT record, so unless Eve
Re: [whatwg] TCPConnection feedback
Rewriting the HTTP spec is not feasible and I'm not even convinced its a good idea. HTTP has always been request/response so it would make a lot more sense to simply use a new protocol then confuse millions of developers/administrators who thought they understood HTTP. As pointed out by others HTTP can preform asynchronously and persistently under certain circumstances (ie TLS handshake). Microsoft describe the process here: http://msdn.microsoft.com/en-us/library/aa380513(VS.85).aspx I think this will be a far better solution than opening a second communication channel to the server (ref my other posts). Currently CGI has the web server offload a bunch of environment variables that the CGI script decodes. What's missing then is a way to pass the whole socket off to the script. The Fast CGI protocol is closer to the mark. Wikipedia says: Environment information and page requests are sent from the web server to the process over a TCP connection (for remote processes) or Unix domain sockets (for local processes). Responses are returned from the process to the web server over the same connection. The connection may be closed at the end of a response, but the web server and the process are left standing. Also, it is quite common with modules linked directly with the server (ISAPI and Apache modules). For CGI (not FastCGI) i assume some inter process communication would have to be defined - but I do not think this forum should think about that, it is inevitable that changes must be done somewhere on the server side if we want to achieve two way communication with the same script that initially generated the page. So Fast CGI achieves all the main goals for WebSocket (proxy support, virtual hosting, ssl support) using existing access control rules and configs that ISPs are familiar with. The only thing that is not supported is persistent bi-directional communication (at least I have found nothing to indicate this). However based on the description above the only limiting factor seems to be an assumption that all links in the chain close the connection after the initial server response (making it bi-directional but not persistent). It also isn't strictly asynchronous since the client and server apparently cannot send/receive simultaneously. I do not believe that proxies enforce rules regarding who is able to send data over the channel, therefore I do not believe asynchronous communications will be a problem - even though nothing indicates it being done today. I propose a new protocol called Asynchrous CGI that extends Fast CGI to support asynchonous and persistent channels rather than the creation of an entirely new WebSockets protocol from scratch. The name implies that this is a server side protocol, used between the web server and the web application - so I believe that the name should be something like WebSocket still. Server side, there are multiple alternative interfaces to CGI (ISAPI, NSAPI, Apache modules etc). I believe support for WebSocket should be an issue between the web browser and the web server, and we should not consider what happens on the server side. A new working group could start working on extending FastCGI, but server side support will quickly be developed by those server providers wanting to support it. I assume a new working group will have to be created for extending FastCGI. If we put the access control in anything but the protocol it means that we are relying on an external service for security, so it would have to be something that is completely locked down. I don't really see what the mechanism would be. Can you propose a method for doing this so as to allow raw tcp connections without security complications? I don't understand your point. Existing services use firewalls, authentication, host-allow, etc as appropriate. The only new issue TCPConnection or WebConnection introduce is the concept of an non-user-initiated connection. In other words a remote untrusted server causing the local machine to make a connection without an explicit user action (such as checking mail in Outlook). I believe the proposed DNS extension combined with some form of explicit user-initiated priviledge elevation reduce the two main threats: DDOS and browser-based brute-force attacks. Michael informed me about a problem with my DNS extension proposal, and that is that anybody can register a domain called evil.com and add the required TXT records and point the domain to the target server. I think the DNS extension can still be used, but it has to involve reverse lookup on the target IP-address. The reverse lookup returns a host name, which must be queried for TXT-records that specify hosts that can have scripts allowed to connect. Does anybody have any views on this? What do you mean by brute-force access, and how could the proposed protocol *snip* I already have already provided two examples in previous posts but to *snip* This is not an issue,
[whatwg] TCPConnection feedback
On Fri, Jun 20, 2008 at 1:19 PM, Frode Børli [EMAIL PROTECTED] wrote: I think this will be a far better solution than opening a second communication channel to the server (ref my other posts). Would that be so much of a problem though? Your web application opens multiple connections today already, so you can not be sure that requests that belong to the same session all come on the same HTTP connection. If you have a proxy inbetween, I'd imagine you can't even be sure that requests on the same connection belong to the same user. The HTTP handshake would most likely send the cookies though, so it shouldn't be that difficult to associate your persistent connection with an existing session. If you don't want to rely on cookies, you could always make your own session tracking mechanism on top of this protocol using JS. I think if persistent connections are supported broadly, this could also be the start of a new design pattern, where you use HTTP only to serve static resources and do all dynamic stuff through a single persistent connection. Such applications wouldn't need any session information outside the connection at all. Of course it would be up to the web author how he designs his application. It's just a thought. However, if it's still desired, maybe we could add it as an additional option in HTML instead of HTTP? Idea: Add an additional HTML element. If it is present, the browser will not close the connection after it downloaded the document, but instead send an OPTIONS uri Upgrade: ... request and present and give the page's scripts access to a default WebSocket object that represents this connection. On Fri, Jun 20, 2008 at 10:26 AM, Shannon [EMAIL PROTECTED] wrote: I don't understand your point. Existing services use firewalls, authentication, host-allow, etc as appropriate. The only new issue TCPConnection or WebConnection introduce is the concept of an non-user-initiated connection. In other words a remote untrusted server causing the local machine to make a connection without an explicit user action (such as checking mail in Outlook). I believe the proposed DNS extension combined with some form of explicit user-initiated priviledge elevation reduce the two main threats: DDOS and browser-based brute-force attacks. On Fri, Jun 20, 2008 at 1:52 PM, Frode Børli [EMAIL PROTECTED] wrote: I have a proposal for a cross domain security framework that i think should be implemented in browsers, java applets, flash applets and more. If we use any kind of verification that happens outside the connections, we should at least a hint inside the connection, which host the browser wants to connect though. I think raw TCP connections would cause major security holes with shared hosts, even if cross-domain connections need a DNS record. Consider the following scenario: Bob and Eve have bought space on a run-of-the-mill XAMPP web hoster. They have different domains but happen to be on the same IP. Now Eve wants do bruteforce Bob's password-protected web application. So she adds a script to her relatively popular site that does the following: 1) Open a TCP connection to her own domain on port 80. As far as the browser is concerned, both origin and IP adress match the site one's, so no cross domain checks are performed. 2) Forge HTTP requests on that connection with Bob's site in the Host header and bruteforce the password. The Browser can't do anything about this, because it treats the TCP connection as opaque. The firewall just sees a legitimate HTTP request on a legitimate port. If Bob checks his logs, he will see requests from IPs all oer the world. Because Eve has control about the Referrer and Access-Control-Origin headers, there is no trace that leads back to her. She might even have send the headers with forged values to shift suspicion to another site. Note that the web hoster doesn't need to support WebSocket server side in any way for this to work. We could strengthen the security requirements, so that even same-domain requests need permission. However, then we had about the same hole as soon as the web host updates its services and gives Eve permission to access her own site. Access-Control: allow * exclude evilsite.example.com Access-Control-Max-Age: 86400 Access-Control-Policy-Path: / I think the idea is good, but I do not like the exact implementation. I think the server should be able to see which script is initiating the connection (header sent from the client), and then the server can respond with a HTTP 401 Access Denied. No need to specify anything more. No need to specify which hosts that are allowed, since the script can decide that on a number of parameters (like IP-address etc). (Sorry for the late reply, I had a few technical problems) I think the Access-Control-Origin header is exactly what you are looking for: http://www.w3.org/TR/access-control/#access-control-origin0 This would be sent by the client as a part of the Access Control checks. The idea
Re: [whatwg] TCPConnection feedback
I fail to see how virtual hosting will work for this anyway. I mean we're
not talking about Apache/IIS here, we're talking about custom applications,
scripts or devices - possibly implemented in firmware or a few lines of
perl. Adding vhost control to the protocol is just silly since the
webserver won't ever see the request and the customer application should be
able to use any method it likes to differentiate its services. Even URI
addressing is silly since again the application may have no concept of
paths or queries. It is simply a service running on a port. The only
valid use case for all this added complexity is proxying but nobody has
tested yet whether proxies will handle this (short of enabling encryption,
and even that is untested).
Actually, I've already tested this protocol against some typical forward
proxy setups and it hasn't caused any problems so far.
I'm thinking here that this proposal is basically rewriting the CGI
protocol (web server handing off managed request to custom scripts) with the
ONLY difference being the asynchronous nature of the request. Perhaps more
consideration might be given to how the CGI/HTTP protocols might be updated
to allow async communication.
Rewriting the HTTP spec is not feasible and I'm not even convinced its a
good idea. HTTP has always been request/response so it would make a lot more
sense to simply use a new protocol then confuse millions of
developers/administrators who thought they understood HTTP.
Having said that I still see a very strong use case for low-level
client-side TCP and UDP. There are ways to manage the security risks that
require further investigation. Even if it must be kept same-domain that is
better than creating a new protocol that won't work with existing services.
Even if that sounds like a feature - it isn't. There are better ways to
handle access-control for non-WebConnection devices than sending garbage to
the port.
If we put the access control in anything but the protocol it means that we
are relying on an external service for security, so it would have to be
something that is completely locked down. I don't really see what the
mechanism would be. Can you propose a method for doing this so as to allow
raw tcp connections without security complications?
[If a] protocol is decided on, and it is allowed to connect to any
IP-address - then DDOS attacks can still be performed: If one million web
browsers connect to any port on a single server, it does not matter which
protocol the client tries to communicate with. The server will still have
problems.
Couldn't this already be done today, though? You can already today
connect to an arbitrary server on an arbitrary port using forms,
img, script src= and all other references that cannot be
cross-domain protected for backwards compatibillity reasons. The whole
hotlinking issue is basically the result of that.
How would WebSocket connections be more harmful than something like
setInterval(function(){
var img = new Image();
img.src = http://victim.example.com/; http://victim.example.com/ +
generateLongRandomString();
}, 1000);
for example would?
It's more harmful because an img tag (to my knowledge) cannot be used to
brute-force access, whereas a socket connection could. With the focus on
DDOS it is important to remember that these sockets will enable full
read/write access to arbitrary services whereas existing methods can only
write once per connection and generally not do anything useful with the
response.
What do you mean by brute-force access, and how could the proposed protocol
be used to do it. Can you provide an example?
Also, the proposed protocol will do a single HTTP request, just like the img
tag, and the response be hidden from the attacker if it wasn't the right
response. From a potential attacker's point of view, this is a write once
per connection where the only control they have over the request is the
value of the url. Attacking with this protocol is identical to attacking
with an image tag in every way that I can think of.
-Michael Carter
Re: [whatwg] TCPConnection feedback
able to use any method it likes to differentiate its services. Even URI addressing is silly since again the application may have no concept of paths or queries. It is simply a service running on a port. The only valid use case for all this added complexity is proxying but nobody has tested yet whether proxies will handle this (short of enabling encryption, and even that is untested). I think we should have both a pure TCPSocket, and also a ServerSocket that keeps the same connection as the original document was downloaded from. The ServerSocket will make it very easy for web developers to work with, since the ServerSocket object will be available both from the server side and the client side while the page is being generated. I am posting a separate proposal that describes my idea soon. Actually, I've already tested this protocol against some typical forward proxy setups and it hasn't caused any problems so far. Could you test keeping the same connection as the webpage was fetched from, open? So that when the server script responds with its HTML-code - the connection is not closed, but used for kept alive for two way communications? This gives the following benefits: The script on the server decides if the connection should be closed or kept open. (Protection against DDOS attacks) This allows implementing server side listening to client side events, and vice versa. If this works, then the XMLHttpRequest object could be updated to allow two way communications in exactly the same way. Also, by adding a SessionID header sent from the client (instead of storing session ids in cookies), the web server could transparently rematch any client with its corresponding server side process in case of disconnect. I'm thinking here that this proposal is basically rewriting the CGI protocol (web server handing off managed request to custom scripts) with the ONLY difference being the asynchronous nature of the request. Perhaps more consideration might be given to how the CGI/HTTP protocols might be updated to allow async communication. Rewriting the HTTP spec is not feasible and I'm not even convinced its a good idea. HTTP has always been request/response so it would make a lot more sense to simply use a new protocol then confuse millions of developers/administrators who thought they understood HTTP. The HTTP spec has these features already: 1: Header: Connection: Keep-Alive 2: Status: HTTP 101 Switching Protocol No need to rewrite the HTTP spec at all probably. Having said that I still see a very strong use case for low-level client-side TCP and UDP. There are ways to manage the security risks that require further investigation. Even if it must be kept same-domain that is better than creating a new protocol that won't work with existing services. Even if that sounds like a feature - it isn't. There are better ways to handle access-control for non-WebConnection devices than sending garbage to the port. If we put the access control in anything but the protocol it means that we are relying on an external service for security, so it would have to be something that is completely locked down. I don't really see what the mechanism would be. Can you propose a method for doing this so as to allow raw tcp connections without security complications? TCPConnections are only allowed to the server where the script was downloaded from (same as Flash and Java applets). A DNS TXT record can create a white list of servers whose scripts can connect. Also the TCPConnection possibly should be allowed to connect to local network resources, after a security warning - but only if the server has a proper HTTPS certificate. It's more harmful because an img tag (to my knowledge) cannot be used to brute-force access, whereas a socket connection could. With the focus on DDOS it is important to remember that these sockets will enable full read/write access to arbitrary services whereas existing methods can only write once per connection and generally not do anything useful with the response. What do you mean by brute-force access, and how could the proposed protocol be used to do it. Can you provide an example? With the security measures I suggest above, there is no need for protection against brute force attacks. Most developers only use one server per site, and those that have multiple servers will certainly be able to add a TXT-record to the DNS.
Re: [whatwg] TCPConnection feedback
I think we should have both a pure TCPSocket, and also a ServerSocket that keeps the same connection as the original document was downloaded from. The ServerSocket will make it very easy for web developers to work with, since the ServerSocket object will be available both from the server side and the client side while the page is being generated. I am posting a separate proposal that describes my idea soon. I don't see the benefit of making sure that its the same connection that the page was generated from. Actually, I've already tested this protocol against some typical forward proxy setups and it hasn't caused any problems so far. Could you test keeping the same connection as the webpage was fetched from, open? So that when the server script responds with its HTML-code - the connection is not closed, but used for kept alive for two way communications? If you establish a Connection: Keep-Alive with the proxy server, it will leave the connection open to you, but that doesn't mean that it will leave the connection open to the back end server as the Connection header is a single-hop header. This gives the following benefits: The script on the server decides if the connection should be closed or kept open. (Protection against DDOS attacks) With the proposed spec, the server can close the connection at any point. This allows implementing server side listening to client side events, and vice versa. If this works, then the XMLHttpRequest object could be updated to allow two way communications in exactly the same way. The previously proposed protocol already allows the server side listening to client side events, and vice versa. Rather or not to put that in the XMLHttpRequest interface is another issue. I think making XHR bi-directional is a bad idea because its confusing. Better to use a brand new api, like WebSocket. Also, by adding a SessionID header sent from the client (instead of storing session ids in cookies), the web server could transparently rematch any client with its corresponding server side process in case of disconnect. Isn't that what cookies are supposed to do? Regardless, it sounds like an application-level concern that should be layered on top of the protocol. I'm thinking here that this proposal is basically rewriting the CGI protocol (web server handing off managed request to custom scripts) with the ONLY difference being the asynchronous nature of the request. Perhaps more consideration might be given to how the CGI/HTTP protocols might be updated to allow async communication. Rewriting the HTTP spec is not feasible and I'm not even convinced its a good idea. HTTP has always been request/response so it would make a lot more sense to simply use a new protocol then confuse millions of developers/administrators who thought they understood HTTP. The HTTP spec has these features already: 1: Header: Connection: Keep-Alive 2: Status: HTTP 101 Switching Protocol No need to rewrite the HTTP spec at all probably. You can't use HTTP 101 Switching Protocols without a Connection: Upgrade header. I think you'll note that the proposal that started this thread uses just this combination. Having said that I still see a very strong use case for low-level client-side TCP and UDP. There are ways to manage the security risks that require further investigation. Even if it must be kept same-domain that is better than creating a new protocol that won't work with existing services. Even if that sounds like a feature - it isn't. There are better ways to handle access-control for non-WebConnection devices than sending garbage to the port. If we put the access control in anything but the protocol it means that we are relying on an external service for security, so it would have to be something that is completely locked down. I don't really see what the mechanism would be. Can you propose a method for doing this so as to allow raw tcp connections without security complications? TCPConnections are only allowed to the server where the script was downloaded from (same as Flash and Java applets). A DNS TXT record can create a white list of servers whose scripts can connect. Also the TCPConnection possibly should be allowed to connect to local network resources, after a security warning - but only if the server has a proper HTTPS certificate. How would a DNS TXT record solve the problem? I could register evil.com and point it at an arbitrary ip address and claim that anyone who wants to can connect. It's more harmful because an img tag (to my knowledge) cannot be used to brute-force access, whereas a socket connection could. With the focus on DDOS it is important to remember that these sockets will enable full read/write access to arbitrary services whereas existing methods can only write once per connection and generally not do anything useful with the response. What do you mean by
Re: [whatwg] TCPConnection feedback
I think we should have both a pure TCPSocket, and also a ServerSocket
that keeps the same connection as the original document was downloaded
from. The ServerSocket will make it very easy for web developers to
work with, since the ServerSocket object will be available both from
the server side and the client side while the page is being generated.
I am posting a separate proposal that describes my idea soon.
I don't see the benefit of making sure that its the same connection that the
page was generated from.
It does not have to be exactly the same connection, but I think it
should be handled by the web server because then there is no need to
think about transferring state information between for example a PHP
script and a WebSocketServer. It would be almost like creating a
desktop application. Simply because it would be easy for
webdevelopers: Sample PHP script:
There is probably a better approach to implementing this in php, but
its just a concept:
input id='test' type='button';
script type='text/javascript'
// when the button is clicked, raise the test_click event
handler on the server.
document.getElementById('test').addEventListener('click',
document.serverSocket.createEventHandler('test_click');
// when the server raises the message event, alert the message
document.serverSocket.addEventListener('message', alert);
/script
?php
// magic PHP method that is called whenever a client side event is
sent to the server
function __serverSocketEvent($name, $event)
{
if($name == 'test_click')
server_socket_event(message, You clicked the button);
}
?
If you establish a Connection: Keep-Alive with the proxy server, it will
leave the connection open to you, but that doesn't mean that it will leave
the connection open to the back end server as the Connection header is a
single-hop header.
So it is not possible at all? There are no mechanisms in HTTP and
proxy servers that facilitates keeping the connection alive all the
way trough to the web server?
If a Session ID (or perhaps a Request ID) is added to the headers then
it is possible to create server side logic that makes it easier for
web developers. When session ids are sent trough cookies, web servers
and proxy servers have no way to identiy a session (since only the
script knows which cookie is the session id). The SessionID header
could be used by load balancers and more - and it could also be used
by for example IIS/Apache to connect a secondary socket to the script
that created the page (and ultimately achieving what I want).
The script on the server decides if the connection should be closed or
kept open. (Protection against DDOS attacks)
With the proposed spec, the server can close the connection at any point.
I stated it as a benefit in the context of the web server handling the
requests. An image would close the connection immediately, but a
script could decide to keep it open. All servers can ofcourse close
any connection any time.
This allows implementing server side listening to client side events,
and vice versa. If this works, then the XMLHttpRequest object could be
updated to allow two way communications in exactly the same way.
The previously proposed protocol already allows the server side listening to
client side events, and vice versa. Rather or not to put that in the
XMLHttpRequest interface is another issue. I think making XHR bi-directional
is a bad idea because its confusing. Better to use a brand new api, like
WebSocket.
If the implementation works as I tried to examplify in the PHP script
above; a document.serverSocket object is available, then the xhr
object should also have a .serverSocket object.
document.serverSocket.addEventListener(...)
xhr.serverSocket.addEventListener(...)
I am sure this can be achieved regardless of the protocol.
Also, by adding a SessionID header sent from the client (instead of
storing session ids in cookies), the web server could transparently
rematch any client with its corresponding server side process in case
of disconnect.
Isn't that what cookies are supposed to do? Regardless, it sounds like an
application-level concern that should be layered on top of the protocol.
One important advantage is that javascript.cookie can be used for
hijacking sessions by sending the cookie trough for example an
img-tag. If javascript can't access the SessionID then sessions cant
be hijacked through XSS attacks.
Also I think load balancers and web servers and other applications
that do not have intimate knowledge about the web application should
be able to pair WebSocket connections with the actual http request.
How else can load balancers be created if they have to load balance
both pages and websockets to the same webserver? The load balancer
does not know what part of the cookie identifies the session.
I am sure that some clever people will find other uses if the session
id and request id is available for each request
Re: [whatwg] TCPConnection feedback
without informing the user. This would allow a popular page (say a facebook profile or banner ad) to perform massive DOS against web servers using visitors browsers without any noticeable feedback (though I guess this is also true of current HTTPXMLRequestObjects). XMLHttpRequest only allows connections to the origin server ip of the script that created the object. If a TCPConnection is supposed to be able to connect to other services, then some sort of mechanism must be implemented so that the targeted web server must perform some sort of approval. The method of approval must be engineered in such a way that approval process itself cannot be the target of the dos attack. I can imagine something implemented on the DNS servers and then some digital signing of the script using public/private key certificates. I propose that there be requirements that limit the amount and type of data a client can send before receiving a valid server response. If the client must send information trough the TCPConnection initially, then we effectively stop existing servers such as IRC-servers from being able to accept connections without needing a rewrite. There should also be a recommendation that UAs display some form of status feedback to indicate a background connection is occurring. Agree. HIXIE.3) No existing SMTP server (or any non-TCPConnection server) is going to send back the appropriate handshake response. If TCPConnection is limited to connect only to the origin server, or servers validated by certificates, then this will never be a problem. If we take active measures against STMP, then we should do the same against POP3, IMAP etc as well. It is always possible that non-http services are running on port 80. One logical reason would be as a workaround for strict firewalls. So the main defense against abuse is not the port number but the handshake. The original TCP Connection spec required the client to send only Hello\n and the server to send only Welcome\n. The new proposal complicates things since the server/proxy could send any valid HTTP headers and it would be up to the UA to determine their validity. Since the script author can also inject URIs into the handshake this becomes a potential flaw. Consider the code: The protocol should not require any data (not even hello - it should function as an ordinary TCPConnection similar to implementations in java, c# or any other major programming language. If not, it should be called something else - as it is not a TCP connection.
Re: [whatwg] TCPConnection feedback
Frode Børli wrote: XMLHttpRequest only allows connections to the origin server ip of the script that created the object. If a TCPConnection is supposed to be able to connect to other services, then some sort of mechanism must be implemented so that the targeted web server must perform some sort of approval. The method of approval must be engineered in such a way that approval process itself cannot be the target of the dos attack. I can imagine something implemented on the DNS servers and then some digital signing of the script using public/private key certificates. Using DNS is an excellent idea, though I would debate whether the certificate is needed in addition the DNS record. Perhaps the DNS record could simply list domains authorised to provide scripted access. The distributed nature and general robustness of DNS servers provides the most solid protection against denial of service and brute-force cracking which are the primary concerns here. Access-control should probably be handled by the hosts usual firewall and authentication methods which is trivial once the unauthorised redirect issue is dealt with. The biggest issue I see is that most UAs are probably not wired to read DNS records directly. This means adding DNS access and parsing libraries for this one feature. Having said that I can see a whole range of security issues that could be addressed by DNS access so maybe this is something that HTML5 could address as a more general feature. One feature that comes to mind would be to advertise expected server outages or /.'ing via DNS so the UAs could tell the user Hey, this site might not respond so maybe come back later. It is worth considering allowing scripts to access devices without said DNS rules but with a big fat UA warning, requiring user approval. Something like This site is attempting to access a remote service or device at the address 34.45.23.54:101 (POP3). This could be part of a legitimate service but may also be an attempt to perform a malicious task. If you do not trust this site you should say no here.. This would address the needs of private networks and home appliances that wish to utilise TCPConnection services without having the desire or ability to mess with DNS zone files. The protocol should not require any data (not even hello - it should function as an ordinary TCPConnection similar to implementations in java, c# or any other major programming language. If not, it should be called something else - as it is not a TCP connection. I agree completely. Just providing async HTTP is a weak use case compared to allowing client-side access to millions of existing (opted-in) services and gadgets. Shannon
Re: [whatwg] TCPConnection feedback
The protocol should not require any data (not even hello - it should function as an ordinary TCPConnection similar to implementations in java, c# or any other major programming language. If not, it should be called something else - as it is not a TCP connection. I agree completely. Just providing async HTTP is a weak use case compared to allowing client-side access to millions of existing (opted-in) services and gadgets. Shannon It's clear that we need some kind of opt-in strategy or all web viewers will become spam bots in short order. While I think it will be extremely useful to have a raw tcp connection in the browser, and indeed you could use an external service like dns to handle connection authorization, I think that it will be much more difficult to drive adoption to that kind of standard. In the meantime, we need to make the protocol enforce the opt-in. In that case I agree that the name shouldn't be TCPConnection. I propose SocketConnection instead. -Michael Carter
Re: [whatwg] TCPConnection feedback
On Wed, 18 Jun 2008, Michael Carter wrote: In that case I agree that the name shouldn't be TCPConnection. I propose SocketConnection instead. I was thinking WebSocket (with the protocol itself called the Web Socket Protocol or Web Socket Communication Protocol or some such). -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] TCPConnection feedback
On Wed, Jun 18, 2008 at 12:59 PM, Ian Hickson [EMAIL PROTECTED] wrote: On Wed, 18 Jun 2008, Michael Carter wrote: In that case I agree that the name shouldn't be TCPConnection. I propose SocketConnection instead. I was thinking WebSocket (with the protocol itself called the Web Socket Protocol or Web Socket Communication Protocol or some such). That sounds pretty good. Worth noting is that http://en.wikipedia.org/wiki/Internet_socket suggests that the term Socket doesn't refer to tcp/ip, rather it refers to multiple protocols. You can have a UDPSocket or an IPSocket just as easily as you could have a TCPSocket. In this context, neither WebSocket nor SocketConnection really present a naming problem. -Michael Carter
Re: [whatwg] TCPConnection feedback
Still I do not believe it should have a specific protocol. If a protocol is decided on, and it is allowed to connect to any IP-address - then DDOS attacks can still be performed: If one million web browsers connect to any port on a single server, it does not matter which protocol the client tries to communicate with. The server will still have problems. Aren't there and identical set of objections to the cross-domain access-control header? Or microsofts XDR object? Even without Websocket, browsers will be making fully cross-domain requests, and the only question left is how exactly to implement the security in the protocol. That said, there's no additional harm in allowing WebSocket to establish cross-domain connections, but there are many benefits. -Michael Carter
[whatwg] TCPConnection feedback
Still I do not believe it should have a specific protocol.
I think a major problem with raw TCP connections is that they would be
the nightmare of every administrator. If web pages could use every
sort of homebrew protocol on all possible ports, how could you still
sensibly configure a firewall without the danger of accidentally
disabling mary sue grandmother's web application?
Also keep in mind the issue list Ian brought up in the other mail.
Things like URI based adressing and virtual hosting would not be
possible with raw TCP. That would make this feature a lot less useable
for authors that do not have full access over their server, like in
shared hosting situations, for example.
[If a] protocol is decided on, and it is allowed to connect to any IP-address
- then DDOS attacks can still be performed: If one million web
browsers connect to any port on a single server, it does not matter
which protocol the client tries to communicate with. The server will
still have problems.
Couldn't this already be done today, though? You can already today
connect to an arbitrary server on an arbitrary port using forms,
img, script src= and all other references that cannot be
cross-domain protected for backwards compatibillity reasons. The whole
hotlinking issue is basically the result of that.
How would WebSocket connections be more harmful than something like
setInterval(function(){
var img = new Image();
img.src = http://victim.example.com/; + generateLongRandomString();
}, 1000);
for example would?
Re: [whatwg] TCPConnection feedback
I think a major problem with raw TCP connections is that they would be the nightmare of every administrator. If web pages could use every sort of homebrew protocol on all possible ports, how could you still sensibly configure a firewall without the danger of accidentally disabling mary sue grandmother's web application? This already happens. Just yesterday we (an ISP) had a company unable to access webmail on port 81 due to an overzealous firewall administrator. But how is a web server on port 81 more unsafe than one on 80? It isn't the port that matters, it's the applications that may (or may not) be using them that need to be controlled. Port-based blocking of whole networks is a fairly naive approach today. Consider that the main reason for these nazi firewalls is two-fold: 1.) to prevent unauthorised/unproductive activities (at schools, libraries or workplaces); and 2.) to prevent viruses connecting out. Port-blocking to resolve these things doesn't work anymore since: 1.) even without plugins a Web 2.0 browser provides any number of games, chat sites and other 'time-wasters'; and 2.) free (or compromised) web hosting can provide viruses with update and control mechanisms without creating suspicion by using uncommon ports; and 3.) proxies exist (commercial and free) to tunnel any type of traffic over port 80. On the other hand port control interferes with legitimate services (like running multiple web servers on a single IP). So what I'm saying here is that network admins can do what they want but calling the policy of blocking non-standard ports sensible and then basing standards on it is another thing. It's pretty obvious that port-based firewalling will be obsoleted by protocol sniffing and IP/DNS black/whitelists sooner rather than later. Your argument misses the point anyway. Using your browser as an IRC client is no different to downloading mIRC or using a web-based chat site. The genie of running arbitrary services from a web client escaped the bottle years ago with the introduction of javascript and plugins. We are looking at browser as a desktop rather than browser as a reader and I don't think that's something that will ever be reversed. Since we're on the threshold of the Web Applications age, and this is the Web Applications Working Group we should be doing everything we can to enable those applications while maintaining security. Disarming the browser is a valid goal ONLY once we've exhausted the possibility of making it safe. Also keep in mind the issue list Ian brought up in the other mail. Things like URI based adressing and virtual hosting would not be possible with raw TCP. That would make this feature a lot less useable for authors that do not have full access over their server, like in shared hosting situations, for example. I fail to see how virtual hosting will work for this anyway. I mean we're not talking about Apache/IIS here, we're talking about custom applications, scripts or devices - possibly implemented in firmware or a few lines of perl. Adding vhost control to the protocol is just silly since the webserver won't ever see the request and the customer application should be able to use any method it likes to differentiate its services. Even URI addressing is silly since again the application may have no concept of paths or queries. It is simply a service running on a port. The only valid use case for all this added complexity is proxying but nobody has tested yet whether proxies will handle this (short of enabling encryption, and even that is untested). I'm thinking here that this proposal is basically rewriting the CGI protocol (web server handing off managed request to custom scripts) with the ONLY difference being the asynchronous nature of the request. Perhaps more consideration might be given to how the CGI/HTTP protocols might be updated to allow async communication. Having said that I still see a very strong use case for low-level client-side TCP and UDP. There are ways to manage the security risks that require further investigation. Even if it must be kept same-domain that is better than creating a new protocol that won't work with existing services. Even if that sounds like a feature - it isn't. There are better ways to handle access-control for non-WebConnection devices than sending garbage to the port. [If a] protocol is decided on, and it is allowed to connect to any IP-address - then DDOS attacks can still be performed: If one million web browsers connect to any port on a single server, it does not matter which protocol the client tries to communicate with. The server will still have problems. Couldn't this already be done today, though? You can already today connect to an arbitrary server on an arbitrary port using forms, img, script src= and all other references that cannot be cross-domain protected for backwards compatibillity reasons. The whole hotlinking issue is
[whatwg] TCPConnection feedback
Hello, We've had a number of discussions in the #whatwg channel thus far about the TCPConnection specification in section 6, and the consesus is that there is utility in an asynchronous, bi-directional communication mechanism, but the current proposal has a number of issues that need to be resolved. There are a list of requirements for the protocol: Hixie basically my main requirements are that: HIXIE.1) there be the ability for one process to have a full-duplex communication channel to the script running in the web page HIXIE.2) the server-side be implementable in a fully conformant way in just a few lines of perl without support libraries HIXIE.3) that it be safe from abuse (e.g. can't connect to smtp servers) HIXIE.4) that it work from within fascist firewalls There are some issues with the specification as it stands: --- othermaciej my two problems with it are: (1) it uses port/host addressing instead of URI addressing, which is a poor fit for the Web model othermaciej (2) it's bad to send non-http over the assigned ports for http and https othermaciej (3) I am worried that connection to arbitrary ports could lead to security issues, although Hixie tried hard to avoid them -- The complete list from multiple discussions, I believe, is: ISSUE.1) lack of URI addressing ISSUE.2) sending non-http(s) over http(s) ports ISSUE.3) inability to traverse forward proxies ISSUE.4) lack of cross-domain access control ISSUE.5) DNS rebinding security holes ISSUE.6) lack of load balancing integration ISSUE.7) lack of authentication integration ISSUE.8) virtual hosting with secure communication (no Host header, and even if there was, there's no way to indicate this header *before* the secure handshake) I propose that we PROPOSAL.1) change the initial handshake of the protocol to be HTTP based to leverage existing solutions to these problems. PROPOSAL.2) modify the API to use URIs instead of port/host pairs I believe that the HTTP/1.1 OPTIONS method, Upgrade header, and 101 Switching Protocols response is the best avenue to take because these parts of HTTP were specifically designed to: (1) See if the server can speak the new protocol (2) switch the protocol mid-stream. How the changes solve the problems = ISSUE.1) We added URI addressing ISSUE.2) We now only send valid HTTP(s) over HTTP(s) ports. ISSUE.3) We can send a CONNECT to the proxy, thus eliminating this problem. ISSUE.4) We can use the Access-Control header to solve this problem. ISSUE.5) The Host header solves this problem ISSUE.6) Many load balancers use cookies or urls. Using HTTP allows both cookies and urls, so we can integrate with existing HTTP load balancers. ISSUE.7) Cookies allow standard authentication mechansisms to be applied to the TCPConnection ISSUE.8) We can follow RFC 2817 for TLS upgrading mid stream, Examples Normal Handshake In javascript on a page served from the domain example.com, the following call is issued: tcp = new TCPConnection(http://example.com/some/url;) C: OPTIONS /some/url HTTP/1.1\r\n Host: example.com\r\n Upgrade: TCPConnection/1.0\r\n Connection: Upgrade\r\n \r\n S: HTTP/1.1 101 Switching Protocols\r\n Connection: Upgrade\r\n Upgrade: TCPConnection/1.0\r\n \r\n C, S: [ Bi-directional communication ] Secure Handshake In javascript on a page served from the domain example.com, the following call is issued: tcp = new TCPConnection(https://example.com/some/url;) C, S (port 443): [ SSL handshake ] C: OPTIONS /some/url HTTP/1.1\r\n Host: example.com\r\n Upgrade: TCPConnection/1.0\r\n Connection: Upgrade\r\n \r\n S: HTTP/1.1 101 Switching Protocols\r\n Connection: Upgrade\r\n Upgrade: TCPConnection/1.0\r\n \r\n C, S: [ Bi-directional communication ] Cross-domain Access Controlled Handshake In javascript on a page served from the domain example.com, the following call is issued: tcp = new TCPConnection(http://www.example.com/some/url;) C: OPTIONS /some/url HTTP/1.1\r\n Host: www.example.com\r\n Upgrade: TCPConnection/1.0\r\n Connection: Upgrade\r\n \r\n S: HTTP/1.1 101 Switching Protocols\r\n Connection: Upgrade\r\n Upgrade: TCPConnection/1.0\r\n Access-Control: allow example.com\r\n \r\n C, S: [ Bi-directional communication ] Handshake Through a Forward Proxy (with Proxy Authentication) - In javascript from a page served in the domain example.com, and the browser is behind a forward proxy, the following call is issued: tcp = new TCPConnection(http://example.com/some/url;) C: CONNECT example.com:80 HTTP/1.1\r\n Proxy-authorization: basic aGVsbG86d29ybGQ=\r\n \r\n OPTIONS /some/url HTTP/1.1\r\n Host: example.com\r\n Upgrade: TCPConnection/1.0\r\n Connection: Upgrade\r\n \r\n S: HTTP/1.1 101 Switching Protocols\r\n Connection: Upgrade\r\n Upgrade: TCPConnection/1.0\r\n \r\n C, S: [ Bi-directional communication ]
Re: [whatwg] TCPConnection feedback
ISSUE.2) We now only send valid HTTP(s) over HTTP(s) ports.
I understand the reasoning but I do not believe this should be limited
to ports 80 and 443. By doing so we render the protocol difficult to use
as many (if not most) custom services would need to run on another port
to avoid conflict with the primary webserver. I understand the logic for
large public sites where fascist firewalls might prohibit other ports
but for custom services (ie, remote-control, telnet emulation or
accounting access) used within a company network this would be a real
pain (requiring setup of reverse proxy or dedicated server). This would
make a mockery of the whole premise of implementing services using a
few lines of perl.
Limiting to ports 80 and 443 doesn't really solve the security issues
anyway. Many firewall/routers can be configured on this port and
denial-of-service is just as effective (or more) against port 80 as any
other port. The new proposal to use HTTP headers effectively allows
arbitrary (in length and content) strings to be sent to any port 80
device without informing the user. This would allow a popular page (say
a facebook profile or banner ad) to perform massive DOS against web
servers using visitors browsers without any noticeable feedback (though
I guess this is also true of current HTTPXMLRequestObjects).
I propose that there be requirements that limit the amount and type of
data a client can send before receiving a valid server response. The
requirements should limit:
* Length of initial client handshake
* Encoding of characters to those valid in URIs (ie, no arbitrary binary
data)
* Number or retries per URI
* Number of simultaneous connections
* Total number of connection attempts per script domain (to all URIs)
There should also be a recommendation that UAs display some form of
status feedback to indicate a background connection is occurring.
HIXIE.3) No existing SMTP server (or any non-TCPConnection server) is going
to send back the appropriate handshake response.
It is always possible that non-http services are running on port 80. One
logical reason would be as a workaround for strict firewalls. So the
main defense against abuse is not the port number but the handshake. The
original TCP Connection spec required the client to send only Hello\n
and the server to send only Welcome\n. The new proposal complicates
things since the server/proxy could send any valid HTTP headers and it
would be up to the UA to determine their validity. Since the script
author can also inject URIs into the handshake this becomes a potential
flaw. Consider the code:
tcp = TCPConnection('http://mail.domain.ext/\\r\\nHELO HTTP/1.1 101
Switching Protocols\\r\\n' )
client
OPTIONS \r\n
HELO HTTP/1.1 101 Switching Protocols\r\n
HTTP/1.1\r\n
server
250 mail.domain.ext Hello \r\n
HTTP/1.1 101 Switching Protocols\r\n
[111.111.111.111], pleased to meet you
As far as a naive UA and mail server is concerned we have now issued a
valid challenge and received a valid response (albeit with some
unrecognised/malformed headers). The parsing rules will need to be very
strict to prevent this kind of attack. Limiting to port 80 reduces the
number of target servers but does not prevent the attack (or others like
it).
It may be that simply stripping newlines and non-ascii from URIs is all
that's required since most text-based protocols are line oriented
anyway. It depends largely on how OPTIONS and CONNECT are interpreted.
One last thing. Does anybody know how async communication would affect
common proxies (forward and reverse)? I imagine they can handle large
amounts of POST data but how do they feel about a forcibly held-open
by-directional communication that never calls POST or GET? How would
caches respond without expires or max-age headers? Would this hog
threads causing apache/squid to stop serving requests? Would this work
through Tor?
Shannon
Re: [whatwg] TCPConnection feedback
On Wed, 18 Jun 2008, Shannon wrote: ISSUE.2) We now only send valid HTTP(s) over HTTP(s) ports. I understand the reasoning but I do not believe this should be limited to ports 80 and 443. You misunderstand; it's not the ports that are limited, it's just that the traffic can now pass for HTTP. This would all still work over any arbitrary port. HIXIE.3) No existing SMTP server (or any non-TCPConnection server) is going to send back the appropriate handshake response. It is always possible that non-http services are running on port 80. One logical reason would be as a workaround for strict firewalls. So the main defense against abuse is not the port number but the handshake. Indeed, we would need to very carefully define exactly what the server must send back, much like in the original protocol -- it would just look a lot more like HTTP. This would include at least one custom header or value that you wouldn't see elsewhere (e.g. the Upgrade: header with the magic value). Since the script author can also inject URIs into the handshake this becomes a potential flaw. Indeed, we'd have to throw if the URI wasn't a valid URI (e.g. if it included newlines). One last thing. Does anybody know how async communication would affect common proxies (forward and reverse)? I imagine they can handle large amounts of POST data but how do they feel about a forcibly held-open by-directional communication that never calls POST or GET? That's basically what TLS is, right? The simple solution would be to just tunnel everything through TLS when you hit an uncooperative proxy. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
