Re: [whatwg] Issues with Web Sockets API
17.10.2009, в 3:20, Ian Hickson написал(а): I'm not really sure what else to say to be honest. Should I just leave it at cookies and nothing else? Really I just want to support Basic (and I guess Digest) authentication (primarily for over-TLS connections), so that sites that use Basic auth, like, say, porn sites, or the W3C, can also use it for their Web Socket connections. I could just limit it that way; would that work? Formally limiting support for Basic auth would be workable, I guess. Implementation of Digest authentication is already non-trivial enough for me to wish that we don't implement it at first. Or perhaps authentication should be limited to cookies in v1 indeed. If /code/, interpreted as ASCII, is 401, then let /mode/ be _authenticate_. Otherwise, fail the Web Socket connection and abort these steps. 407 (proxy authenticate) also likely needs to be supported. Proxies wouldn't work with WebSockets in general. Could you please elaborate? I thought there was a setup that could work with most deployed HTTPS proxies - one could run WebSockets server on port 443. Oh, I see what you're saying. Proxy authentication of this nature is covered by step 2 of the handshake algorithm, as part of connect to that proxy and ask it to open a TCP/IP connection to the host given by / host/ and the port given by /port/. There's even an example showing auth headers being sent to the proxy. By the time we get down to parsing the response, we're long past the point where we might be authenticating to a proxy. Is that a problem? Hmm, I actually don't know for sure. Step 2 only covers the case when the user agent is configured to use a proxy - but an organization may have a transparent proxy intercepting requests. But I do not know if such a proxy can practically request authentication by returning a 407 response (blurring the meaning of transparent a bit, but anyway). I could add support for 407 here and just say that you jump back to step 2 and include the authentication this time, would that work? If the answer to my above concern is yes, then it should work, as long as the text doesn't require double TLS handshake or something like that. Some authentication schemes (e.g. NTLM) work on connection basis, so I don't think that closing the connection right after receiving a challenge can work with them. Yeah, that's quite possible. Is this something you plan to correct in the spec? Is there much to correct? I don't understand what would need to change here. Does NTLM not work with HTTP without pipelining? You probably meant HTTP persistent connections here, not pipelining. Yes, since NTLM authentication works on connections and not individual requests, closing the connection after receiving a challenge will make it inoperable, as far as I know. Or do you mean that you would rather have authentication be a first-class primitive operation in Web Socket, instead of relying on the HTTP features? We could do that: instead of faking an HTTP communication, we could have a header in the handshake that means after this, the client must send one more handshake consisting of an authentication token, and if the UA fails to send the right extra bit, then fail. I think if we did this, we'd want to punt until version 2, though. Yes, I think that relying on HTTP specs to define authentication to Web Sockets takes the fake HTTP handshake concept too far. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
(cc'ing hybi by request, since changes to the WebSocket spec are discussed below) On Thu, 22 Oct 2009, Alexey Proskuryakov wrote: 17.10.2009, в 3:20, Ian Hickson написал(а): I'm not really sure what else to say to be honest. Should I just leave it at cookies and nothing else? Really I just want to support Basic (and I guess Digest) authentication (primarily for over-TLS connections), so that sites that use Basic auth, like, say, porn sites, or the W3C, can also use it for their Web Socket connections. I could just limit it that way; would that work? Formally limiting support for Basic auth would be workable, I guess. Implementation of Digest authentication is already non-trivial enough for me to wish that we don't implement it at first. Or perhaps authentication should be limited to cookies in v1 indeed. Ok, I've removed the authentication stuff from this version. If /code/, interpreted as ASCII, is 401, then let /mode/ be _authenticate_. Otherwise, fail the Web Socket connection and abort these steps. 407 (proxy authenticate) also likely needs to be supported. Proxies wouldn't work with WebSockets in general. Could you please elaborate? I thought there was a setup that could work with most deployed HTTPS proxies - one could run WebSockets server on port 443. Oh, I see what you're saying. Proxy authentication of this nature is covered by step 2 of the handshake algorithm, as part of connect to that proxy and ask it to open a TCP/IP connection to the host given by /host/ and the port given by /port/. There's even an example showing auth headers being sent to the proxy. By the time we get down to parsing the response, we're long past the point where we might be authenticating to a proxy. Is that a problem? Hmm, I actually don't know for sure. Step 2 only covers the case when the user agent is configured to use a proxy - but an organization may have a transparent proxy intercepting requests. But I do not know if such a proxy can practically request authentication by returning a 407 response (blurring the meaning of transparent a bit, but anyway). I don't see how such a proxy could handle WebSocket anyway. The WebSocket protocol isn't really intended to be proxied. I could add support for 407 here and just say that you jump back to step 2 and include the authentication this time, would that work? If the answer to my above concern is yes, then it should work, as long as the text doesn't require double TLS handshake or something like that. I've added text in the spec that says to handle 407 basically like HTTP, closing the connection and jumping back to step 2. Or do you mean that you would rather have authentication be a first-class primitive operation in Web Socket, instead of relying on the HTTP features? We could do that: instead of faking an HTTP communication, we could have a header in the handshake that means after this, the client must send one more handshake consisting of an authentication token, and if the UA fails to send the right extra bit, then fail. I think if we did this, we'd want to punt until version 2, though. Yes, I think that relying on HTTP specs to define authentication to Web Sockets takes the fake HTTP handshake concept too far. Ok. We'll revisit this in the next version, I guess. I'm not sure there's really much point doing anything beyond cookie auth anyway. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Issues with Web Sockets API
On Wed, 14 Oct 2009, Alexey Proskuryakov wrote: 13.10.2009, в 4:11, Ian Hickson написал(а): Is this meant to mimic some behavior that existing clients have for HTTP already? Yes, as it says, the idea is for UAs to send the same headers they would send if the protocol had been HTTP. For HTTP, this depends on authentication scheme in use. For Basic and Digest authentication in particular, clients are allowed to make certain assumptions about protection spaces: A client MAY preemptively send the corresponding Authorization header with requests for resources in that space without receipt of another challenge from the server. I don't think the Web Sockets protocol is sufficiently similar to HTTP to defer to RFC 2617 or other HTTP specs here. Also, implementing just support the same authentication mechanisms you do for HTTP is a tall order, since HTTP back-ends don't (always?) expose the necessary APIs for encryption. I'm not really sure what else to say to be honest. Should I just leave it at cookies and nothing else? Really I just want to support Basic (and I guess Digest) authentication (primarily for over-TLS connections), so that sites that use Basic auth, like, say, porn sites, or the W3C, can also use it for their Web Socket connections. I could just limit it that way; would that work? If /code/, interpreted as ASCII, is 401, then let /mode/ be _authenticate_. Otherwise, fail the Web Socket connection and abort these steps. 407 (proxy authenticate) also likely needs to be supported. Proxies wouldn't work with WebSockets in general. Could you please elaborate? I thought there was a setup that could work with most deployed HTTPS proxies - one could run WebSockets server on port 443. Oh, I see what you're saying. Proxy authentication of this nature is covered by step 2 of the handshake algorithm, as part of connect to that proxy and ask it to open a TCP/IP connection to the host given by /host/ and the port given by /port/. There's even an example showing auth headers being sent to the proxy. By the time we get down to parsing the response, we're long past the point where we might be authenticating to a proxy. Is that a problem? I could add support for 407 here and just say that you jump back to step 2 and include the authentication this time, would that work? Some authentication schemes (e.g. NTLM) work on connection basis, so I don't think that closing the connection right after receiving a challenge can work with them. Yeah, that's quite possible. Is this something you plan to correct in the spec? Is there much to correct? I don't understand what would need to change here. Does NTLM not work with HTTP without pipelining? Or do you mean that you would rather have authentication be a first-class primitive operation in Web Socket, instead of relying on the HTTP features? We could do that: instead of faking an HTTP communication, we could have a header in the handshake that means after this, the client must send one more handshake consisting of an authentication token, and if the UA fails to send the right extra bit, then fail. I think if we did this, we'd want to punt until version 2, though. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Issues with Web Sockets API
13.10.2009, в 4:11, Ian Hickson написал(а): Is this meant to mimic some behavior that existing clients have for HTTP already? Yes, as it says, the idea is for UAs to send the same headers they would send if the protocol had been HTTP. For HTTP, this depends on authentication scheme in use. For Basic and Digest authentication in particular, clients are allowed to make certain assumptions about protection spaces: A client MAY preemptively send the corresponding Authorization header with requests for resources in that space without receipt of another challenge from the server. I don't think the Web Sockets protocol is sufficiently similar to HTTP to defer to RFC 2617 or other HTTP specs here. Also, implementing just support the same authentication mechanisms you do for HTTP is a tall order, since HTTP back-ends don't (always?) expose the necessary APIs for encryption. If /code/, interpreted as ASCII, is 401, then let /mode/ be _authenticate_. Otherwise, fail the Web Socket connection and abort these steps. 407 (proxy authenticate) also likely needs to be supported. Proxies wouldn't work with WebSockets in general. Could you please elaborate? I thought there was a setup that could work with most deployed HTTPS proxies - one could run WebSockets server on port 443. A feature that doesn't work behind proxies suddenly makes a lot less sense - one could use it to control a toy railroad over intranet, but do other use cases survive? Some authentication schemes (e.g. NTLM) work on connection basis, so I don't think that closing the connection right after receiving a challenge can work with them. Yeah, that's quite possible. Is this something you plan to correct in the spec? If practical uses of Web Sockets are all going to be over SSL (for proxy compatibility reasons), then even Basic auth seems ok for many cases, but NTLM still has important benefits over it. The primary benefit I'm aware of is that passwords don't need to be stored on the server. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
On Mon, 31 Aug 2009, Alexey Proskuryakov wrote: 9. If the client has any authentication information ... that would be relevant to a resource accessed over HTTP, if /secure/ is false, or HTTPS, if it is true, on host /host/, port /port/, with /resource name/ as the path (and possibly query parameters), then HTTP headers that would be appropriate for that information should be sent at this point. [RFC2616] [RFC2109] [RFC2965] I'm not sure how this part translates into actual behavior. What if there are several sets of credentials already known to the client, for example? What would you do in the same situation for HTTP URLs? Also, what if the client has already performed digest authentication with several nonce values? Same question. Is this meant to mimic some behavior that existing clients have for HTTP already? Yes, as it says, the idea is for UAs to send the same headers they would send if the protocol had been HTTP. If /code/, interpreted as ASCII, is 401, then let /mode/ be _authenticate_. Otherwise, fail the Web Socket connection and abort these steps. 407 (proxy authenticate) also likely needs to be supported. Proxies wouldn't work with WebSockets in general. - If the entry's name is www-authenticate Obtain credentials in a manner consistent with the requirements for handling the |WWW-Authenticate| header in HTTP, and then close the connection (if the server has not already done so) Some authentication schemes (e.g. NTLM) work on connection basis, so I don't think that closing the connection right after receiving a challenge can work with them. Yeah, that's quite possible. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Issues with Web Sockets API
On Thu, 13 Aug 2009, Alexey Proskuryakov wrote: 13.08.2009, в 4:42, Ian Hickson написал(а): A note explaining that the close event will be dispatched at server's discretion (or on subsequent connection timeout), potentially long time after close() is called, would likely make the intention clearer. It won't be long after the close(). Why not? My understanding is that it can take arbitrary amount of time - the server can choose not to close its side of connection for years, and to send data over it. It's unrelated to the server. You call close(), the UA closes the connection, thus the connection is closed, which causes the UA to fire an event. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Issues with Web Sockets API
04.08.2009, в 16:47, Ian Hickson написал(а): I've added support for redirects. While I was at it I also added support for authentication. Reading the authentication part of the latest draft, I had several comments: 9. If the client has any authentication information ... that would be relevant to a resource accessed over HTTP, if /secure/ is false, or HTTPS, if it is true, on host /host/, port /port/, with /resource name/ as the path (and possibly query parameters), then HTTP headers that would be appropriate for that information should be sent at this point. [RFC2616] [RFC2109] [RFC2965] I'm not sure how this part translates into actual behavior. What if there are several sets of credentials already known to the client, for example? Also, what if the client has already performed digest authentication with several nonce values? Is this meant to mimic some behavior that existing clients have for HTTP already? If /code/, interpreted as ASCII, is 401, then let /mode/ be _authenticate_. Otherwise, fail the Web Socket connection and abort these steps. 407 (proxy authenticate) also likely needs to be supported. - If the entry's name is www-authenticate Obtain credentials in a manner consistent with the requirements for handling the |WWW- Authenticate| header in HTTP, and then close the connection (if the server has not already done so) Some authentication schemes (e.g. NTLM) work on connection basis, so I don't think that closing the connection right after receiving a challenge can work with them. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
13.08.2009, в 4:42, Ian Hickson написал(а): A note explaining that the close event will be dispatched at server's discretion (or on subsequent connection timeout), potentially long time after close() is called, would likely make the intention clearer. It won't be long after the close(). Why not? My understanding is that it can take arbitrary amount of time - the server can choose not to close its side of connection for years, and to send data over it. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
On Tue, 28 Jul 2009, Alexey Proskuryakov wrote: 28.07.2009, в 16:40, Ian Hickson написал(а): 3) A Web Sockets server cannot respond with a redirect to another URL. I'm not sure if the intention is to leave this to implementations, or to add in Web Sockets v2, but it definitely looks like an important feature to me, maybe something that needs to be in v1. What's the use case? Why does this need to be at the connection layer rather than the application layer? (Why would we need this, when TCP doesn't have it? Would you also need redirect-like functonality in IRC, IMAP, SSH, and other such protocols?) Just like with HTTP, redirects will make it possible to move services to a different location. I guess the use cases are exactly the same that tell us that we should allow redirects with cross-site XMLHttpRequest (but I can't enumerate those). Other protocols do not get accessed from Web pages, so transparent redirect support is not needed to keep web apps working after services they use move. The Web has redirects all over the place, and WebSocket has Web in its name :) Finally, since it's likely that WebSocket servers will share ports with HTTP ones, one can expect that returning a 307 for all requests (including those with Upgrade: WebSocket) will be enough to preserve application functionality. Redirects surely add a lot of complexity though. I've added support for redirects. While I was at it I also added support for authentication. So far I've only changed the client handshake part of this, I expect in time I'll update the intro and the server-side part, but I wanted to get feedback on the client-side part first. 11) There is no way for the client to know that the connection has been closed. For example: - socket.close() is called from JavaScript; - onclose handler is invoked; - more data arrives from the server, and onmessage is dispatched (which I think is correct, and it matches what TCP does); - finally, a TCP FIN arrives, indicating that there will be no more data from the server (the underlying TCP connection is in TIME_WAIT state after that); - the client never learns that the server is done sending data. The onclose only fires once the connection has closed, which is after the TCP FIN, so it happens after the last 'message' event. Maybe this could be clarified in the spec. The current text is: The close() method must close the Web Socket connection or connection attempt, if any. If the connection is already closed, it must do nothing. Closing the connection causes a close event to be fired and the readyState attribute's value to change, as described below. I was reading it as a requirement to immediately change readyState to CLOSED, and to fire a close event. If all this happens asynchronously after the server agrees to close the connection, then my example will work fine, of course. The text Closing the connection causes a close event to be fired and the readyState attribute's value to change, as described below is not normative (it has no musts in it). It's just letting you know that subsequent requirements as described lower in the spec do result in that. Is there some way I can make that clearer? On Tue, 28 Jul 2009, Jeremy Orlow wrote: They can be implemented at the application layer though, right? I suppose this does add complexity, but it doesn't seem like it'd add _that_ much. Especially if you're already doing authentication at the application level as well. I definitely agree these are good items for v2, but I think what's already in the spec is a good start. It ended up not adding too much, so I just went ahead and added it now. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Issues with Web Sockets API
04.08.2009, в 16:47, Ian Hickson написал(а): The text Closing the connection causes a close event to be fired and the readyState attribute's value to change, as described below is not normative (it has no musts in it). It's just letting you know that subsequent requirements as described lower in the spec do result in that. Is there some way I can make that clearer? A note explaining that the close event will be dispatched at server's discretion (or on subsequent connection timeout), potentially long time after close() is called, would likely make the intention clearer. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
On Wed, 29 Jul 2009, Fumitoshi Ukai (��~\飼�~V~G�~U~O) wrote: I'm wondering why send() doesn't raise INVALID_STATE_ERR exception when readyState is CLOSED and is required to queue up the data after closed. What is the reason for this behavior? So that scripts don't get unexpected exceptions randomly based on network weather. 8) Many HTTPS proxies only allow connecting to port 443. Do you have the data on whether relying on existing proxies to establish connections to arbitrary ports is practical? I do not. I expect most people to use direct connections over port 81 or TLS over port 443, as discussed in the introduction. You mean TLS over port 815 here, although TLS over port 443 would be practical because of common firewall configuration? No, I meant what I wrote. I don't expect most people to use port 815, though it is the right thing to do. Maybe on the long term 815 will get used more. 11) There is no way for the client to know that the connection has been closed. For example: - socket.close() is called from JavaScript; - onclose handler is invoked; - more data arrives from the server, and onmessage is dispatched (which I think is correct, and it matches what TCP does); - finally, a TCP FIN arrives, indicating that there will be no more data from the server (the underlying TCP connection is in TIME_WAIT state after that); - the client never learns that the server is done sending data. The onclose only fires once the connection has closed, which is after the TCP FIN, so it happens after the last 'message' event. so, it means we might receive message events even after we call websocket.close(), until we get close event? Yes. But that's the case anyway -- events are asynchronous, so consider the case of receiving two messages. Both are queued up, then eventually the first is dispatched. If in response to that you close the connection, that doesn't stop the second being dispatched, since it was already queued up. The attribute now lets you just wait until the buffer is empty, which is more or less equivalent, I think. Do we need to check bufferedAmount periodically by setTimeout? How about firing some event when buffer is empty? If you need to send the information anyway, just send it. This is really only useful in the case where you're sending information that you can omit completely, for example, if you're sending events on an interval, in which case you can just avoid sending messages if there is any buffered data. There's no point buffering the data in JS while waiting for the send buffer to be empty, since the buffer in JS and the send buffer are from the same pool of RAM. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Issues with Web Sockets API
On Wed, Jul 29, 2009 at 1:33 AM, Ian Hickson i...@hixie.ch wrote: Yes. But that's the case anyway -- events are asynchronous, so consider the case of receiving two messages. Both are queued up, then eventually the first is dispatched. If in response to that you close the connection, that doesn't stop the second being dispatched, since it was already queued up. I'd note that this conforms to the behavior of MessagePorts - close disentangles the ports, but already-received/queued messages are still delivered.
Re: [whatwg] Issues with Web Sockets API
On Tue, 14 Jul 2009, Jeremy Orlow wrote: I think 'readyState' should just go away since an application will have to keep track of state updates through the fired events and use try/catch blocks around all API calls anyway. The attribute is mostly present for debugging purposes. I wouldn't expect anyone to actually use it for production work. Is there precedent for other portions of the API that are mostly for debugging purposes? (I can't think of anything off the top of my head.) readyState on Document and video aren't realy useful for anything but debugging either, as far as I can tell. Also, maybe it should be noted as such in the spec? I don't really see much benefit to including such a statement; if someone wants to use it for a non-debugging reason, why not do so? If it's only for debugging purposes, maybe a cleaner way to define it is to simply be the last event fired on a given WebSocket? I don't really understand what problem we would be trying to solve by changing that. One other random question: in the IDL for WebSockets, the three constants for ready state are all defined as shorts but the value of ready state is a long. Is this an oversight? Fixed. On Mon, 27 Jul 2009, Alexey Proskuryakov wrote: I agree with Michael that send() should not silently drop data that could not be sent. It is very easy to fill send buffers, and if bytes get silently dropped, implementing app-level acks becomes quite difficult. I've made it clear that if bytes can't be sent, the connection must be closed. However, I do not think that raising an exception is an appropriate answer. Often, the TCP implementation takes a part of data given to it, and asks to resubmit the rest later. So, just returning an integer result from send() would be best in my opinion. I think we are best off abstracting away this level of complexity from authors, especially since we'd need to make sure that data was not sent half-way through a UTF-8 sequence, and since the framing is under the control of the UA, not the application. There's no way to retry a partially-successful send() from the API here. 1) Web Sockets is specified to send whatever authentication credentials the client has for the resource. However, there is no challenge-response sequence specified, which seems to prevent using common auth schemes. HTTP Basic needs to know an authentication realm for the credentials, and other schemes need a cryptographic challenge (e.g. nonce for Digest auth). I expect to address this in more detail in a future version. For now, use in-band authentication in the WebSocket once you are connected. We may find that that is actually enough. 2) It is not specified what the server does when credentials are incorrect, so I assume that the intended behavior is to close the connection. Unlike HTTP 401 response, this doesn't give the client a chance to ask the user again. Also, if the server is on a different host, especially one that's not shared with an HTTP server, there isn't a way to obtain credentials, in the first place. How we address this will likely depend on how we address the earlier point. 3) A Web Sockets server cannot respond with a redirect to another URL. I'm not sure if the intention is to leave this to implementations, or to add in Web Sockets v2, but it definitely looks like an important feature to me, maybe something that needs to be in v1. What's the use case? Why does this need to be at the connection layer rather than the application layer? (Why would we need this, when TCP doesn't have it? Would you also need redirect-like functonality in IRC, IMAP, SSH, and other such protocols?) 4) If the user agent already has a Web Socket connection to the remote host identified by /host/ (even if known by another name), wait until that connection has been established or for that connection to have failed. It doesn't look like host identified by /host/ is defined anywhere. Does this requirement say that IP addresses should be compared, instead of host names? Right. I've tried to clarify this. I'm not sure if this is significant for preventing DoS attacks, and anyway, the IP address may not be known before a request is sent. This puts an unusual burden on the implementation. Without this requirement, you can just have a DNS server return the victim IP for a wildcard DNS entry, and then just have attackers open connections to thousands of hosts. 5) We probably need to specify a keep-alive feature to avoid proxy connection timeout. I do not have factual data on whether common proxies implement connection timeout, but I'd expect them to often do. This seems like something that would be easy to deal with at the application layer, if desired. 6) The spec should probably explicitly permit blocking some ports from use with Web Sockets at UA's discretion. In practice, the list would likely
Re: [whatwg] Issues with Web Sockets API
On Tue, Jul 28, 2009 at 4:40 PM, Ian Hickson i...@hixie.ch wrote: On Tue, 14 Jul 2009, Jeremy Orlow wrote: If it's only for debugging purposes, maybe a cleaner way to define it is to simply be the last event fired on a given WebSocket? I don't really understand what problem we would be trying to solve by changing that. I was just suggesting that perhaps the model could be simplified and that this might make it less confusing to web and UA developers. I haven't studied this issue in detail though, so if you think it's a silly idea, there's a good chance it is. As for the rest of your changes, I think they should work well.
Re: [whatwg] Issues with Web Sockets API
On Tue, Jul 28, 2009 at 4:40 PM, Ian Hickson i...@hixie.ch wrote: On Tue, 14 Jul 2009, Jeremy Orlow wrote: I think 'readyState' should just go away since an application will have to keep track of state updates through the fired events and use try/catch blocks around all API calls anyway. The attribute is mostly present for debugging purposes. I wouldn't expect anyone to actually use it for production work. Is there precedent for other portions of the API that are mostly for debugging purposes? (I can't think of anything off the top of my head.) readyState on Document and video aren't realy useful for anything but debugging either, as far as I can tell. Also, maybe it should be noted as such in the spec? I don't really see much benefit to including such a statement; if someone wants to use it for a non-debugging reason, why not do so? If it's only for debugging purposes, maybe a cleaner way to define it is to simply be the last event fired on a given WebSocket? I don't really understand what problem we would be trying to solve by changing that. One other random question: in the IDL for WebSockets, the three constants for ready state are all defined as shorts but the value of ready state is a long. Is this an oversight? Fixed. On Mon, 27 Jul 2009, Alexey Proskuryakov wrote: I agree with Michael that send() should not silently drop data that could not be sent. It is very easy to fill send buffers, and if bytes get silently dropped, implementing app-level acks becomes quite difficult. I've made it clear that if bytes can't be sent, the connection must be closed. However, I do not think that raising an exception is an appropriate answer. Often, the TCP implementation takes a part of data given to it, and asks to resubmit the rest later. So, just returning an integer result from send() would be best in my opinion. I think we are best off abstracting away this level of complexity from authors, especially since we'd need to make sure that data was not sent half-way through a UTF-8 sequence, and since the framing is under the control of the UA, not the application. There's no way to retry a partially-successful send() from the API here. 1) Web Sockets is specified to send whatever authentication credentials the client has for the resource. However, there is no challenge-response sequence specified, which seems to prevent using common auth schemes. HTTP Basic needs to know an authentication realm for the credentials, and other schemes need a cryptographic challenge (e.g. nonce for Digest auth). I expect to address this in more detail in a future version. For now, use in-band authentication in the WebSocket once you are connected. We may find that that is actually enough. 2) It is not specified what the server does when credentials are incorrect, so I assume that the intended behavior is to close the connection. Unlike HTTP 401 response, this doesn't give the client a chance to ask the user again. Also, if the server is on a different host, especially one that's not shared with an HTTP server, there isn't a way to obtain credentials, in the first place. How we address this will likely depend on how we address the earlier point. 3) A Web Sockets server cannot respond with a redirect to another URL. I'm not sure if the intention is to leave this to implementations, or to add in Web Sockets v2, but it definitely looks like an important feature to me, maybe something that needs to be in v1. What's the use case? Why does this need to be at the connection layer rather than the application layer? (Why would we need this, when TCP doesn't have it? Would you also need redirect-like functonality in IRC, IMAP, SSH, and other such protocols?) 4) If the user agent already has a Web Socket connection to the remote host identified by /host/ (even if known by another name), wait until that connection has been established or for that connection to have failed. It doesn't look like host identified by /host/ is defined anywhere. Does this requirement say that IP addresses should be compared, instead of host names? Right. I've tried to clarify this. I'm not sure if this is significant for preventing DoS attacks, and anyway, the IP address may not be known before a request is sent. This puts an unusual burden on the implementation. Without this requirement, you can just have a DNS server return the victim IP for a wildcard DNS entry, and then just have attackers open connections to thousands of hosts. 5) We probably need to specify a keep-alive feature to avoid proxy connection timeout. I do not have factual data on whether common proxies implement connection timeout, but I'd expect them to often do. This seems like something that would be easy to deal with at the application layer, if desired. 6) The spec should probably explicitly permit
Re: [whatwg] Issues with Web Sockets API
28.07.2009, в 16:40, Ian Hickson написал(а): 3) A Web Sockets server cannot respond with a redirect to another URL. I'm not sure if the intention is to leave this to implementations, or to add in Web Sockets v2, but it definitely looks like an important feature to me, maybe something that needs to be in v1. What's the use case? Why does this need to be at the connection layer rather than the application layer? (Why would we need this, when TCP doesn't have it? Would you also need redirect-like functonality in IRC, IMAP, SSH, and other such protocols?) Just like with HTTP, redirects will make it possible to move services to a different location. I guess the use cases are exactly the same that tell us that we should allow redirects with cross-site XMLHttpRequest (but I can't enumerate those). Other protocols do not get accessed from Web pages, so transparent redirect support is not needed to keep web apps working after services they use move. The Web has redirects all over the place, and WebSocket has Web in its name :) Finally, since it's likely that WebSocket servers will share ports with HTTP ones, one can expect that returning a 307 for all requests (including those with Upgrade: WebSocket) will be enough to preserve application functionality. Redirects surely add a lot of complexity though. 10) Web Socket handshake uses CRLF line endings strictly. Does this add much to security? It prevents using telnet/netcat for debugging, which is something I personally use often when working on networking issues. If there is no practical reason for this, I'd suggest relaxing this aspect of parsing. Do you mean client-server or server-client? Originally, I meant both, but now I've been told that telnet on Mac OS X translates line breaks to CRLF by default (even though it's not telnet's own default, according to man page). Netcat doesn't perform such translation, so simulating a server with netcat won't work unless plain CR is allowed. Other platforms and tools commonly used for HTTP debugging may have different defaults and limitations. This is not a huge deal, of course. 11) There is no way for the client to know that the connection has been closed. For example: - socket.close() is called from JavaScript; - onclose handler is invoked; - more data arrives from the server, and onmessage is dispatched (which I think is correct, and it matches what TCP does); - finally, a TCP FIN arrives, indicating that there will be no more data from the server (the underlying TCP connection is in TIME_WAIT state after that); - the client never learns that the server is done sending data. The onclose only fires once the connection has closed, which is after the TCP FIN, so it happens after the last 'message' event. Maybe this could be clarified in the spec. The current text is: The close() method must close the Web Socket connection or connection attempt, if any. If the connection is already closed, it must do nothing. Closing the connection causes a close event to be fired and the readyState attribute's value to change, as described below. I was reading it as a requirement to immediately change readyState to CLOSED, and to fire a close event. If all this happens asynchronously after the server agrees to close the connection, then my example will work fine, of course. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
On Tue, Jul 28, 2009 at 8:57 PM, Alexey Proskuryakov a...@webkit.org wrote: 28.07.2009, в 16:40, Ian Hickson написал(а): 3) A Web Sockets server cannot respond with a redirect to another URL. I'm not sure if the intention is to leave this to implementations, or to add in Web Sockets v2, but it definitely looks like an important feature to me, maybe something that needs to be in v1. What's the use case? Why does this need to be at the connection layer rather than the application layer? (Why would we need this, when TCP doesn't have it? Would you also need redirect-like functonality in IRC, IMAP, SSH, and other such protocols?) Just like with HTTP, redirects will make it possible to move services to a different location. I guess the use cases are exactly the same that tell us that we should allow redirects with cross-site XMLHttpRequest (but I can't enumerate those). Other protocols do not get accessed from Web pages, so transparent redirect support is not needed to keep web apps working after services they use move. The Web has redirects all over the place, and WebSocket has Web in its name :) Finally, since it's likely that WebSocket servers will share ports with HTTP ones, one can expect that returning a 307 for all requests (including those with Upgrade: WebSocket) will be enough to preserve application functionality. Redirects surely add a lot of complexity though. They can be implemented at the application layer though, right? I suppose this does add complexity, but it doesn't seem like it'd add _that_ much. Especially if you're already doing authentication at the application level as well. I definitely agree these are good items for v2, but I think what's already in the spec is a good start.
Re: [whatwg] Issues with Web Sockets API
06.07.2009, в 21:30, Ian Hickson написал(а): postMessage() may want another exception condition... 'too much data pending exception'... consider calling postMessage in a while(true) loop... at some point the system is going to have to give up queing the data if its not actually making its way out on the wire. The spec doesn't specify how UAs are to handle hitting hardware limitations or system limitations, because it's often difficult to truly control how those cases are handled. I agree with Michael that send() should not silently drop data that could not be sent. It is very easy to fill send buffers, and if bytes get silently dropped, implementing app-level acks becomes quite difficult. With TCP, the basic guarantee is that bytes are not lost until the connection is lost, so app-level acks only require confirming the last processed command, and losing this guarantee would be quite unfortunate. Most (all?) system TCP implementations certainly have ways to deal with flow control. However, I do not think that raising an exception is an appropriate answer. Often, the TCP implementation takes a part of data given to it, and asks to resubmit the rest later. So, just returning an integer result from send() would be best in my opinion. The thread has such a nice title that I'm going to throw some additional issues in :) 1) Web Sockets is specified to send whatever authentication credentials the client has for the resource. However, there is no challenge-response sequence specified, which seems to prevent using common auth schemes. HTTP Basic needs to know an authentication realm for the credentials, and other schemes need a cryptographic challenge (e.g. nonce for Digest auth). 2) It is not specified what the server does when credentials are incorrect, so I assume that the intended behavior is to close the connection. Unlike HTTP 401 response, this doesn't give the client a chance to ask the user again. Also, if the server is on a different host, especially one that's not shared with an HTTP server, there isn't a way to obtain credentials, in the first place. I'm not sure how to best handle this, other than to copy more HTTP behaviors. 3) A Web Sockets server cannot respond with a redirect to another URL. I'm not sure if the intention is to leave this to implementations, or to add in Web Sockets v2, but it definitely looks like an important feature to me, maybe something that needs to be in v1. 4) If the user agent already has a Web Socket connection to the remote host identified by /host/ (even if known by another name), wait until that connection has been established or for that connection to have failed. It doesn't look like host identified by /host/ is defined anywhere. Does this requirement say that IP addresses should be compared, instead of host names? I'm not sure if this is significant for preventing DoS attacks, and anyway, the IP address may not be known before a request is sent. This puts an unusual burden on the implementation. 5) We probably need to specify a keep-alive feature to avoid proxy connection timeout. I do not have factual data on whether common proxies implement connection timeout, but I'd expect them to often do. 6) The spec should probably explicitly permit blocking some ports from use with Web Sockets at UA's discretion. In practice, the list would likely be the same as for HTTP, see e.g. http://www.mozilla.org/projects/netlib/PortBanning.html . 7) use a SOCKS proxy for WebSocket connections, if available, or failing that, to prefer an HTTPS proxy over an HTTP proxy It is not clear what definition of proxy types is used here. To me, an HTTPS proxy is one that supports CONNECT to port 443, and an HTTP proxy (if we're making a distinction from HTTPS) is one that intercepts and forwards GET requests. However, this understanding contradicts an example in paragraph 3.1.3, and also, it's not clear how a GET proxy could be used for Web Sockets. 8) Many HTTPS proxies only allow connecting to port 443. Do you have the data on whether relying on existing proxies to establish connections to arbitrary ports is practical? 9) There is no limit to the number of established Web Socket connections a user agent can have with a single remote host. Does this mean that Web Socket connections are exempt from the normal 4-connection (or so) limit? Why is it OK? 10) Web Socket handshake uses CRLF line endings strictly. Does this add much to security? It prevents using telnet/netcat for debugging, which is something I personally use often when working on networking issues. If there is no practical reason for this, I'd suggest relaxing this aspect of parsing. 11) There is no way for the client to know that the connection has been closed. For example: - socket.close() is called from JavaScript; - onclose handler is invoked; - more data arrives from the
Re: [whatwg] Issues with Web Sockets API
27.07.2009, в 12:35, Maciej Stachowiak написал(а): However, I do not think that raising an exception is an appropriate answer. Often, the TCP implementation takes a part of data given to it, and asks to resubmit the rest later. So, just returning an integer result from send() would be best in my opinion. With WebSocket, another possibility is for the implementation to buffer pending data that could not yet be sent to the TCP layer, so that the client of WebSocket doesn't have to be exposed to system limitations. At that point, an exception is only needed if the implementation runs out of memory for buffering. With a system TCP implementation, the buffering would be in kernel space, which is a scarce resource, but user space memory inside the implementation is no more scarce than user space memory held by the Web application waiting to send to the WebSocket. I agree that this will help if the application sends data in burst mode, but what if it just constantly sends more than the network can transmit? It will never learn that it's misbehaving, and will just take more and more memory. An example where adapting to network bandwidth is needed is of course file uploading, but even if we dismiss it as a special case that can be served with custom code, there's also e.g. captured video or audio that can be downgraded in quality for slow connections. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
On Mon, Jul 27, 2009 at 1:14 PM, Alexey Proskuryakov a...@webkit.org wrote: 27.07.2009, в 12:35, Maciej Stachowiak написал(а): However, I do not think that raising an exception is an appropriate answer. Often, the TCP implementation takes a part of data given to it, and asks to resubmit the rest later. So, just returning an integer result from send() would be best in my opinion. With WebSocket, another possibility is for the implementation to buffer pending data that could not yet be sent to the TCP layer, so that the client of WebSocket doesn't have to be exposed to system limitations. At that point, an exception is only needed if the implementation runs out of memory for buffering. With a system TCP implementation, the buffering would be in kernel space, which is a scarce resource, but user space memory inside the implementation is no more scarce than user space memory held by the Web application waiting to send to the WebSocket. I agree that this will help if the application sends data in burst mode, but what if it just constantly sends more than the network can transmit? It will never learn that it's misbehaving, and will just take more and more memory. An example where adapting to network bandwidth is needed is of course file uploading, but even if we dismiss it as a special case that can be served with custom code, there's also e.g. captured video or audio that can be downgraded in quality for slow connections. Maybe the right behavior is to buffer in user-space (like Maciej explained) up until a limit (left up to the UA) and then anything beyond that results in an exception. This seems like it'd handle bursty communication and would keep the failure model simple.
Re: [whatwg] Issues with Web Sockets API
On Mon, Jul 27, 2009 at 1:14 PM, Alexey Proskuryakov a...@webkit.org wrote: 27.07.2009, в 12:35, Maciej Stachowiak написал(а): However, I do not think that raising an exception is an appropriate answer. Often, the TCP implementation takes a part of data given to it, and asks to resubmit the rest later. So, just returning an integer result from send() would be best in my opinion. With WebSocket, another possibility is for the implementation to buffer pending data that could not yet be sent to the TCP layer, so that the client of WebSocket doesn't have to be exposed to system limitations. At that point, an exception is only needed if the implementation runs out of memory for buffering. With a system TCP implementation, the buffering would be in kernel space, which is a scarce resource, but user space memory inside the implementation is no more scarce than user space memory held by the Web application waiting to send to the WebSocket. I agree that this will help if the application sends data in burst mode, but what if it just constantly sends more than the network can transmit? It will never learn that it's misbehaving, and will just take more and more memory. I would suggest that the solution to this situation is an appropriate application-level protocol (i.e. acks) to allow the application to have no more than (say) 1MB of data outstanding. I'm just afraid that we're burdening the API to handle degenerative cases that the vast majority of users won't encounter. Specifying in the API that any arbitrary send() invocation could throw some kind of retry exception or return some kind of error code is really really cumbersome. An example where adapting to network bandwidth is needed is of course file uploading, but even if we dismiss it as a special case that can be served with custom code, there's also e.g. captured video or audio that can be downgraded in quality for slow connections. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
27.07.2009, в 13:20, Jeremy Orlow написал(а): I agree that this will help if the application sends data in burst mode, but what if it just constantly sends more than the network can transmit? It will never learn that it's misbehaving, and will just take more and more memory. An example where adapting to network bandwidth is needed is of course file uploading, but even if we dismiss it as a special case that can be served with custom code, there's also e.g. captured video or audio that can be downgraded in quality for slow connections. Maybe the right behavior is to buffer in user-space (like Maciej explained) up until a limit (left up to the UA) and then anything beyond that results in an exception. This seems like it'd handle bursty communication and would keep the failure model simple. This sounds like the best approach to me. 27.07.2009, в 13:27, Drew Wilson написал(а): I would suggest that the solution to this situation is an appropriate application-level protocol (i.e. acks) to allow the application to have no more than (say) 1MB of data outstanding. I'm just afraid that we're burdening the API to handle degenerative cases that the vast majority of users won't encounter. Specifying in the API that any arbitrary send() invocation could throw some kind of retry exception or return some kind of error code is really really cumbersome. Having a send() that doesn't return anything and doesn't raise exceptions would be a clear signal that send() just blocks until it's possible to send data to me, and I'm sure to many others, as well. There is no reason to silently drop data sent over a TCP connection - after all, we could as well base the protocol on UDP if we did, and lose nothing. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
On Mon, Jul 27, 2009 at 1:36 PM, Alexey Proskuryakov a...@webkit.org wrote: 27.07.2009, в 13:20, Jeremy Orlow написал(а): I agree that this will help if the application sends data in burst mode, but what if it just constantly sends more than the network can transmit? It will never learn that it's misbehaving, and will just take more and more memory. An example where adapting to network bandwidth is needed is of course file uploading, but even if we dismiss it as a special case that can be served with custom code, there's also e.g. captured video or audio that can be downgraded in quality for slow connections. Maybe the right behavior is to buffer in user-space (like Maciej explained) up until a limit (left up to the UA) and then anything beyond that results in an exception. This seems like it'd handle bursty communication and would keep the failure model simple. This sounds like the best approach to me. 27.07.2009, в 13:27, Drew Wilson написал(а): I would suggest that the solution to this situation is an appropriate application-level protocol (i.e. acks) to allow the application to have no more than (say) 1MB of data outstanding. I'm just afraid that we're burdening the API to handle degenerative cases that the vast majority of users won't encounter. Specifying in the API that any arbitrary send() invocation could throw some kind of retry exception or return some kind of error code is really really cumbersome. Having a send() that doesn't return anything and doesn't raise exceptions would be a clear signal that send() just blocks until it's possible to send data to me, and I'm sure to many others, as well. There is no reason to silently drop data sent over a TCP connection - after all, we could as well base the protocol on UDP if we did, and lose nothing. There's another option besides blocking, raising an exception, and dropping data: unlimited buffering in user space. So I'm saying we should not put any limits on the amount of user-space buffering we're willing to do, any more than we put any limits on the amount of other types of user-space memory allocation a page can perform. - WBR, Alexey Proskuryakov
Re: [whatwg] Issues with Web Sockets API
On Mon, Jul 27, 2009 at 1:44 PM, Drew Wilson atwil...@google.com wrote: On Mon, Jul 27, 2009 at 1:36 PM, Alexey Proskuryakov a...@webkit.orgwrote: 27.07.2009, в 13:20, Jeremy Orlow написал(а): I agree that this will help if the application sends data in burst mode, but what if it just constantly sends more than the network can transmit? It will never learn that it's misbehaving, and will just take more and more memory. An example where adapting to network bandwidth is needed is of course file uploading, but even if we dismiss it as a special case that can be served with custom code, there's also e.g. captured video or audio that can be downgraded in quality for slow connections. Maybe the right behavior is to buffer in user-space (like Maciej explained) up until a limit (left up to the UA) and then anything beyond that results in an exception. This seems like it'd handle bursty communication and would keep the failure model simple. This sounds like the best approach to me. 27.07.2009, в 13:27, Drew Wilson написал(а): I would suggest that the solution to this situation is an appropriate application-level protocol (i.e. acks) to allow the application to have no more than (say) 1MB of data outstanding. I'm just afraid that we're burdening the API to handle degenerative cases that the vast majority of users won't encounter. Specifying in the API that any arbitrary send() invocation could throw some kind of retry exception or return some kind of error code is really really cumbersome. Having a send() that doesn't return anything and doesn't raise exceptions would be a clear signal that send() just blocks until it's possible to send data to me, and I'm sure to many others, as well. There is no reason to silently drop data sent over a TCP connection - after all, we could as well base the protocol on UDP if we did, and lose nothing. There's another option besides blocking, raising an exception, and dropping data: unlimited buffering in user space. So I'm saying we should not put any limits on the amount of user-space buffering we're willing to do, any more than we put any limits on the amount of other types of user-space memory allocation a page can perform. I agree with Alexey that applications need feedback when they're consistentiently exceeding what your net connection can handle. I think an application getting an exception rather than filling up its buffer until it OOMs is a much better experience for the user and the web developer. If you have application level ACKs (which you probably should--especially in high-throughput uses), you really shouldn't even hit the buffer limits that a UA might have in place. I don't really think that having a limit on the buffer size is a problem and that, if anything, it'll promote better application level flow control. J
Re: [whatwg] Issues with Web Sockets API
On Mon, Jul 27, 2009 at 2:02 PM, Jeremy Orlow jor...@chromium.org wrote: On Mon, Jul 27, 2009 at 1:44 PM, Drew Wilson atwil...@google.com wrote: On Mon, Jul 27, 2009 at 1:36 PM, Alexey Proskuryakov a...@webkit.orgwrote: 27.07.2009, в 13:20, Jeremy Orlow написал(а): I agree that this will help if the application sends data in burst mode, but what if it just constantly sends more than the network can transmit? It will never learn that it's misbehaving, and will just take more and more memory. An example where adapting to network bandwidth is needed is of course file uploading, but even if we dismiss it as a special case that can be served with custom code, there's also e.g. captured video or audio that can be downgraded in quality for slow connections. Maybe the right behavior is to buffer in user-space (like Maciej explained) up until a limit (left up to the UA) and then anything beyond that results in an exception. This seems like it'd handle bursty communication and would keep the failure model simple. This sounds like the best approach to me. 27.07.2009, в 13:27, Drew Wilson написал(а): I would suggest that the solution to this situation is an appropriate application-level protocol (i.e. acks) to allow the application to have no more than (say) 1MB of data outstanding. I'm just afraid that we're burdening the API to handle degenerative cases that the vast majority of users won't encounter. Specifying in the API that any arbitrary send() invocation could throw some kind of retry exception or return some kind of error code is really really cumbersome. Having a send() that doesn't return anything and doesn't raise exceptions would be a clear signal that send() just blocks until it's possible to send data to me, and I'm sure to many others, as well. There is no reason to silently drop data sent over a TCP connection - after all, we could as well base the protocol on UDP if we did, and lose nothing. There's another option besides blocking, raising an exception, and dropping data: unlimited buffering in user space. So I'm saying we should not put any limits on the amount of user-space buffering we're willing to do, any more than we put any limits on the amount of other types of user-space memory allocation a page can perform. I agree with Alexey that applications need feedback when they're consistentiently exceeding what your net connection can handle. I think an application getting an exception rather than filling up its buffer until it OOMs is a much better experience for the user and the web developer. I'm assuming that no actual limits would be specified in the specification, so it would be entirely up to a given UserAgent to decide how much buffering it is willing to provide. Doesn't that imply that a well-behaved web application would be forced to check for exceptions from all send() invocations, since there's no way to know a priori whether limits imposed by an application via its app-level protocol would be sufficient to stay under a given user-agent's internal limits? Even worse, to be broadly deployable the app-level protocol would have to enforce the lowest-common-denominator buffering limit, which would inhibit throughput on platforms that support higher buffers. In practice, I suspect most implementations would adopt a just blast out as much data as possible until the system throws an exception, then set a timer to retry the send in 100ms approach. But perhaps that's your intention? If so, then I'd suggest changing the API to just have a canWrite notification like other async socket APIs provide (or something similar) to avoid the clunky catch-and-retry idiom. Personally, I think that's overkill for the vast majority of use cases which would be more than happy with a simple send(), and I'm not sure why we're obsessing over limiting memory usage in this case when we allow pages to use arbitrary amounts of memory elsewhere. If you have application level ACKs (which you probably should--especially in high-throughput uses), you really shouldn't even hit the buffer limits that a UA might have in place. I don't really think that having a limit on the buffer size is a problem and that, if anything, it'll promote better application level flow control. J
Re: [whatwg] Issues with Web Sockets API
On Jul 27, 2009, at 2:14 PM, Alexey Proskuryakov wrote: 27.07.2009, в 12:35, Maciej Stachowiak написал(а): However, I do not think that raising an exception is an appropriate answer. Often, the TCP implementation takes a part of data given to it, and asks to resubmit the rest later. So, just returning an integer result from send() would be best in my opinion. With WebSocket, another possibility is for the implementation to buffer pending data that could not yet be sent to the TCP layer, so that the client of WebSocket doesn't have to be exposed to system limitations. At that point, an exception is only needed if the implementation runs out of memory for buffering. With a system TCP implementation, the buffering would be in kernel space, which is a scarce resource, but user space memory inside the implementation is no more scarce than user space memory held by the Web application waiting to send to the WebSocket. I agree that this will help if the application sends data in burst mode, but what if it just constantly sends more than the network can transmit? It will never learn that it's misbehaving, and will just take more and more memory. An example where adapting to network bandwidth is needed is of course file uploading, but even if we dismiss it as a special case that can be served with custom code, there's also e.g. captured video or audio that can be downgraded in quality for slow connections. If an application could usefully choose to do something other than buffer in memory (as applies to both of your examples), then yes, it would be useful to tell it when to back off on the send rate. But this could also be combined with buffering inside the implementation but outside the kernel, so the client of WebSocket never has to resend whole or partial packets, it can just note that it should back off on the send rate, and delay future packets. Regards, Maciej
Re: [whatwg] Issues with Web Sockets API
On Jul 27, 2009, at 2:44 PM, Drew Wilson wrote: There's another option besides blocking, raising an exception, and dropping data: unlimited buffering in user space. So I'm saying we should not put any limits on the amount of user-space buffering we're willing to do, any more than we put any limits on the amount of other types of user-space memory allocation a page can perform. I think even unlimited buffering needs to be combined with at least a hint to the WebSocket client to back off the send rate, because it's possible to send so much data that it exceeds the available address space, for example when uploading a very large file piece by piece, or when sending a live media stream that requires more bandwidth than the connection can deliver. In the first case, it is possible, though highly undesirable, to spool the data to be sent to disk; in the latter case, doing that would just inevitably fill the disk. Obviously we need more web platform capabilities to make such use cases a reality, but they are foreseeable and we should deal with them in some reasonable way. Regards, Maciej
Re: [whatwg] Issues with Web Sockets API
Why not just allow unlimited buffering, but also provide an API to query how much data is currently buffered (approximate only, so it would be OK to just return the size of data buffered in user space)? Then applications that care and can adapt can do so. But most applications will not need to. The problem of partial writes being incorrectly handled is pernicious and I definitely think partial writes should not be exposed to applications. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Issues with Web Sockets API
Obviously we need more web platform capabilities to make such use cases a reality, but they are foreseeable and we should deal with them in some reasonable way. Couldn't agree more. The proposed websocket interface is too dumbed down. The caller doesn't know what the impl is doing, and the impl doesn't know what the caller is trying to do. As a consequence, there is no reasonable action that either can take when buffers start overflowing. Typically, the network layer provides sufficient status info to its caller that, allowing the higher level code to do something reasonable in light of how the network layer is performing. That kind of status info is simply missing from the websocket interface. I think its possible to add to the interface features that would facilitate more demanding uses cases without complicating the simple use cases. I think that would be an excellent goal for this API. On Mon, Jul 27, 2009 at 5:30 PM, Maciej Stachowiak m...@apple.com wrote: On Jul 27, 2009, at 2:44 PM, Drew Wilson wrote: There's another option besides blocking, raising an exception, and dropping data: unlimited buffering in user space. So I'm saying we should not put any limits on the amount of user-space buffering we're willing to do, any more than we put any limits on the amount of other types of user-space memory allocation a page can perform. I think even unlimited buffering needs to be combined with at least a hint to the WebSocket client to back off the send rate, because it's possible to send so much data that it exceeds the available address space, for example when uploading a very large file piece by piece, or when sending a live media stream that requires more bandwidth than the connection can deliver. In the first case, it is possible, though highly undesirable, to spool the data to be sent to disk; in the latter case, doing that would just inevitably fill the disk. Obviously we need more web platform capabilities to make such use cases a reality, but they are foreseeable and we should deal with them in some reasonable way. Regards, Maciej
Re: [whatwg] Issues with Web Sockets API
On Mon, Jul 6, 2009 at 9:30 PM, Ian Hickson i...@hixie.ch wrote: 1) The 'readyState' attribute can never actually be used by an application and is redundant. Initially, the 'readyState' attribute is set to CONNECTING, but while the object is in this state the user is not permitted to interact with the WebSocket in any way. The only useful thing that a user could do is set event handlers and wait for the 'open' event to fire. When the WebSocket becomes connected, the readyState becomes 1 and the 'open' event is fired. Once the WebSocket is open, the spec states that whenever the connection is closed the readyState changes to CLOSED and a 'close' event is enqueued. However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. A user will have to wrap all calls to postMessage() (or send() if the function is renamed) in a try/catch block in order to handle INVALID_STATE_ERRs. Once the 'close' event has been received the readyState attribute is useless since the state of the WebSocket is known and can never change. I think 'readyState' should just go away since an application will have to keep track of state updates through the fired events and use try/catch blocks around all API calls anyway. The attribute is mostly present for debugging purposes. I wouldn't expect anyone to actually use it for production work. Is there precedent for other portions of the API that are mostly for debugging purposes? (I can't think of anything off the top of my head.) Also, maybe it should be noted as such in the spec? On Fri, 26 Jun 2009, James Robinson wrote: Not changing variables out from under executing JavaScript makes a lot of sense, but if that was the case then it's not clear when the readyState could be updated. The spec states When the *Web Socket connection is closed*, the readyState attribute's value must be changed to CLOSED (2), and the user agent must queue a task to fire a simple event called close at the WebSocket object. If the browser cannot mutate the readyState until JavaScript stops running then it would either have to either enqueue a second task to change readyState at some point in the future or set the readyState right before dispatching the 'close' event. The latter would be much nicer to implement - but then it does make the readyState completely useless as it would always be exactly equivalent to the last event that was fired on a given WebSocket. I've left it as is (the attribute changes on the fly), which is possibly risky, but more consistent with how such attributes are handled in general. If it's only for debugging purposes, maybe a cleaner way to define it is to simply be the last event fired on a given WebSocket? One other random question: in the IDL for WebSockets, the three constants for ready state are all defined as shorts but the value of ready state is a long. Is this an oversight? J
Re: [whatwg] Issues with Web Sockets API
On Mon, Jul 6, 2009 at 9:30 PM, Ian Hickson i...@hixie.ch wrote: On Fri, 26 Jun 2009, James Robinson wrote: 0) postMessage() looks as if it is intended to mimic MessagePort.postMessage(), but the arguments and error conditions are different. While it would be conceptually nice to treat a web socket in the same way as a message port, it's not possible to treat the two postMessage() functions in the same way. I'd recommend the WebSocket version be renamed to something like send() to avoid confusion and false expectations. Fair enough. Done. There's similar oddness with receiving events that satisfy the MessageEvent interface - since all fields except 'data' will necessarily be invalid I don't see the value in receiving something more complex. I would like to avoid introducing four different event types for the four different types of 'message' events being introduced, which is why I overloaded the same interface for all four. I don't think it's a problem. 1) The 'readyState' attribute can never actually be used by an application and is redundant. Initially, the 'readyState' attribute is set to CONNECTING, but while the object is in this state the user is not permitted to interact with the WebSocket in any way. The only useful thing that a user could do is set event handlers and wait for the 'open' event to fire. When the WebSocket becomes connected, the readyState becomes 1 and the 'open' event is fired. Once the WebSocket is open, the spec states that whenever the connection is closed the readyState changes to CLOSED and a 'close' event is enqueued. However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. A user will have to wrap all calls to postMessage() (or send() if the function is renamed) in a try/catch block in order to handle INVALID_STATE_ERRs. Once the 'close' event has been received the readyState attribute is useless since the state of the WebSocket is known and can never change. I think 'readyState' should just go away since an application will have to keep track of state updates through the fired events and use try/catch blocks around all API calls anyway. The attribute is mostly present for debugging purposes. I wouldn't expect anyone to actually use it for production work. On Fri, 26 Jun 2009, Drew Wilson wrote: Yes, but the closed state of a given WebSocket doesn't have to exactly match the state of the underlying TCP connection, in the same way that document.cookies doesn't exactly match the current set of cookies that the network stack may be tracking (they can differ when HTTP responses are received in the background while JS is executing). So if the remote server closes the TCP connection, it generates a close event which marks the WebSocket as closed. It means that you could have a situation where you post messages to a WebSocket which aren't received by the server because the connection is closed, but that's true regardless due to the asynchronous nature of the networking protocol. I've changed the spec to not throw an exception on send() if the connection is closed. On Fri, 26 Jun 2009, Kelly Norton wrote: One thing about postMessage that I'm curious about. Since it has to report failure synchronously by throwing an INVALID_STATE_ERR, that seems to imply that all data must be written to a socket before returning and cannot be asynchronously delivered to an I/O thread without adding some risk of silently dropping messages. Seems like the right choice would be to allow outbound messages to drop, which would mean that developers would be forced to do their own handshaking. send() doesn't report I/O errors, it only throws an exception if the connection isn't open yet (and previously, if the connection had died prior to it being called, though that is no longer the case), or if the input is invalid. I'm also not sure there is good coverage of error conditions in the spec. The only methods of error notification are exceptions in postMessage and onclose. In fact, only onclose actually reports an error; the exception from send() now only reports a misuse of the API, not a network error. I had assumed that a WebSocket that fails to connect would invoke onclose asynchronously, but I didn't see that in the spec. It's there, though subtle. :-) The constructor in the API spec invokes the Establish a Web Socket connection algorithm from the protocol spec. That algorithm then invokes the fail the Web Socket connection algorithm upon failure, and that algorithm says to invoke the close the Web Socket connection algorithm, and that algorithm says that this means that Web Socket connection is closed, and the API spec says When the Web Socket connection is closed, the readyState attribute's value
Re: [whatwg] Issues with Web Sockets API
On Tue, 7 Jul 2009, Michael Nordman wrote: I think when we add support for file upload, we'll make it so that it automagically supports this case. That is, you'll say upload this file in small bits and then if you later say send this text message, the text message will be sent before any pending file bits. We can use a separate type of packet in the WebSocket stream to do this. Sounds like that would require a protocol change. Is the message framing spec'd in such a way that a new 'packet type' can be introduced in a backward compatible fashion? Yes. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Issues with Web Sockets API
On Fri, 26 Jun 2009, James Robinson wrote: 0) postMessage() looks as if it is intended to mimic MessagePort.postMessage(), but the arguments and error conditions are different. While it would be conceptually nice to treat a web socket in the same way as a message port, it's not possible to treat the two postMessage() functions in the same way. I'd recommend the WebSocket version be renamed to something like send() to avoid confusion and false expectations. Fair enough. Done. There's similar oddness with receiving events that satisfy the MessageEvent interface - since all fields except 'data' will necessarily be invalid I don't see the value in receiving something more complex. I would like to avoid introducing four different event types for the four different types of 'message' events being introduced, which is why I overloaded the same interface for all four. I don't think it's a problem. 1) The 'readyState' attribute can never actually be used by an application and is redundant. Initially, the 'readyState' attribute is set to CONNECTING, but while the object is in this state the user is not permitted to interact with the WebSocket in any way. The only useful thing that a user could do is set event handlers and wait for the 'open' event to fire. When the WebSocket becomes connected, the readyState becomes 1 and the 'open' event is fired. Once the WebSocket is open, the spec states that whenever the connection is closed the readyState changes to CLOSED and a 'close' event is enqueued. However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. A user will have to wrap all calls to postMessage() (or send() if the function is renamed) in a try/catch block in order to handle INVALID_STATE_ERRs. Once the 'close' event has been received the readyState attribute is useless since the state of the WebSocket is known and can never change. I think 'readyState' should just go away since an application will have to keep track of state updates through the fired events and use try/catch blocks around all API calls anyway. The attribute is mostly present for debugging purposes. I wouldn't expect anyone to actually use it for production work. On Fri, 26 Jun 2009, Drew Wilson wrote: Yes, but the closed state of a given WebSocket doesn't have to exactly match the state of the underlying TCP connection, in the same way that document.cookies doesn't exactly match the current set of cookies that the network stack may be tracking (they can differ when HTTP responses are received in the background while JS is executing). So if the remote server closes the TCP connection, it generates a close event which marks the WebSocket as closed. It means that you could have a situation where you post messages to a WebSocket which aren't received by the server because the connection is closed, but that's true regardless due to the asynchronous nature of the networking protocol. I've changed the spec to not throw an exception on send() if the connection is closed. On Fri, 26 Jun 2009, Kelly Norton wrote: One thing about postMessage that I'm curious about. Since it has to report failure synchronously by throwing an INVALID_STATE_ERR, that seems to imply that all data must be written to a socket before returning and cannot be asynchronously delivered to an I/O thread without adding some risk of silently dropping messages. Seems like the right choice would be to allow outbound messages to drop, which would mean that developers would be forced to do their own handshaking. send() doesn't report I/O errors, it only throws an exception if the connection isn't open yet (and previously, if the connection had died prior to it being called, though that is no longer the case), or if the input is invalid. I'm also not sure there is good coverage of error conditions in the spec. The only methods of error notification are exceptions in postMessage and onclose. In fact, only onclose actually reports an error; the exception from send() now only reports a misuse of the API, not a network error. I had assumed that a WebSocket that fails to connect would invoke onclose asynchronously, but I didn't see that in the spec. It's there, though subtle. :-) The constructor in the API spec invokes the Establish a Web Socket connection algorithm from the protocol spec. That algorithm then invokes the fail the Web Socket connection algorithm upon failure, and that algorithm says to invoke the close the Web Socket connection algorithm, and that algorithm says that this means that Web Socket connection is closed, and the API spec says When the Web Socket connection is closed, the readyState attribute's value must be changed to CLOSED (2), and the user agent must queue a task to fire a simple event
Re: [whatwg] Issues with Web Sockets API
On Sat, Jun 27, 2009 at 3:18 PM, Jeff Walden jwalden+wha...@mit.edujwalden%2bwha...@mit.edu wrote: On 26.6.09 16:49, Michael Nordman wrote: Progress bars are routinely implemented without get hi-level application acks from the other side. XMLHttpRequest.upload.onprogress is one such example. That they can be implemented this way does not imply they must be implemented this way. I don't see why the acks users will pretty much invariably have (if they're implementing something so complex as to want data-sent events) layered atop the base WebSocket don't suffice for this (and more accurately than simple out-of-the queue status notifications can establish). diagnostics Cell-phone signal strength bars are a form of diagnostics... existence proof of diagnostics being a significant use case. Is WebSocket the optimal way to satisfy that use case? (Also, to be clear, I wasn't suggesting that diagnostics aren't interesting, but they seem quite orthogonal to the primary use case of supporting two-way message passing.) This info about the status of the WebSocket would be easy to provide to callers of this API. There are easily found valid use cases for this additional status info. What compelling reason is there to not do so? Seems like low-hanging fruit if you ask me. The use case may be valid, but I don't see it as compelling. I don't see the gain as worth the added complexity to the interface, which at present is exactly as simple as it can be to support two-way message passing. I expect the messages passed will usually follow send-ack sequencing (in one direction or the other, and particularly for users who care about exact data transmission), in which case reception of an ack signals progress has been made. Jeff I didn't lobby for a particular use case, so not sure which of the handful i alluded to was deemed valid but not compelling. The point was there probably is a use case that would be deemed compelling once it was unleashed on the world. As for complexity websocket.onmessagesent attribute hardly qualifies? I stand by my low hanging fruit characterization :)
Re: [whatwg] Issues with Web Sockets API
Cellular phone signal strength bars are not implemented on top of Web Sockets so this cannot make a use case. This is rather a hardware thing. On the other hand, I concur that a network queue overflow should be handled differently than an out-of-memory condition. For example, if you have an infinite enumerator sending data to memory à la chargen service, an overflow failure is guaranteed and the programmer will have to do something about it; on the other hand, if the same enumerator sends data to a network, the failure will be random, depending on enumerator speed, network throughput and receiver capabilities. I think that the output buffer should not grow unbounded and the sending worker should be stopped while the buffer is full. Cheers, Chris
Re: [whatwg] Issues with Web Sockets API
Given the evident complexity of the Web Sockets protocol with respect to acknowledgements, events and updating host state for the script, is there a modelling diagram to view? Chris
Re: [whatwg] Issues with Web Sockets API
On 26.6.09 16:49, Michael Nordman wrote: Progress bars are routinely implemented without get hi-level application acks from the other side. XMLHttpRequest.upload.onprogress is one such example. That they can be implemented this way does not imply they must be implemented this way. I don't see why the acks users will pretty much invariably have (if they're implementing something so complex as to want data-sent events) layered atop the base WebSocket don't suffice for this (and more accurately than simple out-of-the queue status notifications can establish). diagnostics Cell-phone signal strength bars are a form of diagnostics... existence proof of diagnostics being a significant use case. Is WebSocket the optimal way to satisfy that use case? (Also, to be clear, I wasn't suggesting that diagnostics aren't interesting, but they seem quite orthogonal to the primary use case of supporting two-way message passing.) This info about the status of the WebSocket would be easy to provide to callers of this API. There are easily found valid use cases for this additional status info. What compelling reason is there to not do so? Seems like low-hanging fruit if you ask me. The use case may be valid, but I don't see it as compelling. I don't see the gain as worth the added complexity to the interface, which at present is exactly as simple as it can be to support two-way message passing. I expect the messages passed will usually follow send-ack sequencing (in one direction or the other, and particularly for users who care about exact data transmission), in which case reception of an ack signals progress has been made. Jeff
[whatwg] Issues with Web Sockets API
Hello, I'm very excited about the concept of web sockets and look forward to building apps with it but the web sockets API at http://dev.w3.org/html5/websockets/ has some issues. Many issues seem to be inherited from the original XmlHttpRequest specification, which was extremely useful but not a very good spec. I'm sure I'm not the only one who has spent far too many hours dealing with underspecified or poorly implemented XHR flavors and would love to avoid doing that in the future. I know several vendors have started work on implementations already but I hope that this feedback is still useful. 0) postMessage() looks as if it is intended to mimic MessagePort.postMessage() or , but the arguments and error conditions are different. While it would be conceptually nice to treat a web socket in the same way as a message port, it's not possible to treat the two postMessage() functions in the same way. I'd recommend the WebSocket version be renamed to something like send() to avoid confusion and false expectations. There's similar oddness with receiving events that satisfy the MessageEvent interface - since all fields except 'data' will necessarily be invalid I don't see the value in receiving something more complex. 1) The 'readyState' attribute can never actually be used by an application and is redundant. Initially, the 'readyState' attribute is set to CONNECTING, but while the object is in this state the user is not permitted to interact with the WebSocket in any way. The only useful thing that a user could do is set event handlers and wait for the 'open' event to fire. When the WebSocket becomes connected, the readyState becomes 1 and the 'open' event is fired. Once the WebSocket is open, the spec states that whenever the connection is closed the readyState changes to CLOSED and a 'close' event is enqueued. However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. A user will have to wrap all calls to postMessage() (or send() if the function is renamed) in a try/catch block in order to handle INVALID_STATE_ERRs. Once the 'close' event has been received the readyState attribute is useless since the state of the WebSocket is known and can never change. I think 'readyState' should just go away since an application will have to keep track of state updates through the fired events and use try/catch blocks around all API calls anyway. - James
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 9:18 AM, James Robinson jam...@google.com wrote: However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. Is this true? Based on our prior discussion surrounding cookies, it seems like as a general rule we try to keep state from changing dynamically while JS code is executing for exactly these reasons.
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 9:46 AM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 9:18 AM, James Robinson jam...@google.com wrote: However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. Is this true? Based on our prior discussion surrounding cookies, it seems like as a general rule we try to keep state from changing dynamically while JS code is executing for exactly these reasons. I think this is a very different beast. The state of a network connection may change asynchronously whether we like it or not. Unlike who may access cookies or local storage, the state of the network connection is not something we solely control. -Darin
Re: [whatwg] Issues with Web Sockets API
Yes, but the closed state of a given WebSocket doesn't have to exactly match the state of the underlying TCP connection, in the same way that document.cookies doesn't exactly match the current set of cookies that the network stack may be tracking (they can differ when HTTP responses are received in the background while JS is executing). So if the remote server closes the TCP connection, it generates a close event which marks the WebSocket as closed. It means that you could have a situation where you post messages to a WebSocket which aren't received by the server because the connection is closed, but that's true regardless due to the asynchronous nature of the networking protocol. -atw On Fri, Jun 26, 2009 at 9:52 AM, Darin Fisher da...@chromium.org wrote: On Fri, Jun 26, 2009 at 9:46 AM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 9:18 AM, James Robinson jam...@google.comwrote: However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. Is this true? Based on our prior discussion surrounding cookies, it seems like as a general rule we try to keep state from changing dynamically while JS code is executing for exactly these reasons. I think this is a very different beast. The state of a network connection may change asynchronously whether we like it or not. Unlike who may access cookies or local storage, the state of the network connection is not something we solely control. -Darin
Re: [whatwg] Issues with Web Sockets API
One thing about postMessage that I'm curious about. Since it has to report failure synchronously by throwing an INVALID_STATE_ERR, that seems to imply that all data must be written to a socket before returning and cannot be asynchronously delivered to an I/O thread without adding some risk of silently dropping messages. Seems like the right choice would be to allow outbound messages to drop, which would mean that developers would be forced to do their own handshaking. I'm also not sure there is good coverage of error conditions in the spec. The only methods of error notification are exceptions in postMessage and onclose. I had assumed that a WebSocket that fails to connect would invoke onclose asynchronously, but I didn't see that in the spec. Without that you don't even have the ability to know if a socket failed to establish a connection (short of readyState polling). The spec also doesn't indicate that the readyState should transition to CLOSED on connection failure. (Description of the disconnect() method is careful to mention that it closes a connection or a connection attempt, but description of when onclose is fired just mentions a connection closing). I definitely think there should be a way to receive an event if a connection fails to establish; I would hate to have to poll another readyState. /kel On Fri, Jun 26, 2009 at 1:34 PM, Drew Wilson atwil...@google.com wrote: Yes, but the closed state of a given WebSocket doesn't have to exactly match the state of the underlying TCP connection, in the same way that document.cookies doesn't exactly match the current set of cookies that the network stack may be tracking (they can differ when HTTP responses are received in the background while JS is executing). So if the remote server closes the TCP connection, it generates a close event which marks the WebSocket as closed. It means that you could have a situation where you post messages to a WebSocket which aren't received by the server because the connection is closed, but that's true regardless due to the asynchronous nature of the networking protocol. -atw On Fri, Jun 26, 2009 at 9:52 AM, Darin Fisher da...@chromium.org wrote: On Fri, Jun 26, 2009 at 9:46 AM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 9:18 AM, James Robinson jam...@google.comwrote: However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. Is this true? Based on our prior discussion surrounding cookies, it seems like as a general rule we try to keep state from changing dynamically while JS code is executing for exactly these reasons. I think this is a very different beast. The state of a network connection may change asynchronously whether we like it or not. Unlike who may access cookies or local storage, the state of the network connection is not something we solely control. -Darin -- If you received this communication by mistake, you are entitled to one free ice cream cone on me. Simply print out this email including all relevant SMTP headers and present them at my desk to claim your creamy treat. We'll have a laugh at my emailing incompetence, and play a game of ping pong. (offer may not be valid in all States).
Re: [whatwg] Issues with Web Sockets API
Oh and one more thing: Doesn't it seem strange that disconnect() causes an onclose event to be dispatched? Should the method not be close() to be consistent with open(), onopen, onclose? /kel On Fri, Jun 26, 2009 at 4:14 PM, Kelly Norton knor...@google.com wrote: One thing about postMessage that I'm curious about. Since it has to report failure synchronously by throwing an INVALID_STATE_ERR, that seems to imply that all data must be written to a socket before returning and cannot be asynchronously delivered to an I/O thread without adding some risk of silently dropping messages. Seems like the right choice would be to allow outbound messages to drop, which would mean that developers would be forced to do their own handshaking. I'm also not sure there is good coverage of error conditions in the spec. The only methods of error notification are exceptions in postMessage and onclose. I had assumed that a WebSocket that fails to connect would invoke onclose asynchronously, but I didn't see that in the spec. Without that you don't even have the ability to know if a socket failed to establish a connection (short of readyState polling). The spec also doesn't indicate that the readyState should transition to CLOSED on connection failure. (Description of the disconnect() method is careful to mention that it closes a connection or a connection attempt, but description of when onclose is fired just mentions a connection closing). I definitely think there should be a way to receive an event if a connection fails to establish; I would hate to have to poll another readyState. /kel On Fri, Jun 26, 2009 at 1:34 PM, Drew Wilson atwil...@google.com wrote: Yes, but the closed state of a given WebSocket doesn't have to exactly match the state of the underlying TCP connection, in the same way that document.cookies doesn't exactly match the current set of cookies that the network stack may be tracking (they can differ when HTTP responses are received in the background while JS is executing). So if the remote server closes the TCP connection, it generates a close event which marks the WebSocket as closed. It means that you could have a situation where you post messages to a WebSocket which aren't received by the server because the connection is closed, but that's true regardless due to the asynchronous nature of the networking protocol. -atw On Fri, Jun 26, 2009 at 9:52 AM, Darin Fisher da...@chromium.org wrote: On Fri, Jun 26, 2009 at 9:46 AM, Drew Wilson atwil...@google.comwrote: On Fri, Jun 26, 2009 at 9:18 AM, James Robinson jam...@google.comwrote: However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. Is this true? Based on our prior discussion surrounding cookies, it seems like as a general rule we try to keep state from changing dynamically while JS code is executing for exactly these reasons. I think this is a very different beast. The state of a network connection may change asynchronously whether we like it or not. Unlike who may access cookies or local storage, the state of the network connection is not something we solely control. -Darin -- If you received this communication by mistake, you are entitled to one free ice cream cone on me. Simply print out this email including all relevant SMTP headers and present them at my desk to claim your creamy treat. We'll have a laugh at my emailing incompetence, and play a game of ping pong. (offer may not be valid in all States). -- If you received this communication by mistake, you are entitled to one free ice cream cone on me. Simply print out this email including all relevant SMTP headers and present them at my desk to claim your creamy treat. We'll have a laugh at my emailing incompetence, and play a game of ping pong. (offer may not be valid in all States).
Re: [whatwg] Issues with Web Sockets API
Does disconnect() attempt to flush pending messages or drop them? There isn't a way to determine if the WebSocket is successfully sending the postMessage data? For all the caller knows, its just backing up and not going anywhere. Something that might add value is an onmessagesent event that fires after a postMessage has put the bits on the wire. postMessage() may want another exception condition... 'too much data pending exception'... consider calling postMessage in a while(true) loop... at some point the system is going to have to give up queing the data if its not actually making its way out on the wire. On Fri, Jun 26, 2009 at 1:19 PM, Kelly Norton knor...@google.com wrote: Oh and one more thing: Doesn't it seem strange that disconnect() causes an onclose event to be dispatched? Should the method not be close() to be consistent with open(), onopen, onclose? /kel On Fri, Jun 26, 2009 at 4:14 PM, Kelly Norton knor...@google.com wrote: One thing about postMessage that I'm curious about. Since it has to report failure synchronously by throwing an INVALID_STATE_ERR, that seems to imply that all data must be written to a socket before returning and cannot be asynchronously delivered to an I/O thread without adding some risk of silently dropping messages. Seems like the right choice would be to allow outbound messages to drop, which would mean that developers would be forced to do their own handshaking. I'm also not sure there is good coverage of error conditions in the spec. The only methods of error notification are exceptions in postMessage and onclose. I had assumed that a WebSocket that fails to connect would invoke onclose asynchronously, but I didn't see that in the spec. Without that you don't even have the ability to know if a socket failed to establish a connection (short of readyState polling). The spec also doesn't indicate that the readyState should transition to CLOSED on connection failure. (Description of the disconnect() method is careful to mention that it closes a connection or a connection attempt, but description of when onclose is fired just mentions a connection closing). I definitely think there should be a way to receive an event if a connection fails to establish; I would hate to have to poll another readyState. /kel On Fri, Jun 26, 2009 at 1:34 PM, Drew Wilson atwil...@google.com wrote: Yes, but the closed state of a given WebSocket doesn't have to exactly match the state of the underlying TCP connection, in the same way that document.cookies doesn't exactly match the current set of cookies that the network stack may be tracking (they can differ when HTTP responses are received in the background while JS is executing). So if the remote server closes the TCP connection, it generates a close event which marks the WebSocket as closed. It means that you could have a situation where you post messages to a WebSocket which aren't received by the server because the connection is closed, but that's true regardless due to the asynchronous nature of the networking protocol. -atw On Fri, Jun 26, 2009 at 9:52 AM, Darin Fisher da...@chromium.orgwrote: On Fri, Jun 26, 2009 at 9:46 AM, Drew Wilson atwil...@google.comwrote: On Fri, Jun 26, 2009 at 9:18 AM, James Robinson jam...@google.comwrote: However, users can't usefully check the readyState to see if the WebSocket is still open because there are not and cannot be any synchronization guarantees about when the WebSocket may close. Is this true? Based on our prior discussion surrounding cookies, it seems like as a general rule we try to keep state from changing dynamically while JS code is executing for exactly these reasons. I think this is a very different beast. The state of a network connection may change asynchronously whether we like it or not. Unlike who may access cookies or local storage, the state of the network connection is not something we solely control. -Darin -- If you received this communication by mistake, you are entitled to one free ice cream cone on me. Simply print out this email including all relevant SMTP headers and present them at my desk to claim your creamy treat. We'll have a laugh at my emailing incompetence, and play a game of ping pong. (offer may not be valid in all States). -- If you received this communication by mistake, you are entitled to one free ice cream cone on me. Simply print out this email including all relevant SMTP headers and present them at my desk to claim your creamy treat. We'll have a laugh at my emailing incompetence, and play a game of ping pong. (offer may not be valid in all States).
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 1:14 PM, Kelly Norton knor...@google.com wrote: One thing about postMessage that I'm curious about. Since it has to report failure synchronously by throwing an INVALID_STATE_ERR, that seems to imply that all data must be written to a socket before returning and cannot be asynchronously delivered to an I/O thread without adding some risk of silently dropping messages. I don't think that's the intent of the spec - the intent is that INVALID_STATE_ERR is sent if the port is in a closed state, not if there's an I/O error after send. But Michael's right, I don't think there's any way to determine that the server received the message - I guess the intent is that applications will build their own send/ack protocol on top of postMessage(), as you note. -atw
Re: [whatwg] Issues with Web Sockets API
Not changing variables out from under executing JavaScript makes a lot of sense, but if that was the case then it's not clear when the readyState could be updated. The spec states When the *Web Socket connection is closed*, the readyStatehttp://dev.w3.org/html5/websockets/#dom-websocket-readystate attribute's value must be changed to CLOSEDhttp://dev.w3.org/html5/websockets/#dom-websocket-closed (2), and the user agent must queue a task to fire a simple event called close at the WebSocket http://dev.w3.org/html5/websockets/#websocket object. If the browser cannot mutate the readyState until JavaScript stops running then it would either have to either enqueue a second task to change readyState at some point in the future or set the readyState right before dispatching the 'close' event. The latter would be much nicer to implement - but then it does make the readyState completely useless as it would always be exactly equivalent to the last event that was fired on a given WebSocket. The spec seems to be also a bit undecided about error handling. The postMessage() description says If the connection is not established ( readyState http://dev.w3.org/html5/websockets/#dom-websocket-readystate is not OPEN http://dev.w3.org/html5/websockets/#dom-websocket-open), it must raise an INVALID_STATE_ERR exception. The first part implies that trying to postMessage() when the underlying socket is disconnected requires an exception to the thrown synchronously, but if the readyState cannot be changed while javascript is running then the parenthetical would indicate that a browser could not throw an exception until the readyState was updated. An exception for 'too much data pending' has a similar problem - there's no way for the browser to know that there is too much data enqueued without blocking to check with its I/O thread (or process). An asynchronous model would be better with an error signaled by a later callback. I think a better way to do error handling is to have an asynchronous onerror callback or event when the browser notes that a message did not make it to the other side. - James On Fri, Jun 26, 2009 at 4:52 PM, Michael Nordman micha...@google.comwrote: Does disconnect() attempt to flush pending messages or drop them? There isn't a way to determine if the WebSocket is successfully sending the postMessage data? For all the caller knows, its just backing up and not going anywhere. Something that might add value is an onmessagesent event that fires after a postMessage has put the bits on the wire. postMessage() may want another exception condition... 'too much data pending exception'... consider calling postMessage in a while(true) loop... at some point the system is going to have to give up queing the data if its not actually making its way out on the wire. On Fri, Jun 26, 2009 at 1:19 PM, Kelly Norton knor...@google.com wrote: Oh and one more thing: Doesn't it seem strange that disconnect() causes an onclose event to be dispatched? Should the method not be close() to be consistent with open(), onopen, onclose? /kel On Fri, Jun 26, 2009 at 4:14 PM, Kelly Norton knor...@google.com wrote: One thing about postMessage that I'm curious about. Since it has to report failure synchronously by throwing an INVALID_STATE_ERR, that seems to imply that all data must be written to a socket before returning and cannot be asynchronously delivered to an I/O thread without adding some risk of silently dropping messages. Seems like the right choice would be to allow outbound messages to drop, which would mean that developers would be forced to do their own handshaking. I'm also not sure there is good coverage of error conditions in the spec. The only methods of error notification are exceptions in postMessage and onclose. I had assumed that a WebSocket that fails to connect would invoke onclose asynchronously, but I didn't see that in the spec. Without that you don't even have the ability to know if a socket failed to establish a connection (short of readyState polling). The spec also doesn't indicate that the readyState should transition to CLOSED on connection failure. (Description of the disconnect() method is careful to mention that it closes a connection or a connection attempt, but description of when onclose is fired just mentions a connection closing). I definitely think there should be a way to receive an event if a connection fails to establish; I would hate to have to poll another readyState. /kel On Fri, Jun 26, 2009 at 1:34 PM, Drew Wilson atwil...@google.comwrote: Yes, but the closed state of a given WebSocket doesn't have to exactly match the state of the underlying TCP connection, in the same way that document.cookies doesn't exactly match the current set of cookies that the network stack may be tracking (they can differ when HTTP responses are received in the background while JS is executing). So if the remote server closes the TCP connection, it
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 5:01 PM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 1:14 PM, Kelly Norton knor...@google.com wrote: One thing about postMessage that I'm curious about. Since it has to report failure synchronously by throwing an INVALID_STATE_ERR, that seems to imply that all data must be written to a socket before returning and cannot be asynchronously delivered to an I/O thread without adding some risk of silently dropping messages. I don't think that's the intent of the spec - the intent is that INVALID_STATE_ERR is sent if the port is in a closed state, not if there's an I/O error after send. But Michael's right, I don't think there's any way to determine that the server received the message - I guess the intent is that applications will build their own send/ack protocol on top of postMessage(), as you note. -atw The concept of a port being in a closed state is not very well defined - if the state means only the readyState status, then when can the state legally be updated? If it has some meaning closer to the state of the underlying connection, then it can't be queried synchronously without very expensive synching to the I/O thread or process. Forcing applications to build their own send/ack functionality would be pretty tragic considering that WebSockets are built on top of TCP. - James
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 2:11 PM, James Robinson jam...@google.com wrote: Forcing applications to build their own send/ack functionality would be pretty tragic considering that WebSockets are built on top of TCP. - James Every time I've written a response/reply protocol on TCP I've needed to put in my own acks - how else do you know your message has been delivered to the remote app layer? One could argue that WebSockets should do this for you, but I like leaving this up to the app as it gives them more flexibility. -atw
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 3:16 PM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 2:11 PM, James Robinson jam...@google.com wrote: Forcing applications to build their own send/ack functionality would be pretty tragic considering that WebSockets are built on top of TCP. - James Every time I've written a response/reply protocol on TCP I've needed to put in my own acks - how else do you know your message has been delivered to the remote app layer? Classic networking problem... if you do send the ack... how does the ack sender know the other side has received it... and so on. One could argue that WebSockets should do this for you, but I like leaving this up to the app as it gives them more flexibility. Yes. But knowing if the data your queuing to be sent is backing up in your local system instead of being pushed out is different than knowing if the remote side has received it and processed it. The former can be done w/o changing the websocket network protocol, the latter cannot. -atw
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 3:25 PM, Michael Nordman micha...@google.comwrote: On Fri, Jun 26, 2009 at 3:16 PM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 2:11 PM, James Robinson jam...@google.comwrote: Forcing applications to build their own send/ack functionality would be pretty tragic considering that WebSockets are built on top of TCP. - James Every time I've written a response/reply protocol on TCP I've needed to put in my own acks - how else do you know your message has been delivered to the remote app layer? Classic networking problem... if you do send the ack... how does the ack sender know the other side has received it... and so on. Precisely, and complicated by the fact that the app layers I've worked with don't actually expose TCP acks to the app, so you can't even tell that the remote side has acked your packets. One could argue that WebSockets should do this for you, but I like leaving this up to the app as it gives them more flexibility. Yes. But knowing if the data your queuing to be sent is backing up in your local system instead of being pushed out is different than knowing if the remote side has received it and processed it. The former can be done w/o changing the websocket network protocol, the latter cannot. Is the queued up data is backing up problem any different from somebody doing a ton of async XHR requests, some of which may need to be queued before being sent?
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 3:33 PM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 3:25 PM, Michael Nordman micha...@google.comwrote: On Fri, Jun 26, 2009 at 3:16 PM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 2:11 PM, James Robinson jam...@google.comwrote: Forcing applications to build their own send/ack functionality would be pretty tragic considering that WebSockets are built on top of TCP. - James Every time I've written a response/reply protocol on TCP I've needed to put in my own acks - how else do you know your message has been delivered to the remote app layer? Classic networking problem... if you do send the ack... how does the ack sender know the other side has received it... and so on. Precisely, and complicated by the fact that the app layers I've worked with don't actually expose TCP acks to the app, so you can't even tell that the remote side has acked your packets. One could argue that WebSockets should do this for you, but I like leaving this up to the app as it gives them more flexibility. Yes. But knowing if the data your queuing to be sent is backing up in your local system instead of being pushed out is different than knowing if the remote side has received it and processed it. The former can be done w/o changing the websocket network protocol, the latter cannot. Is the queued up data is backing up problem any different from somebody doing a ton of async XHR requests, some of which may need to be queued before being sent? No. But the difference is each XHR tells you when its been sent and gives you the response when its received. With this info, apps can rate limit things. WebSocket.postMessage doesn't tell you when that message has been sent. Suppose your sending 'i'm alive' messages. If the message you sent 5 minutes ago hasn't really been sent, you wouldn't want to queue another 'i'm alive'. If you're uploading a large data set incrementally across many distinct postMessage calls (perhaps to leave room for other control messages interspersed amoungst them, or to present progress info), how do you know when to queue more data to be sent.
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 3:47 PM, Michael Nordman micha...@google.comwrote: No. But the difference is each XHR tells you when its been sent and gives you the response when its received. With this info, apps can rate limit things. WebSocket.postMessage doesn't tell you when that message has been sent. Well, yes and no. You know when you get a response back because readyState = HEADERS_RECEIVED. But there's nothing between OPEN and HEADERS_RECEIVED that tells you anything about bits on the wire. Suppose your sending 'i'm alive' messages. If the message you sent 5 minutes ago hasn't really been sent, you wouldn't want to queue another 'i'm alive'. If your goal is never to send another heartbeat until you know your previous one has been delivered, then that seems like a motivation to add an app-layer heartbeat ack. Treating queued but not yet put on the wire any differently from put on the wire but not yet acked or acked, but still queued for delivery to the app layer on the remote end seems like a false distinction. If you're uploading a large data set incrementally across many distinct postMessage calls (perhaps to leave room for other control messages interspersed amoungst them, or to present progress info), how do you know when to queue more data to be sent. I could keep saying app level acks! but I don't want to beat a dead horse, and honestly, I'm not entirely certain that I'm right :)
Re: [whatwg] Issues with Web Sockets API
On 26.6.09 13:52, Michael Nordman wrote: There isn't a way to determine if the WebSocket is successfully sending the postMessage data? For all the caller knows, its just backing up and not going anywhere. You can't really know data has been successfully sent until you get a response that acknowledges it. Getting it on the wire doesn't mean it'll actually be received; I'm not really sure how much value knowing that's happened actually has, other than for doing network diagnostics (which hardly seems a use case to support). postMessage() may want another exception condition... 'too much data pending exception'... consider calling postMessage in a while(true) loop... at some point the system is going to have to give up queing the data if its not actually making its way out on the wire. Nothing prevents the data from being thrown in a FIFO queue until it actually can be sent, and I don't see a reason why OOM in the event queueing failed should be handled differently from any other OOM. Jeff
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 4:34 PM, Jeff Walden jwalden+wha...@mit.edujwalden%2bwha...@mit.edu wrote: On 26.6.09 13:52, Michael Nordman wrote: There isn't a way to determine if the WebSocket is successfully sending the postMessage data? For all the caller knows, its just backing up and not going anywhere. You can't really know data has been successfully sent until you get a response that acknowledges it. Getting it on the wire doesn't mean it'll actually be received; I'm not really sure how much value knowing that's happened actually has, other than for doing network diagnostics (which hardly seems a use case to support). Progress bars are routinely implemented without get hi-level application acks from the other side. XMLHttpRequest.upload.onprogress is one such example. diagnostics Cell-phone signal strength bars are a form of diagnostics... existence proof of diagnostics being a significant use case. postMessage() may want another exception condition... 'too much data pending exception'... consider calling postMessage in a while(true) loop... at some point the system is going to have to give up queing the data if its not actually making its way out on the wire. Nothing prevents the data from being thrown in a FIFO queue until it actually can be sent, and I don't see a reason why OOM in the event queueing failed should be handled differently from any other OOM. This info about the status of the WebSocket would be easy to provide to callers of this API. There are easily found valid use cases for this additional status info. What compelling reason is there to not do so? Seems like low-hanging fruit if you ask me. Jeff
Re: [whatwg] Issues with Web Sockets API
On Fri, Jun 26, 2009 at 3:16 PM, Drew Wilson atwil...@google.com wrote: On Fri, Jun 26, 2009 at 2:11 PM, James Robinson jam...@google.com wrote: Forcing applications to build their own send/ack functionality would be pretty tragic considering that WebSockets are built on top of TCP. - James Every time I've written a response/reply protocol on TCP I've needed to put in my own acks - how else do you know your message has been delivered to the remote app layer? This seems especially true given that WebSocket connections may be proxied. -Darin One could argue that WebSockets should do this for you, but I like leaving this up to the app as it gives them more flexibility. -atw
