Re: endianness
So, my suspicion was correct, I am using DataView methods like this (https://github.com/Ayms/abstract-tls/blob/master/lib/abstract-tls.js#l280-332 here and on other projects) for historical reasons, because I started from node.js's buffers methods and then switched to ArrayBuffers with node.js's like methods, because in some cases I needed to use both at the same time or to be able to switch easily from one to another. Then the use is not correct in theory, now performances seem to be OK (except node.js), so maybe most of the implementations are instantiating once a DataView for a given ArrayBuffer and keeping reference to it when you call it again? Regards, Le 03/04/2013 00:52, Kenneth Russell a écrit : Yes: API consistency. An ArrayBuffer is opaque; to work with the data it contains, instantiate a view. On Tue, Apr 2, 2013 at 3:36 PM, Kevin Gadd kevin.g...@gmail.com wrote: Is there a reason why DataView wasn't specified as static methods that take an ArrayBuffer in the first place? That would solve the problem of figuring out when/how often to create DataView instances, and eliminate the garbage created by using DataViews. On Tue, Apr 2, 2013 at 3:23 PM, Kenneth Russell k...@google.com wrote: On Tue, Apr 2, 2013 at 3:03 PM, Aymeric Vitte vitteayme...@gmail.com wrote: Le 02/04/2013 04:24, Kenneth Russell a écrit : Agreed. DataView's methods are all simple and should be easy to optimize. Because they include a conditional byte swap, they can't run quite as fast as the typed arrays' accessors -- but they shouldn't need to. DataView was designed to support file and network I/O, where throughput is limited by the disk or network connection. The typed array views were designed for in-memory assembly of data to be submitted to the graphics card, sound card, etc., and must run as fast as possible. When you are streaming things, what's the correct use of DataViews? ie : you are supposed to create each time you want to read some bytes a DataView (which can be optimized or whatever, but still with some costs)? Maybe it's outside of the scope of this discussion, I have already provided examples, I still suspect that I am using it wrongly or that ArrayBuffers are more adapted to webgl (ie static buffer manipulation) than network streaming (ie dynamic buffer manipulation). Probably I am wrong but really would like to know then what's the correct use. If I understand your question, then the correct use of DataView is: upon receiving an ArrayBuffer, create a DataView referring to it. When iterating down the contents of the ArrayBuffer, continue to use the same DataView instance, just incrementing the offset. In abstract-tls/lib/abstract-tls.js there are some operations which create a new DataView just to read or write a single element; this isn't the correct usage. http://www.html5rocks.com/en/tutorials/webgl/typed_arrays/ may be a useful reference. If you're handling streaming data then presumably you're receiving multiple ArrayBuffers, one after the other. You should create one DataView per buffer. The only challenge is properly handling the boundary from one buffer to the next, if the boundary is within an element like a uint16 or uint32. -Ken -- -kg -- jCore Email : avi...@jcore.fr iAnonym : http://www.ianonym.com node-Tor : https://www.github.com/Ayms/node-Tor GitHub : https://www.github.com/Ayms Web :www.jcore.fr Webble : www.webble.it Extract Widget Mobile : www.extractwidget.com BlimpMe! : www.blimpme.com ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness (was: Observability of NaN distinctions — is this a concern?)
On 28 March 2013 21:42, Mark S. Miller erig...@google.com wrote: prohibitively depends on your tolerance. Modern machines can usually do register-to-register byte order reversal rather speedily. Which big endian machines do you have in mind? For WebGL, which expects native endianness on its end, you'll have to convert whole arrays, not just single values. AFAICS, that would be costly enough to basically make WebGL useless on these platforms. /Andreas ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness (was: Observability of NaN distinctions — is this a concern?)
On 28 March 2013 23:01, David Herman dher...@mozilla.com wrote: On Mar 27, 2013, at 6:51 AM, Andreas Rossberg rossb...@google.com wrote: There actually are (third-party) projects with ports of V8 and/or Chromium to big endian architectures. It would be helpful to have more information about what these platforms and projects are. Yeah, unfortunately I can't say without being killed by those third parties. WebGL code should not break or become prohibitively expensive on them all of a sudden. But WebGL code doesn't break either way. It's if we *don't* mandate little-endian that code breaks. No. WebGL expects native endianness today, and existing JS typed array code produces native endianness today. This is true on both little and big endian platforms. So existing code will typically run on both classes of platforms if it does not do something dirty. However, if the new spec suddenly mandates some platforms to switch endianness on one end, then existing code will no longer run as is on those platforms. See also Vlad's reply. /Andreas ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness
On 4/1/2013 10:24 PM, Kenneth Russell wrote: On Sun, Mar 31, 2013 at 1:42 PM, Kevin Gadd kevin.g...@gmail.com wrote: One could also argue that people using typed arrays to alias and munge individual values should be using DataView instead. If it performs poorly, that can hopefully be addressed in the JS runtimes (the way it's specified doesn't seem to prevent it from being efficient). Agreed. DataView's methods are all simple and should be easy to optimize. Because they include a conditional byte swap, they can't run quite as fast as the typed arrays' accessors -- but they shouldn't need to. Side note: in theory they could run extremely fast, as long as the endianness parameter was a constant. Since you generally know the format of the data coming over the network, it should be except in rare cases. The implementation knows the native endianness and can just choose the right load/store variant when generating JIT code for the call. However, they can also read from unaligned offsets, and so that will slow things down a bit. Either way -- they could be much faster than they are now. - Vlad ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness
Le 02/04/2013 04:24, Kenneth Russell a écrit : Agreed. DataView's methods are all simple and should be easy to optimize. Because they include a conditional byte swap, they can't run quite as fast as the typed arrays' accessors -- but they shouldn't need to. DataView was designed to support file and network I/O, where throughput is limited by the disk or network connection. The typed array views were designed for in-memory assembly of data to be submitted to the graphics card, sound card, etc., and must run as fast as possible. When you are streaming things, what's the correct use of DataViews? ie : you are supposed to create each time you want to read some bytes a DataView (which can be optimized or whatever, but still with some costs)? Maybe it's outside of the scope of this discussion, I have already provided examples, I still suspect that I am using it wrongly or that ArrayBuffers are more adapted to webgl (ie static buffer manipulation) than network streaming (ie dynamic buffer manipulation). Probably I am wrong but really would like to know then what's the correct use. Regards, -- jCore Email : avi...@jcore.fr iAnonym : http://www.ianonym.com node-Tor : https://www.github.com/Ayms/node-Tor GitHub : https://www.github.com/Ayms Web :www.jcore.fr Webble : www.webble.it Extract Widget Mobile : www.extractwidget.com BlimpMe! : www.blimpme.com ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness
On Tue, Apr 2, 2013 at 3:03 PM, Aymeric Vitte vitteayme...@gmail.com wrote: Le 02/04/2013 04:24, Kenneth Russell a écrit : Agreed. DataView's methods are all simple and should be easy to optimize. Because they include a conditional byte swap, they can't run quite as fast as the typed arrays' accessors -- but they shouldn't need to. DataView was designed to support file and network I/O, where throughput is limited by the disk or network connection. The typed array views were designed for in-memory assembly of data to be submitted to the graphics card, sound card, etc., and must run as fast as possible. When you are streaming things, what's the correct use of DataViews? ie : you are supposed to create each time you want to read some bytes a DataView (which can be optimized or whatever, but still with some costs)? Maybe it's outside of the scope of this discussion, I have already provided examples, I still suspect that I am using it wrongly or that ArrayBuffers are more adapted to webgl (ie static buffer manipulation) than network streaming (ie dynamic buffer manipulation). Probably I am wrong but really would like to know then what's the correct use. If I understand your question, then the correct use of DataView is: upon receiving an ArrayBuffer, create a DataView referring to it. When iterating down the contents of the ArrayBuffer, continue to use the same DataView instance, just incrementing the offset. In abstract-tls/lib/abstract-tls.js there are some operations which create a new DataView just to read or write a single element; this isn't the correct usage. http://www.html5rocks.com/en/tutorials/webgl/typed_arrays/ may be a useful reference. If you're handling streaming data then presumably you're receiving multiple ArrayBuffers, one after the other. You should create one DataView per buffer. The only challenge is properly handling the boundary from one buffer to the next, if the boundary is within an element like a uint16 or uint32. -Ken ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness
Is there a reason why DataView wasn't specified as static methods that take an ArrayBuffer in the first place? That would solve the problem of figuring out when/how often to create DataView instances, and eliminate the garbage created by using DataViews. On Tue, Apr 2, 2013 at 3:23 PM, Kenneth Russell k...@google.com wrote: On Tue, Apr 2, 2013 at 3:03 PM, Aymeric Vitte vitteayme...@gmail.com wrote: Le 02/04/2013 04:24, Kenneth Russell a écrit : Agreed. DataView's methods are all simple and should be easy to optimize. Because they include a conditional byte swap, they can't run quite as fast as the typed arrays' accessors -- but they shouldn't need to. DataView was designed to support file and network I/O, where throughput is limited by the disk or network connection. The typed array views were designed for in-memory assembly of data to be submitted to the graphics card, sound card, etc., and must run as fast as possible. When you are streaming things, what's the correct use of DataViews? ie : you are supposed to create each time you want to read some bytes a DataView (which can be optimized or whatever, but still with some costs)? Maybe it's outside of the scope of this discussion, I have already provided examples, I still suspect that I am using it wrongly or that ArrayBuffers are more adapted to webgl (ie static buffer manipulation) than network streaming (ie dynamic buffer manipulation). Probably I am wrong but really would like to know then what's the correct use. If I understand your question, then the correct use of DataView is: upon receiving an ArrayBuffer, create a DataView referring to it. When iterating down the contents of the ArrayBuffer, continue to use the same DataView instance, just incrementing the offset. In abstract-tls/lib/abstract-tls.js there are some operations which create a new DataView just to read or write a single element; this isn't the correct usage. http://www.html5rocks.com/en/tutorials/webgl/typed_arrays/ may be a useful reference. If you're handling streaming data then presumably you're receiving multiple ArrayBuffers, one after the other. You should create one DataView per buffer. The only challenge is properly handling the boundary from one buffer to the next, if the boundary is within an element like a uint16 or uint32. -Ken -- -kg ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness
Yes: API consistency. An ArrayBuffer is opaque; to work with the data it contains, instantiate a view. On Tue, Apr 2, 2013 at 3:36 PM, Kevin Gadd kevin.g...@gmail.com wrote: Is there a reason why DataView wasn't specified as static methods that take an ArrayBuffer in the first place? That would solve the problem of figuring out when/how often to create DataView instances, and eliminate the garbage created by using DataViews. On Tue, Apr 2, 2013 at 3:23 PM, Kenneth Russell k...@google.com wrote: On Tue, Apr 2, 2013 at 3:03 PM, Aymeric Vitte vitteayme...@gmail.com wrote: Le 02/04/2013 04:24, Kenneth Russell a écrit : Agreed. DataView's methods are all simple and should be easy to optimize. Because they include a conditional byte swap, they can't run quite as fast as the typed arrays' accessors -- but they shouldn't need to. DataView was designed to support file and network I/O, where throughput is limited by the disk or network connection. The typed array views were designed for in-memory assembly of data to be submitted to the graphics card, sound card, etc., and must run as fast as possible. When you are streaming things, what's the correct use of DataViews? ie : you are supposed to create each time you want to read some bytes a DataView (which can be optimized or whatever, but still with some costs)? Maybe it's outside of the scope of this discussion, I have already provided examples, I still suspect that I am using it wrongly or that ArrayBuffers are more adapted to webgl (ie static buffer manipulation) than network streaming (ie dynamic buffer manipulation). Probably I am wrong but really would like to know then what's the correct use. If I understand your question, then the correct use of DataView is: upon receiving an ArrayBuffer, create a DataView referring to it. When iterating down the contents of the ArrayBuffer, continue to use the same DataView instance, just incrementing the offset. In abstract-tls/lib/abstract-tls.js there are some operations which create a new DataView just to read or write a single element; this isn't the correct usage. http://www.html5rocks.com/en/tutorials/webgl/typed_arrays/ may be a useful reference. If you're handling streaming data then presumably you're receiving multiple ArrayBuffers, one after the other. You should create one DataView per buffer. The only challenge is properly handling the boundary from one buffer to the next, if the boundary is within an element like a uint16 or uint32. -Ken -- -kg ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness (was: Observability of NaN distinctions — is this a concern?)
On Sun, Mar 31, 2013 at 1:42 PM, Kevin Gadd kevin.g...@gmail.com wrote: One could also argue that people using typed arrays to alias and munge individual values should be using DataView instead. If it performs poorly, that can hopefully be addressed in the JS runtimes (the way it's specified doesn't seem to prevent it from being efficient). Agreed. DataView's methods are all simple and should be easy to optimize. Because they include a conditional byte swap, they can't run quite as fast as the typed arrays' accessors -- but they shouldn't need to. DataView was designed to support file and network I/O, where throughput is limited by the disk or network connection. The typed array views were designed for in-memory assembly of data to be submitted to the graphics card, sound card, etc., and must run as fast as possible. -kg On Sun, Mar 31, 2013 at 1:37 PM, Vladimir Vukicevic vladi...@mozilla.com wrote: (Apologies for breaking threading -- subscribed too late to have original message to reply to.) David Herman wrote: On Mar 27, 2013, at 6:51 AM, Andreas Rossberg rossberg at google.com wrote: There actually are (third-party) projects with ports of V8 and/or Chromium to big endian architectures. It would be helpful to have more information about what these platforms and projects are. The Wii-U is probably the highest profile example of this; PowerPC base, and they're doing a bunch of HTML5 apps stuff like what they announced at GDC. WebGL code should not break or become prohibitively expensive on them all of a sudden. But WebGL code doesn't break either way. It's if we *don't* mandate little-endian that code breaks. As for the expense, it has to be weighed against content breaking. Not to mention the burden it places on developers to write portable code without even having big-endian user agents to test on. (I suppose they could use DI and shim the typed array constructors with simulated big-endian versions. Ugh...) The problem, as I see it at least, is that if little-endian is mandated, then effectively we *have* broken WebGL, or the possibility o ever having performant WebGL on those platforms. The underlying OpenGL API will always be native-endian. While big-endian processors often do have ways of efficiently doing byte-swapped loads and stores, that doesn't apply to passing bulk data down. For example, vertex skinning is the base method of doing skeletal animation. It's often done on the CPU, and it involves transforming a bunch of floating point data. The result is then uploaded to the GPU for rendering. If typed arrays are fixed to be little endian, that means that on big endian platforms one of two things will need to happen: - the application will need to manually byte swap (in JS) by aliasing the Float32Array as a UInt8Array and twiddling bytes. - the WebGL implementation will need to make a copy of every incoming 1 byte element size buffer and do the byte swapping before passing it down to GL -- it can either allocate a second buffer, swap into it, and then throw it away, or it can swap before the GL call and unswap after the GL call in-place. If the typed array views are specified to be little-endian, the latter is what would need to be done in order to allow existing content to run. I agree that this would make WebGL unusably slow on big-endian architectures. -Ken Both of these are essentially murder for performance; so by attempting to prevent code from breaking you're basically guaranteeing that all code will effectively break due to performance -- except that developers have no option to write portable and performant code. The other thing is, I suspect that a large chunk of code using typed arrays today will work just fine on big endian platforms, provided that the arrays are defined to be native-endian. Very few code actually aliases and does munging; the only issues might come up with are loading data from the network, but those are present regardless. - Vlad ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss -- -kg ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness (was: Observability of NaN distinctions — is this a concern?)
(Apologies for breaking threading -- subscribed too late to have original message to reply to.) David Herman wrote: On Mar 27, 2013, at 6:51 AM, Andreas Rossberg rossberg at google.com https://mail.mozilla.org/listinfo/es-discuss wrote: / There actually are (third-party) projects // with ports of V8 and/or Chromium to big endian architectures. / It would be helpful to have more information about what these platforms and projects are. The Wii-U is probably the highest profile example of this; PowerPC base, and they're doing a bunch of HTML5 apps stuff like what they announced at GDC. / WebGL // code should not break or become prohibitively expensive on them all of // a sudden. / But WebGL code doesn't break either way. It's if we *don't* mandate little-endian that code breaks. As for the expense, it has to be weighed against content breaking. Not to mention the burden it places on developers to write portable code without even having big-endian user agents to test on. (I suppose they could use DI and shim the typed array constructors with simulated big-endian versions. Ugh...) The problem, as I see it at least, is that if little-endian is mandated, then effectively we *have* broken WebGL, or the possibility o ever having performant WebGL on those platforms. The underlying OpenGL API will always be native-endian. While big-endian processors often do have ways of efficiently doing byte-swapped loads and stores, that doesn't apply to passing bulk data down. For example, vertex skinning is the base method of doing skeletal animation. It's often done on the CPU, and it involves transforming a bunch of floating point data. The result is then uploaded to the GPU for rendering. If typed arrays are fixed to be little endian, that means that on big endian platforms one of two things will need to happen: - the application will need to manually byte swap (in JS) by aliasing the Float32Array as a UInt8Array and twiddling bytes. - the WebGL implementation will need to make a copy of every incoming 1 byte element size buffer and do the byte swapping before passing it down to GL -- it can either allocate a second buffer, swap into it, and then throw it away, or it can swap before the GL call and unswap after the GL call in-place. Both of these are essentially murder for performance; so by attempting to prevent code from breaking you're basically guaranteeing that all code will effectively break due to performance -- except that developers have no option to write portable and performant code. The other thing is, I suspect that a large chunk of code using typed arrays today will work just fine on big endian platforms, provided that the arrays are defined to be native-endian. Very few code actually aliases and does munging; the only issues might come up with are loading data from the network, but those are present regardless. - Vlad ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness (was: Observability of NaN distinctions — is this a concern?)
One could also argue that people using typed arrays to alias and munge individual values should be using DataView instead. If it performs poorly, that can hopefully be addressed in the JS runtimes (the way it's specified doesn't seem to prevent it from being efficient). -kg On Sun, Mar 31, 2013 at 1:37 PM, Vladimir Vukicevic vladi...@mozilla.comwrote: (Apologies for breaking threading -- subscribed too late to have original message to reply to.) David Herman wrote: On Mar 27, 2013, at 6:51 AM, Andreas Rossberg rossberg at google.com https://mail.mozilla.org/listinfo/es-discuss wrote: * There actually are (third-party) projects** with ports of V8 and/or Chromium to big endian architectures.* It would be helpful to have more information about what these platforms and projects are. The Wii-U is probably the highest profile example of this; PowerPC base, and they're doing a bunch of HTML5 apps stuff like what they announced at GDC. * WebGL** code should not break or become prohibitively expensive on them all of** a sudden.* But WebGL code doesn't break either way. It's if we *don't* mandate little-endian that code breaks. As for the expense, it has to be weighed against content breaking. Not to mention the burden it places on developers to write portable code without even having big-endian user agents to test on. (I suppose they could use DI and shim the typed array constructors with simulated big-endian versions. Ugh...) The problem, as I see it at least, is that if little-endian is mandated, then effectively we *have* broken WebGL, or the possibility o ever having performant WebGL on those platforms. The underlying OpenGL API will always be native-endian. While big-endian processors often do have ways of efficiently doing byte-swapped loads and stores, that doesn't apply to passing bulk data down. For example, vertex skinning is the base method of doing skeletal animation. It's often done on the CPU, and it involves transforming a bunch of floating point data. The result is then uploaded to the GPU for rendering. If typed arrays are fixed to be little endian, that means that on big endian platforms one of two things will need to happen: - the application will need to manually byte swap (in JS) by aliasing the Float32Array as a UInt8Array and twiddling bytes. - the WebGL implementation will need to make a copy of every incoming 1 byte element size buffer and do the byte swapping before passing it down to GL -- it can either allocate a second buffer, swap into it, and then throw it away, or it can swap before the GL call and unswap after the GL call in-place. Both of these are essentially murder for performance; so by attempting to prevent code from breaking you're basically guaranteeing that all code will effectively break due to performance -- except that developers have no option to write portable and performant code. The other thing is, I suspect that a large chunk of code using typed arrays today will work just fine on big endian platforms, provided that the arrays are defined to be native-endian. Very few code actually aliases and does munging; the only issues might come up with are loading data from the network, but those are present regardless. - Vlad ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss -- -kg ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness (was: Observability of NaN distinctions — is this a concern?)
prohibitively depends on your tolerance. Modern machines can usually do register-to-register byte order reversal rather speedily. Which big endian machines do you have in mind? On Wed, Mar 27, 2013 at 6:51 AM, Andreas Rossberg rossb...@google.comwrote: On 27 March 2013 00:35, David Herman dher...@mozilla.com wrote: [breaking out a new thread since this is orthogonal to the NaN issue] While the Khronos spec never specified an endianness, TC39 agreed in May 2012 to make the byte order explicitly little-endian in ES6: https://mail.mozilla.org/pipermail/es-discuss/2012-May/022834.html The de facto reality is that there are essentially no big-endian browsers for developers to test on. Web content is being invited to introduce byte-order dependencies. DataView is usually held up as the counter-argument, as if the *existence* of a safe alternative API means no one will ever misuse the unsafe one. Even if we don't take into account human nature, Murphy's Law, and the fact that the web is the world's largest fuzz tester, a wholly rational developer may often prefer not to use DataView because it's still easier to read out bytes using [ ] notation instead of DataView methods. I myself -- possibly the one person in the world who cares most about this issue! -- accidentally created a buggy app that wouldn't work on a big-endian system, because I had no way of testing it: https://github.com/dherman/float.js/commit/deb5bf2f5696ce29d9a6c1a6bf7c479a3784fd7b In summary: we already agreed on TC39 to close this loophole, it's the right thing to do, and concern about potential performance issues on non-existent browsers of non-existent systems should not trump portability and correctly executing existing web content. I missed that decision, and as much as I understand the reasoning, I think we need to work on it. There actually are (third-party) projects with ports of V8 and/or Chromium to big endian architectures. WebGL code should not break or become prohibitively expensive on them all of a sudden. /Andreas ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss -- Cheers, --MarkM ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness (was: Observability of NaN distinctions — is this a concern?)
On Mar 27, 2013, at 6:51 AM, Andreas Rossberg rossb...@google.com wrote: There actually are (third-party) projects with ports of V8 and/or Chromium to big endian architectures. It would be helpful to have more information about what these platforms and projects are. WebGL code should not break or become prohibitively expensive on them all of a sudden. But WebGL code doesn't break either way. It's if we *don't* mandate little-endian that code breaks. As for the expense, it has to be weighed against content breaking. Not to mention the burden it places on developers to write portable code without even having big-endian user agents to test on. (I suppose they could use DI and shim the typed array constructors with simulated big-endian versions. Ugh...) Dave ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness
I think one day I would have raised this subject too. While doing [1] to [3], I was wondering during some time if I was using typed arrays correctly, because I found strange to have to create each time a DataView of a possible enormous buffer to read 2 bytes, and what the impact on performances. Not talking about node.js that does not care about the fact that their DataView instantiation is ridiculously slow, it appears that the defect is present in FF too up to a certain version but it's corrected in the latest Nightly, so probably the answer is that there is an impact on performances. Maybe it's not uninteresting to look at [3] where I am trying to compare 3 methods (former string buffers, typed arrays without DataViews, and typed arrays with DataViews) Or maybe I am still not using this correctly, all protocols used here are in big endian order, I did not read everything about the subject and why this decision was made but the basic reflex would be to add an endianness option to typed arrays. Regards, [1] https://github.com/Ayms/node-Tor [2] https://github.com/Ayms/iAnonym [3] https://github.com/Ayms/abstract-tls and https://github.com/Ayms/abstract-tls#performances- Le 27/03/2013 01:18, Brendan Eich a écrit : Kenneth Russell wrote: On Tue, Mar 26, 2013 at 4:35 PM, David Hermandher...@mozilla.com wrote: [breaking out a new thread since this is orthogonal to the NaN issue] While the Khronos spec never specified an endianness, TC39 agreed in May 2012 to make the byte order explicitly little-endian in ES6: https://mail.mozilla.org/pipermail/es-discuss/2012-May/022834.html The de facto reality is that there are essentially no big-endian browsers for developers to test on. Web content is being invited to introduce byte-order dependencies. DataView is usually held up as the counter-argument, as if the *existence* of a safe alternative API means no one will ever misuse the unsafe one. Even if we don't take into account human nature, Murphy's Law, and the fact that the web is the world's largest fuzz tester, a wholly rational developer may often prefer not to use DataView because it's still easier to read out bytes using [ ] notation instead of DataView methods. I myself -- possibly the one person in the world who cares most about this issue! -- accidentally created a buggy app that wouldn't work on a big-endian system, because I had no way of testing it: https://github.com/dherman/float.js/commit/deb5bf2f5696ce29d9a6c1a6bf7c479a3784fd7b In summary: we already agreed on TC39 to close this loophole, it's the right thing to do, and concern about potential performance issues on non-existent browsers of non-existent systems should not trump portability and correctly executing existing web content. I am disappointed that this decision was made without input from both of the editors of the typed array spec and disagree with the statement that it is the right thing to do. First, my apologies to Dave for forgetting the May decision. I was looking in the wrong place for a record of that decision! Second, I understand Ken's point of view (old SGI hacker here) but have to question it on the grounds Dave raises: no one is testing, so there are very likely to be latent little-endian dependencies in deployed code today. So why not codify this? Because it might penalize big-endian systems? The IP protocols use big-endian byte order (owing to the pre-Intel big-endian architectures of IMPs and early Internet machines) and so penalize Intel machines, and we take the hit (and it's small on modern super-scalar CPUs). Ken, a couple of questions: 1. Do you know of any big-endian systems with market share that host JS implementations? It would be great to have a list. 2. Independent of the answer to (1), today's web browsers run on little-endian systems and that has very likely created a de-facto standard. If we codify it, we make the small to empty set of big-endian systems pay a price. Why is this the wrong thing for typed arrays but the right thing (in reverse! majority-architectures penalized) for the IP protocol suite? /be -- jCore Email : avi...@jcore.fr iAnonym : http://www.ianonym.com node-Tor : https://www.github.com/Ayms/node-Tor GitHub : https://www.github.com/Ayms Web :www.jcore.fr Webble : www.webble.it Extract Widget Mobile : www.extractwidget.com BlimpMe! : www.blimpme.com ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness (was: Observability of NaN distinctions — is this a concern?)
On Tue, Mar 26, 2013 at 4:35 PM, David Herman dher...@mozilla.com wrote: [breaking out a new thread since this is orthogonal to the NaN issue] While the Khronos spec never specified an endianness, TC39 agreed in May 2012 to make the byte order explicitly little-endian in ES6: https://mail.mozilla.org/pipermail/es-discuss/2012-May/022834.html The de facto reality is that there are essentially no big-endian browsers for developers to test on. Web content is being invited to introduce byte-order dependencies. DataView is usually held up as the counter-argument, as if the *existence* of a safe alternative API means no one will ever misuse the unsafe one. Even if we don't take into account human nature, Murphy's Law, and the fact that the web is the world's largest fuzz tester, a wholly rational developer may often prefer not to use DataView because it's still easier to read out bytes using [ ] notation instead of DataView methods. I myself -- possibly the one person in the world who cares most about this issue! -- accidentally created a buggy app that wouldn't work on a big-endian system, because I had no way of testing it: https://github.com/dherman/float.js/commit/deb5bf2f5696ce29d9a6c1a6bf7c479a3784fd7b In summary: we already agreed on TC39 to close this loophole, it's the right thing to do, and concern about potential performance issues on non-existent browsers of non-existent systems should not trump portability and correctly executing existing web content. I am disappointed that this decision was made without input from both of the editors of the typed array spec and disagree with the statement that it is the right thing to do. -Ken Dave This was not the plan of record on TC39. The plan was to fix the semantics as little-endian. On Mar 25, 2013, at 6:00 PM, Brendan Eich bren...@mozilla.com wrote: Right, thanks for the reminder. It all comes back now, including the how to write correct ending-independent typed array code bit. /be On Mar 25, 2013, at 4:33 PM, Kenneth Russell k...@google.com wrote: On Mon, Mar 25, 2013 at 4:23 PM, Brendan Eich bren...@mozilla.com wrote: Allen Wirfs-Brock wrote: On Mar 25, 2013, at 4:05 PM, Brendan Eich wrote: Allen Wirfs-Brock wrote: BTW, isn't cannonicalization of endian-ness for both integers and floats a bigger interop issue than NaN cannonicalization? I know this was discussed in the past, but it doesn't seem to be covered in the latest Khronos spec. Was there ever a resolution as to whether or not TypedArray [[Set]] operations need to use a cannonical endian-ness? Search for byte order at https://www.khronos.org/registry/typedarray/specs/latest/. I had already search for endian with similar results. It says that the default for DataViews gets/sets that do not specify a byte order is big-endean. It doesn't say anything (that I can find) about such accesses on TypedArray gets/sets. Oh, odd -- I recall that it used to say little-endian. Typed arrays are LE to match dominant architectures, while DataViews are BE to match packed serialization use-cases. Ken, did something get edited out? No. The typed array views (everything except DataView) have used the host machine's endianness from day one by design -- although the typed array spec does not state this explicitly. If desired, text can be added to the specification to this effect. Any change in this behavior will destroy the performance of APIs like WebGL and Web Audio on big-endian architectures. Correctly written code works identically on big-endian and little-endian architectures. See http://www.html5rocks.com/en/tutorials/webgl/typed_arrays/ for a detailed description of the usage of the APIs. DataView, which is designed for input/output, operates on data with a specified endianness. -Ken ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: endianness
Kenneth Russell wrote: On Tue, Mar 26, 2013 at 4:35 PM, David Hermandher...@mozilla.com wrote: [breaking out a new thread since this is orthogonal to the NaN issue] While the Khronos spec never specified an endianness, TC39 agreed in May 2012 to make the byte order explicitly little-endian in ES6: https://mail.mozilla.org/pipermail/es-discuss/2012-May/022834.html The de facto reality is that there are essentially no big-endian browsers for developers to test on. Web content is being invited to introduce byte-order dependencies. DataView is usually held up as the counter-argument, as if the *existence* of a safe alternative API means no one will ever misuse the unsafe one. Even if we don't take into account human nature, Murphy's Law, and the fact that the web is the world's largest fuzz tester, a wholly rational developer may often prefer not to use DataView because it's still easier to read out bytes using [ ] notation instead of DataView methods. I myself -- possibly the one person in the world who cares most about this issue! -- accidentally created a buggy app that wouldn't work on a big-endian system, because I had no way of testing it: https://github.com/dherman/float.js/commit/deb5bf2f5696ce29d9a6c1a6bf7c479a3784fd7b In summary: we already agreed on TC39 to close this loophole, it's the right thing to do, and concern about potential performance issues on non-existent browsers of non-existent systems should not trump portability and correctly executing existing web content. I am disappointed that this decision was made without input from both of the editors of the typed array spec and disagree with the statement that it is the right thing to do. First, my apologies to Dave for forgetting the May decision. I was looking in the wrong place for a record of that decision! Second, I understand Ken's point of view (old SGI hacker here) but have to question it on the grounds Dave raises: no one is testing, so there are very likely to be latent little-endian dependencies in deployed code today. So why not codify this? Because it might penalize big-endian systems? The IP protocols use big-endian byte order (owing to the pre-Intel big-endian architectures of IMPs and early Internet machines) and so penalize Intel machines, and we take the hit (and it's small on modern super-scalar CPUs). Ken, a couple of questions: 1. Do you know of any big-endian systems with market share that host JS implementations? It would be great to have a list. 2. Independent of the answer to (1), today's web browsers run on little-endian systems and that has very likely created a de-facto standard. If we codify it, we make the small to empty set of big-endian systems pay a price. Why is this the wrong thing for typed arrays but the right thing (in reverse! majority-architectures penalized) for the IP protocol suite? /be -Ken Dave This was not the plan of record on TC39. The plan was to fix the semantics as little-endian. On Mar 25, 2013, at 6:00 PM, Brendan Eichbren...@mozilla.com wrote: Right, thanks for the reminder. It all comes back now, including the how to write correct ending-independent typed array code bit. /be On Mar 25, 2013, at 4:33 PM, Kenneth Russellk...@google.com wrote: On Mon, Mar 25, 2013 at 4:23 PM, Brendan Eichbren...@mozilla.com wrote: Allen Wirfs-Brock wrote: On Mar 25, 2013, at 4:05 PM, Brendan Eich wrote: Allen Wirfs-Brock wrote: BTW, isn't cannonicalization of endian-ness for both integers and floats a bigger interop issue than NaN cannonicalization? I know this was discussed in the past, but it doesn't seem to be covered in the latest Khronos spec. Was there ever a resolution as to whether or not TypedArray [[Set]] operations need to use a cannonical endian-ness? Search for byte order at https://www.khronos.org/registry/typedarray/specs/latest/. I had already search for endian with similar results. It says that the default for DataViews gets/sets that do not specify a byte order is big-endean. It doesn't say anything (that I can find) about such accesses on TypedArray gets/sets. Oh, odd -- I recall that it used to say little-endian. Typed arrays are LE to match dominant architectures, while DataViews are BE to match packed serialization use-cases. Ken, did something get edited out? No. The typed array views (everything except DataView) have used the host machine's endianness from day one by design -- although the typed array spec does not state this explicitly. If desired, text can be added to the specification to this effect. Any change in this behavior will destroy the performance of APIs like WebGL and Web Audio on big-endian architectures. Correctly written code works identically on big-endian and little-endian architectures. See http://www.html5rocks.com/en/tutorials/webgl/typed_arrays/ for a detailed description of the usage of the APIs. DataView, which is designed for input/output, operates on data
Re: Typed Arrays and Endianness
On Thu, May 20, 2010 at 4:50 PM, Waldemar Horwat walde...@google.com wrote: Having unspecified endianness in the specification of TypedArrays seems like a show-stopper to me. There are two alternatives here: - If you do any I/O, you want the endianness to be specified. Sending data from one process to another with a different endianness would get you into trouble. - If you don't do any I/O, then the TypedArray is private to ECMAScript. In this case there are programming techniques which will avoid the bulk of the cost of endian switching, so you don't gain much by leaving endianness unspecified in a language like ECMAScript. Let's defer this discussion until after tomorrow's face-to-face meeting. Use cases such as WebGL and audio APIs require the native endianness of the platform to be used, even though data is sent from ECMAScript to another library or process on the same machine. I'd also like to find out what you mean about programming techniques to avoid the cost of endianness switching. -Ken ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Typed Arrays and Endianness
Having unspecified endianness in the specification of TypedArrays seems like a show-stopper to me. There are two alternatives here: - If you do any I/O, you want the endianness to be specified. Sending data from one process to another with a different endianness would get you into trouble. - If you don't do any I/O, then the TypedArray is private to ECMAScript. In this case there are programming techniques which will avoid the bulk of the cost of endian switching, so you don't gain much by leaving endianness unspecified in a language like ECMAScript. Waldemar ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
