Re: proposed relationships of Secure EcmaScript, ES3.1, and ES4.
On Feb 20, 2008, at 6:10 PM, Mike Samuel wrote:
JSON ⊂ ADsafe ⊂ Cajita ⊂ Caja ⊂ ES3 ⊂ ES4
People who know Unicode are dangerous ;).
Yes, we need more of you ;-).
There's three problems according to my reading of http://
www.ietf.org/rfc/rfc4627.txt but only the first is directly related
to syntax:
(1) There are JSON programs that are not valid ES programs.
The JSON program [ \u2028 ] where the unicode escape is replaced
with its literal equivalent is valid according to JSON since the
set of characters that can appear in a string unescaped is
unescaped = %x20-21 / %x23-5B / %x5D-10
but ES does not allow codepoint 0x2028 or 0x2029 to appear
unescaped in a string since they are newline characters.
I wonder if JSON should not change on this point. Is there a use-case
for unescaped line/paragraph separators in strings?
(2) There are JSON programs that have the same text as ES programs
but different meaning.
ES262 says that all format control codepoints, such as 0x200C,
should be stripped out of the program in a pre-lex phase. This is
not consistently implemented:
eval('\u200c'.length) == 0 on SpiderMonkey, and 1 on most
other interpreters
Not lately, meaning post-Firefox-2/JS1.7. Fresh js shell, same
results for Firefox 3 any beta:
js eval('\u200c'.length) == 0
false
js eval('\u200c'.length)
1
See https://bugzilla.mozilla.org/show_bug.cgi?id=274152, where
SpiderMonkey yields to IE JScript's flouting of ECMA-262. IE set a
real-world web standard, and for the better according to people in
certain locales.
According to https://bugzilla.mozilla.org/show_bug.cgi?id=368516#c34,
IE does not report illegal character errors correctly, instead
treating misplaced BOMs as identifiers whose references result in
runtime ReferenceErrors (I don't know what it does with other format-
control characters that occur outside of strings and regexps).
See also the follow-on bug to tolerate mislocated BOMs, https://
bugzilla.mozilla.org/show_bug.cgi?id=368516. Ain't the copy/paste
Internet grand?
JSON does not strip these characters out, so they are treated as
significant.
ES4 is specifying as a bug fix to match other browsers that format-
control characters shall not be stripped; it must also, to be a real-
world web standard, specify tolerance for mislocated BOMs. Postel's
Law bites back!
So JSON and ES4 will agree on this one.
(3) There are JSON programs that can be parsed to ES but that
cannot be serialized back to JSON without losing track of where
info was lost.
JSON does not put any limits on numbers, but ES does. ES will
treat 1e1000 as Infinity. Since JSON does not have a value
Infinity, it is unclear how to implement toJSON(fromJSON([1e1000])).
JSON's grammar is nice and simple, it facilitates exhaustive testing
(Rob Sayre used Koushik Sen's jCUTE to generate all-paths tests for a
Java implementation).
BigInts or BigNums could help in the future, but the installed base
will not have them for a while and their literal syntax, without a
pragma, will have a suffix.
This kind of edge case is unlikely to be a problem in practice,
although such overflow conditions recur throughout the security
exploit literature. Could JSON stand to grow support for the IEEE-754
non-finite values?
/be___
Es4-discuss mailing list
Es4-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es4-discuss
Re: ES3.1 Proposal Working Draft
On Feb 20, 2008, at 1:25 PM, Mark S. Miller wrote: What numbers are representable as double but not decimal? Mike Cowlishaw's page at http://www2.hursley.ibm.com/decimal/ is extremely informative, especially http://www2.hursley.ibm.com/decimal/ decifaq.html; see also the link to http://grouper.ieee.org/groups/754/. See http://www2.hursley.ibm.com/decimal/decifaq6.html#bindigits and http://www2.hursley.ibm.com/decimal/decifaq6.html#binapprox for double to decimal conversion answers. As http://wiki.ecmascript.org/ doku.php?id=proposals:decimal summarizes: * A very small amount of precision which may be present in a double- precision binary fp number is lost during a double → decimal promotion, but that: o No precision is lost in an integral → decimal promotion. o Reading a numeric lexeme as a decimal preserves more precision than reading it as a double. Does decimal have NaN, Infinity, -Infinity, Yes -- from the ES4 RI: -1.0m/0.0m -Infinity 1.0m/0.0m Infinity 0.0m/0.0m NaN and -0.0? Yes: 1.0m/-0.0m -Infinity (Btw, I never liked -0.0. And I especially dislike ES3's behavior that 0.0 === -0.0. However, I would argue against making incompatible changes to this.) Guy Steele edited Edition 1 of ECMA-262 and argued for both of these parts of the standard, based on precedent in related programming languages, as well as advice in IEEE-754 itself. He pointed out something important to numerical programmers: you can walk around the four quadrants using signed zeros with atan2: js Math.atan2(0,0) 0 js Math.atan2(-0,0) 0 js Math.atan2(-0,-0) -3.141592653589793 js Math.atan2(0,-0) 3.141592653589793 /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES3.1 Proposal Working Draft - decimal support
OK, thanks (also for the archive pointer). Mike - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Mike Cowlishaw, IBM Fellow IBM UK (MP8), PO Box 31, Birmingham Road, Warwick, CV34 5JL mailto:[EMAIL PROTECTED] -- http://www2.hursley.ibm.com/mfcsumm.html Brendan Eich [EMAIL PROTECTED] 21/02/2008 09:54 To Mike Cowlishaw/UK/[EMAIL PROTECTED] cc Adam Peller [EMAIL PROTECTED], es4-discuss@mozilla.org es4-discuss es4-discuss@mozilla.org Subject Re: ES3.1 Proposal Working Draft - decimal support On Feb 21, 2008, at 1:25 AM, Mike Cowlishaw wrote: Separately from the decimal discussion, I am a bit confused at how ES3.x or ES4 is migrated to from ES3. If the syntax cannot change then that implies that the semantics change without any indication in the syntax. ES4 has new syntax; the syntax-cannot-change has been an on-then-off- then-on-again principal for ES3.1. In particular ES4 has the m suffix for decimal literals, and it has had some kind of 'use decimal' pragma. That would mean that existing scripts would work differently than before. Surely that pretty much requires some meta-notation (such as opt-in versioning)? Indeed. /be Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES3.1 Proposal Working Draft - decimal support
On Feb 21, 2008, at 1:25 AM, Mike Cowlishaw wrote: Separately from the decimal discussion, I am a bit confused at how ES3.x or ES4 is migrated to from ES3. If the syntax cannot change then that implies that the semantics change without any indication in the syntax. ES4 has new syntax; the syntax-cannot-change has been an on-then-off- then-on-again principal for ES3.1. In particular ES4 has the m suffix for decimal literals, and it has had some kind of 'use decimal' pragma. That would mean that existing scripts would work differently than before. Surely that pretty much requires some meta-notation (such as opt-in versioning)? Indeed. /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
ES3.1 Proposal Working Draft
Maciej wrote on Wed Feb 20 14:28:33 PST 2008: Besides compatibility issues, this would be a significant performance regression for math-heavy code. I would consider this a showstopper to implementing such a change. I'm inclined to agree that it is (unfortunately) probably not a good idea to simply replace the default binary arithmetic with decimal128 -- even though this would give better precision for math applications, as well as all the other benefits of decimal arithmetic. But I don't buy the performance argument -- decimal math packages are respectably fast nowadays. See, for example, the measurements at http://www2.hursley.ibm.com/decimal/dnperf.html -- a decDouble add is a couple of hundred cycles in software. That's roughly the same speed on current processors as the hardware binary floating-point available when ECMAScript was first written. In today's (unpipelined) decimal FP hardware it is much faster than those software measurements, of course, and there's no reason why future implementations should not be within 10%-15% of binary FP. I also agree with Mark's comment that arbitrary-precision integers and arbitrary-precision rationals seem like more generally useful types than decimal floating point, if any numeric types are to be added, but that seems like an issue more for ES4 than 3.1. I really do not understand that comment. Almost every numerate human being on the planet uses decimal arithmetic every day; very few need or use arbitrary-precision integers or rationals of more than a few (decimal) digits. And almost every commercial website and server deals with currency, prices, and measurements. It's true that many websites use encryption -- research for which uses BigNums extensively -- but websites don't need or use a general-purpose integer package for that. Mike - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Mike Cowlishaw, IBM Fellow IBM UK (MP8), PO Box 31, Birmingham Road, Warwick, CV34 5JL mailto:[EMAIL PROTECTED] -- http://www2.hursley.ibm.com/mfcsumm.html Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Is there a published specification that all these implementors will be
using?
Thanks,
Geoff
On Feb 20, 2008, at 3:38 PM, Brendan Eich wrote:
As Jeff has laid out, with helpful comments from Michael O'Brien,
Lars, and Graydon, we are entering a phase of ES4 work where
practical implementations will adopt and implement proposed parts of
the new language. We need to do this to shake out proposals and gain
critical feedback from implementors. We hope to get usability results
from programmers too.
I agree with Michael's point about the RI being both alpha and omega
among implementations, so RI work will continue. But practical
implementations, besides enabling usability testing with real
programmers, will help weed out potential problems to do with
performance and security that the RI blissfully ignores.
As Graydon and Michael point out, the waterfall diagram (even if you
put the RI earlier in the waterfall) does not reflect the wheels
within wheels (waterwheels?) that must cycle at several levels of
design, implementation, and even usability testing, in order to reach
the ES4 spec we aspire to write. So take that waterfall diagram more
as a management crutch ;-).
Finally, we absolutely aspire to build our existing testsuites up to
cover as much of the new language as we can. Test-driven development
is the only way to go (I speak from painful experience back in the
Netscape days :-/).
The good news is that I believe we will have many ES4 implementations
coming up in the next few months, working in parallel to improve the
spec and RI. I know of at least these already under way:
* ES4 RI (SML + ES4 self-hosted)
* MbedThis (C + Java)
* Rhino (Java)
* SpiderMonkey (C)
* Tamarin+ESC (C++ + ES4 self-hosted)
If you are implementing any part of ES4 and want to join forces,
please reply.
We aim to track progress using the infrastructure created by John
Resig:
http://ejohn.org/blog/state-of-ecmascript-4-dec-07/
http://spreadsheets.google.com/pub?key=pFIHldY_CkszsFxMkQOReAQgid=2
I believe that the shared spreadsheet URL given above is correct, but
John can provide the latest information as well as grant write
access. My hope is that implementors can edit the spreadsheet to
record progress, providing verifiable builds and even open source for
their implementations, as they go. Again I'll defer to John on this.
We propose to communicate among implementation teams using es4-
[EMAIL PROTECTED], since (a) the list is not terribly high-traffic,
(b) we aim to operate transparently, and (c) we believe most of you
are interested at least as onlookers, if not as implementors. We can
split lists if we run into a problem, but I don't foresee one.
To provide focused face-to-face time working together and building
rapport among the principals, we are thinking of meeting roughly
every month, with this strawman schedule:
March 17-21 - Mountain View, CA
April 14-18 - Newton, MA
May 12-16 - Vancouver, BC
This is very straw, so please don't flame it or it will ignite! But
please do feel free to suggest alternative dates and locations. We
hope to host anyone who has known reputation on this list and who
wants to help, or who is already implementing. Jon Zeppieri has
already helped with the RI, for example, and he's welcome to attend.
More details on the meetings as we firm things up.
Anxiety about commercial pre-release implementations gaining undue
influence over the proposed standard naturally arises, and I wanted
to address this briefly:
Speaking for Mozilla, we do not intend to throw trump cards during
this new phase of proposed-ES4 development based on implementations.
We've implemented extensions in the past, some of which are popular,
others not so popular. We carry what we create until we can drop it,
ideally in favor of a standardized form that's an actual improvement.
This has gone on since Netscape days.
We try not to mess around with marginal extensions, so the few
mistakes we're still carrying, e.g. the magic __proto__
Object.prototype property, in both its read and write modes, reflect
usability gaps in the language. For example, __proto__ is useful in
the absence of ES4's Map to make an object be a hash:
var obj = {__proto__:null, key1:val1, ... keyN:valN};
and that use-case should be better served by something like ES4's Map
proposal. But as noted in the thread that started here, __proto__ is
not proposed in any way for inclusion in ES4.
I bet the other implementators cited above would make a similar no
trump cards pledge. The point of this proposed-ES4 implementation
exercise is not to develop and deploy production code to the Web,
rather to gain crucial feedback from implementors and programmers --
a kind of open source or permanent beta approach to
standardization, where interoperation among ES4 implementations can
be demonstrated with tests and real code before the standard is
finalized.
Any questions? Feel
Re: ES3.1 Proposal Working Draft
On Feb 21, 2008, at 2:46 AM, Mike Cowlishaw wrote: Maciej wrote on Wed Feb 20 14:28:33 PST 2008: Besides compatibility issues, this would be a significant performance regression for math-heavy code. I would consider this a showstopper to implementing such a change. I'm inclined to agree that it is (unfortunately) probably not a good idea to simply replace the default binary arithmetic with decimal128 -- even though this would give better precision for math applications, as well as all the other benefits of decimal arithmetic. But I don't buy the performance argument -- decimal math packages are respectably fast nowadays. See, for example, the measurements at http://www2.hursley.ibm.com/decimal/dnperf.html -- a decDouble add is a couple of hundred cycles in software. That benchmark isn't very useful because it doesn't compare to hardware binary floating point, and also because they are microbenchmarks so it's hard to tell how much impact there would be on a real app. However, hundreds of cycles even for simple operations like add sounds to me like it would be hundreds of times slower than hardware floating point. That's roughly the same speed on current processors as the hardware binary floating-point available when ECMAScript was first written. That's not really a relevant comparison. When ECMAScript was first written, people weren't using it to write complex web apps. Nowadays it would be be unacceptable even for a high-end phone to deliver the ECMAScript performance as slow as consumer desktops from that era. In today's (unpipelined) decimal FP hardware it is much faster than those software measurements, of course, and there's no reason why future implementations should not be within 10%-15% of binary FP. I do all my browsing on a MacBook Pro and an iPhone. As far as I know, neither of these has any kind of decimal FP hardware, nor do I expect their successors to support it any time soon (though I don't have inside knowledge on this). These systems are towards the high end of what is available to consumers. I also agree with Mark's comment that arbitrary-precision integers and arbitrary-precision rationals seem like more generally useful types than decimal floating point, if any numeric types are to be added, but that seems like an issue more for ES4 than 3.1. I really do not understand that comment. Almost every numerate human being on the planet uses decimal arithmetic every day; very few need or use arbitrary-precision integers or rationals of more than a few (decimal) digits. And almost every commercial website and server deals with currency, prices, and measurements. I don't think currency calculations are the only interesting kind of math. So if we need to add a software-implemented more accurate math type, why not go all the way? At least that is my first impression. This is not directly related to my main point, which is about performance and which I think still stands. Regards, Maciej ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 8:14 AM, Geoffrey Garen wrote: Is there a published specification that all these implementors will be using? To expand a bit on Geoff's comment: I'd like Apple and the WebKit project to get involved with ES4 implementation. But right now, as far as I can tell, there isn't a written record for any of ES4's features that I could point an engineer to and say implement this. The proposals on the wiki are way out of date, it's not easy to find what trac tickets modified them, and there seem to be commonly understood planned changes that aren't even reflected in trac. Before attempting interoperable implementations of particular features, I think we need at minimum a form of the proposal for that feature that is complete and up to date. It doesn't have to be formal specification quality, but there has to be something accurate. Now, it may be that by telling someone to reverse engineer another implementation, or ask the ES4 crowd about every detail of how a feature should be implemented, someone could succeed in implementing. But it seems to me that this undermines the unstated assumption of interoperable *independent* implementations. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. I would raise HTML5 as a particularly laudable example because it achieves this even though much implementation work is happening in parallel with writing the spec. I think we should strive to achieve the same standard for ES4. At feature granularity, someone should first write an up to date accurate document and implementations should be done against that, not against some separate shared understanding of the feature. Regards, Maciej Thanks, Geoff On Feb 20, 2008, at 3:38 PM, Brendan Eich wrote: As Jeff has laid out, with helpful comments from Michael O'Brien, Lars, and Graydon, we are entering a phase of ES4 work where practical implementations will adopt and implement proposed parts of the new language. We need to do this to shake out proposals and gain critical feedback from implementors. We hope to get usability results from programmers too. I agree with Michael's point about the RI being both alpha and omega among implementations, so RI work will continue. But practical implementations, besides enabling usability testing with real programmers, will help weed out potential problems to do with performance and security that the RI blissfully ignores. As Graydon and Michael point out, the waterfall diagram (even if you put the RI earlier in the waterfall) does not reflect the wheels within wheels (waterwheels?) that must cycle at several levels of design, implementation, and even usability testing, in order to reach the ES4 spec we aspire to write. So take that waterfall diagram more as a management crutch ;-). Finally, we absolutely aspire to build our existing testsuites up to cover as much of the new language as we can. Test-driven development is the only way to go (I speak from painful experience back in the Netscape days :-/). The good news is that I believe we will have many ES4 implementations coming up in the next few months, working in parallel to improve the spec and RI. I know of at least these already under way: * ES4 RI (SML + ES4 self-hosted) * MbedThis (C + Java) * Rhino (Java) * SpiderMonkey (C) * Tamarin+ESC (C++ + ES4 self-hosted) If you are implementing any part of ES4 and want to join forces, please reply. We aim to track progress using the infrastructure created by John Resig: http://ejohn.org/blog/state-of-ecmascript-4-dec-07/ http://spreadsheets.google.com/pub?key=pFIHldY_CkszsFxMkQOReAQgid=2 I believe that the shared spreadsheet URL given above is correct, but John can provide the latest information as well as grant write access. My hope is that implementors can edit the spreadsheet to record progress, providing verifiable builds and even open source for their implementations, as they go. Again I'll defer to John on this. We propose to communicate among implementation teams using es4- [EMAIL PROTECTED], since (a) the list is not terribly high-traffic, (b) we aim to operate transparently, and (c) we believe most of you are interested at least as onlookers, if not as implementors. We can split lists if we run into a problem, but I don't foresee one. To provide focused face-to-face time working together and building rapport among the principals, we are thinking of meeting roughly every month, with this strawman schedule: March 17-21 - Mountain View, CA April 14-18 - Newton, MA May 12-16 - Vancouver, BC This is very straw, so please don't flame it or it will ignite! But please do feel free to suggest alternative dates and locations. We hope to host anyone who has known reputation on this list and who wants
Re: AOP Compatibility
Another thought: does ES4 provide enough introspection capability to write proxy objects that wrap an immutable class instance? It seems as though it should be possible to create a single class (with * getter/setter functions) that can wrap any object, emulate its interface and provide AOP advice capabilities. If this is indeed possible, would that prove useful for the situations Kris is concerned about? That is good question. It wouldn't truly solve the problem. When you call dojo.connect to request that your method add advice/a listener, the caller doesn't expect that it is going to be told that it should no longer use the original class instance, but rather a proxy. But this could still mitigate the problem. At least users could proxy class instances such that they are advisable in the same way their ol' dynamic ES3 objects were. I don't know if their is sufficient introspection capability though, I would love to know if that is possible. Thanks, Kris ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: AOP Compatibility
On Feb 21, 2008, at 7:47 AM, Neil Mix wrote: Another thought: does ES4 provide enough introspection capability to write proxy objects that wrap an immutable class instance? It seems as though it should be possible to create a single class (with * getter/setter functions) that can wrap any object, emulate its interface and provide AOP advice capabilities. If this is indeed possible, would that prove useful for the situations Kris is concerned about? Sure, at the price of loss of identity. That may break AOP use-cases. /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Maciej Stachowiak wrote: To expand a bit on Geoff's comment: I'd like Apple and the WebKit project to get involved with ES4 implementation. Great! Though please keep in mind a point in the remainder of your comments: WebKit (and Rhino) are operating from a somewhat newcomer perspective, relative to the committee. The other 5 implementations in question (RI, spidermonkey, mbedthis, futhark, ESC+tamarin) are all written by engineers who are and have been closely following the tickets, proposals and discussion, and modifying the RI to encode their thoughts / experiment with a feature. So the implication that the language designers are a disjoint set from the implementors, or that they haven't been writing their thoughts down, is not historically accurate. If that's becoming more true now, ok, maybe we need to make some adjustments. But understand where we're coming from. Before attempting interoperable implementations of particular features, I think we need at minimum a form of the proposal for that feature that is complete and up to date. It doesn't have to be formal specification quality, but there has to be something accurate. I agree this would be nice, but it'd also be nice to have 9 finished implementations and a finished spec! We don't have these yet. So: is your team *completely* stalled until we have such documents, presumably in english rather than SML? If so, I (or anyone else who understands the issues clearly enough -- they're not tremendously complex) can make a priority of building up foundational implementors documents covering such basic concepts as namespaces, names, multinames, types and fixtures. I think we had hoped the RI and tickets on it to serve this role. Now, it may be that by telling someone to reverse engineer another implementation, or ask the ES4 crowd about every detail of how a feature should be implemented, someone could succeed in implementing. But it seems to me that this undermines the unstated assumption of interoperable *independent* implementations. I do not think it undermines the assumption of independent implementations, but I also don't think there's a perfectly clear line between dependent and independent. Impls inform themselves from somewhere, be it spec or spec-that-is-informed-from-other-impls or other technical reference material. Information flows somehow, and often between impls (even if indirectly). You're not going to be doing WebKit by studying Futhark or Spidermonkey; but I *would* recommend studying the RI (and contributing to it!) I would not worry greatly about the risk of being dependent on it, since it is in a very different language (SML) than WebKit's interpreter and is surely structured quite differently. Study it and understand it, though, as it's as precise as we currently get. The RI was meant to be studied (a.k.a. reverse engineered) and the risk of overspecificity from that is something we all explicitly agreed was better than the risk of deploying underspecified and incompatible impls to the field. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. I would raise HTML5 as a particularly laudable example because it achieves this even though much implementation work is happening in parallel with writing the spec. HTML5 is a laudable example, and I hope we wind up producing something of similar quality. It has also had more energy put into it, more eyes on it, and is a much wider and flatter spec (fewer subtle interlocking issues). Web browsers are also stunningly more complex than programming languages, so the concept of a reference implementation is completely fanciful (though you could do a reference implementation of the parsing rules, say). The ES4 RI is small and digestible. Discounting the builtins and some obvious support code, here is the guts of it according to wc -l: 595 ast.sml 134 multiname.sml 923 lexer.sml 7468 parser.sml 2687 defn.sml 1008 verify.sml 5558 eval.sml 913 type.sml 1254 mach.sml 20540 total This is not a big program, it's really small enough to study and participate in. I'd argue that you'd even be in a good place simply reading ast, multiname, eval, type and mach (8.5 kloc). It's intended to be a place for implementors to encode their thoughts and experiments, refer back to, write canonical algorithms in, etc. Don't be shy about using it that way. It's public source. It was intended to be illustrative (how is this supposed to even work) rather than simply consultative (what is the magic ES4 oracle answer to this). (We're probably going to shift it out of monotone and into mercurial soon, which will make it both easier to fetch and more web-accessible too). Is this an unsatisfactory answer? It's possible -- I hope not true --
Re: ES4 implementation process, teams, and meetings
On Thu, 21 Feb 2008, Brendan Eich wrote: In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. And then Hixie goes and rewrites it. I am calling b.s. here, Maciej. We implemented offline web app support early in Firefox 3, based on such WHAT-WG (ok, not HTML WG at the time) specs. They changed a great deal later. In all fairness, they changed in response to feedback from the Mozilla implementors. To paraphrase you, if the spec didn't change, you would be a passive observer of a standardization process that was all but done. That's not reality, and it arguably is not what you should want -- Mozilla people are and hopefully will remain valued peers in the remaining work on HTML5. If you want to be passive implementors of a finished spec, then wait a few years. (Reply-To set to [EMAIL PROTECTED]) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Michael O'Brien wrote: As a solution: I think we need an intermediate step. Not a spec, but some detailed design notes. Lar's document was a good overview, but drill down on exactly how the mechanisms are meant to work would be very helpful to broaden the base of implementations. Some examples of the things I'd like to see are: * Exactly how the scope rules work for various constructs * Order of initialization * More detail on namespace lookup rules * More detail on unqualified variable lookup * Type matching rules and conversions * More detail on nullability But there is a lot more. We can do this via QA, which is what we have been doing, but it is hard and requires quite a bit of sleuthing. I see. I would have hoped these would be visible more or less directly by inspection: * the scope rules and initialization order of each construct are elaborated production-by-production in eval.sml * the namespace lookup scheme is elaborated in multiname.sml, fixture.sml and defn.sml * the unqualified lookup rule starts in eval.sml (evalLexicalRef - evalRefExpr - resolveOnScopeChain - Multiname.resolve) * the type matching and conversion rules begin with runtime judgments in eval.sml (evalBinaryTypeOp) and then transition to type.sml for their type-term-specific parts (groundMatchesGeneric and findSpecialConversion) * the nullability system is more spread out, covering some part of the type normalizer in type.sml and some part of the property allocation scheme in eval.sml I'm sad to hear that the RI was not sufficiently clear on these points, but I guess this was a risk in the choice of language; SML is terse but also not always familiar. Some of the SML code can be lowered, via a prototype converter I have written, to english. Notational details are erased but the names and structure is preserved. Would this be useful? Another possibility is to expand the internal commentary of the RI, or write some sort of study guide. I'm hesitant to commit to yet another bit of plain english docs, both because everyone who could write them already has lots to do, and because there is nothing guaranteeing that such docs remain current or honest. As we've seen, it's easy for an english doc -- such as a wiki page -- to drift into obsolescence if there's not constant energy spent updating it. -Graydon ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
I'd like Apple and the WebKit project to get involved with ES4 implementation. But right now, as far as I can tell, there isn't a written record for any of ES4's features that I could point an engineer to and say implement this. There's certainly no such spec, or you would be a passive observer of a standardization process that was all but done. That's not reality, and it arguably is not what you should want -- Apple people could be valued peers in the remaining work on ES4. If you want to be passive implementors of a finished spec, then wait till next year. I know that I, at least, don't want to be a passive implementor of a finished spec. I want to be an active contributor to a spec in progress. However, since there is no spec -- at least, not one I can lay my hands on -- I'll have to wait until the people in the know finish their implementations and then write a spec. That's OK with me, I guess, but it seems to bar collaboration in the short term. Once you've finished your implementation and written a spec about it, I hope you won't discount feedback by saying, Well, our implementation is already done, and it's too late for major changes -- why didn't you give us this feedback earlier? Geoff ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 12:48 PM, Geoffrey Garen wrote: I'd like Apple and the WebKit project to get involved with ES4 implementation. But right now, as far as I can tell, there isn't a written record for any of ES4's features that I could point an engineer to and say implement this. There's certainly no such spec, or you would be a passive observer of a standardization process that was all but done. That's not reality, and it arguably is not what you should want -- Apple people could be valued peers in the remaining work on ES4. If you want to be passive implementors of a finished spec, then wait till next year. I know that I, at least, don't want to be a passive implementor of a finished spec. I want to be an active contributor to a spec in progress. Great, glad to hear it. However, since there is no spec -- at least, not one I can lay my hands on Have you looked at the RI? -- I'll have to wait until the people in the know finish their implementations and then write a spec. C'mon, we're explicitly proposing an annealing process where we write specs or design notes, referencing the RI by all means but also illuminating it, and using prose where it's appropriate anyway -- where we won't be putting code in the final spec. We never proposed to implement first, then write a big spec. What's with all these straw men? /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Thu, Feb 21, 2008 at 3:48 PM, Geoffrey Garen [EMAIL PROTECTED] wrote: However, since there is no spec -- at least, not one I can lay my hands on -- I'll have to wait until the people in the know finish their implementations What changes would you like to see in the process? What would make it easier for you to jump in, and follow along once you've started? - Rob ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 10:41 AM, Brendan Eich wrote: On Feb 21, 2008, at 8:30 AM, Maciej Stachowiak wrote: I'd like Apple and the WebKit project to get involved with ES4 implementation. But right now, as far as I can tell, there isn't a written record for any of ES4's features that I could point an engineer to and say implement this. There's certainly no such spec, or you would be a passive observer of a standardization process that was all but done. That's not reality, and it arguably is not what you should want -- Apple people could be valued peers in the remaining work on ES4. If you want to be passive implementors of a finished spec, then wait till next year. We'd like to be active participant. However, it seems like as newcomers/outsiders, we do not have enough information available to participate in early implementation. I am not asking for a finished, formal, final detailed spec. What I am asking is this: for each proposal where you'd like early implementations, before implementation commences please write down enough information about that proposal in some reasonably understandable form to represent the current shared understanding of the insiders/old-timers. That would be enough info for us relative outsiders/newcomers to participate. I don't think it's too much to ask for a rough but up-to-date and accurate first draft. I'm not sure how we are supposed participate otherwise. Maybe it is not expected that we would or should. The proposals on the wiki are way out of date, it's not easy to find what trac tickets modified them, and there seem to be commonly understood planned changes that aren't even reflected in trac. That's a failure to file trac tickets -- could you please list these changes that aren't in the trac? There's no other bug system to track these planned changes, so they had better show up at http://bugs.ecmascript.org/ soon or they won't happen. I have no idea what changes aren't in trac. In the past I've asked questions on #jslang or elsewhere about particular proposals (such as the parametric types proposal) and been told that many things about it had been changed, and the person telling me wasn't sure if all these changes had trac tickets, or if so, what they were. It really seems to me like in many cases there is a shared understanding among many of the insiders that is only recorded inside people's heads. Maybe that's not right, but that is certainly the impression I've gotten every time I have asked questions about where something is documented. Before attempting interoperable implementations of particular features, I think we need at minimum a form of the proposal for that feature that is complete and up to date. It doesn't have to be formal specification quality, but there has to be something accurate. I've worked pretty hard to keep proposals such as iterators and generators up to date; it depends on other proposals which are also not formal spec quality, but stable and meaningful (structural types, type parameters). Cormac has done work recently in formalizing the type system which was important to Graydon's RI work. Great, if some proposals are accurate and up to date enough to drive an initial implementation, then my concern is addressed for those features. But I don't know how to tell which ones those are. Is there a list of which proposals are up to date? Furthermore, this won't help when it comes time to implement proposals that *aren't* up to date. All I'm asking is that they be brought up to date first. So I think you are generalizing unfairly here. It's true that decimal is out of date in the wiki, and there are open trac issues. This is true of other proposals. Now, it may be that by telling someone to reverse engineer another implementation, or ask the ES4 crowd about every detail of how a feature should be implemented, someone could succeed in implementing. Nice strawmen, but no one proposed those things. Then what is proposed? If I ask an engineer on my team to implement a feature such as type annotation, how should I ask them to proceed? But it seems to me that this undermines the unstated assumption of interoperable *independent* implementations. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. And then Hixie goes and rewrites it. I am calling b.s. here, Maciej. We implemented offline web app support early in Firefox 3, based on such WHAT-WG (ok, not HTML WG at the time) specs. They changed a great deal later. I'm not asking for a spec that won't substantially change. The whole point of early implementations is to improve the spec, and sometimes that may take significant redesign. Safari has been hit by this as well, and we accept that as a risk we take on as early
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 10:31 AM, Graydon Hoare wrote: Maciej Stachowiak wrote: To expand a bit on Geoff's comment: I'd like Apple and the WebKit project to get involved with ES4 implementation. Great! Though please keep in mind a point in the remainder of your comments: WebKit (and Rhino) are operating from a somewhat newcomer perspective, relative to the committee. The other 5 implementations in question (RI, spidermonkey, mbedthis, futhark, ESC +tamarin) are all written by engineers who are and have been closely following the tickets, proposals and discussion, and modifying the RI to encode their thoughts / experiment with a feature. So the implication that the language designers are a disjoint set from the implementors, or that they haven't been writing their thoughts down, is not historically accurate. If that's becoming more true now, ok, maybe we need to make some adjustments. But understand where we're coming from. I don't think the sets are disjoint, but they are not identical either. Before attempting interoperable implementations of particular features, I think we need at minimum a form of the proposal for that feature that is complete and up to date. It doesn't have to be formal specification quality, but there has to be something accurate. I agree this would be nice, but it'd also be nice to have 9 finished implementations and a finished spec! We don't have these yet. So: is your team *completely* stalled until we have such documents, presumably in english rather than SML? If so, I (or anyone else who understands the issues clearly enough -- they're not tremendously complex) can make a priority of building up foundational implementors documents covering such basic concepts as namespaces, names, multinames, types and fixtures. I think we had hoped the RI and tickets on it to serve this role. I don't think foundational documents are what we need to implement specific features. What we need are rough cut but accurate specs for the features to implement. I don't think SML + trac is a form that anyone here can easily understand. Now, it may be that by telling someone to reverse engineer another implementation, or ask the ES4 crowd about every detail of how a feature should be implemented, someone could succeed in implementing. But it seems to me that this undermines the unstated assumption of interoperable *independent* implementations. I do not think it undermines the assumption of independent implementations, but I also don't think there's a perfectly clear line between dependent and independent. Impls inform themselves from somewhere, be it spec or spec-that-is-informed-from-other-impls or other technical reference material. Information flows somehow, and often between impls (even if indirectly). You're not going to be doing WebKit by studying Futhark or Spidermonkey; but I *would* recommend studying the RI (and contributing to it!) I would not worry greatly about the risk of being dependent on it, since it is in a very different language (SML) than WebKit's interpreter and is surely structured quite differently. Study it and understand it, though, as it's as precise as we currently get. The RI was meant to be studied (a.k.a. reverse engineered) and the risk of overspecificity from that is something we all explicitly agreed was better than the risk of deploying underspecified and incompatible impls to the field. Well, neither I nor anyone on my team know SML. Nor do we know the internals of the reference implementation, what aspects of it are normative, which are implementation details, and which are considered bugs and are intended to change. Nor would I know where in the RI to look to understand how to implement particular features. For example, let binding was raised as an example of a possible early implementation feature. I don't know where in the ~40 klocs of SML in the repository I should look. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. I would raise HTML5 as a particularly laudable example because it achieves this even though much implementation work is happening in parallel with writing the spec. HTML5 is a laudable example, and I hope we wind up producing something of similar quality. It has also had more energy put into it, more eyes on it, and is a much wider and flatter spec (fewer subtle interlocking issues). Web browsers are also stunningly more complex than programming languages, so the concept of a reference implementation is completely fanciful (though you could do a reference implementation of the parsing rules, say). The ES4 RI is small and digestible. Discounting the builtins and some obvious
Re: ES4 implementation process, teams, and meetings
Maciej Stachowiak wrote: I don't think the sets are disjoint, but they are not identical either. Agreed. I am trying to arrive at an understanding of which camp Apple aspires to (designer, implementor or both) and in particular how you wish to enact that role. Any Rhino hackers (or other implementors) may also wish to chime in. It sounds to me like not having anything to do with the RI is characteristic of how you wish to participate. Is this correct? I don't think foundational documents are what we need to implement specific features. What we need are rough cut but accurate specs for the features to implement. I don't think SML + trac is a form that anyone here can easily understand. Ok. Some of the features are deep and some are shallow. Many of the shallow ones are not in much flux, do not require much insight, and are well-described by the proposals on the wiki. You could readily show one to an engineer and have them implement, say, destructuring assignment, triple quotes, date and time extensions, regexp extensions, a hashcode function, getters and setters, block expressions (let expressions), enumerability control, String.prototype.trim, expression closures. Others, as you've noted, require some clean-up / consolidation to be brought into line with tickets. Decimal and numerics turned out to be an ongoing source of trouble, as did tail calls. We're still discussing these. We can probably do this in parallel with your team implementing the ones that are more stable. The thing we don't have feature sketches for are deep, systemic issues that affect the whole language: types, fixtures and namespaces. Also the rules governing packages, classes and interfaces. These came in implicitly from AS3, and the type system in particular has been modified extensively: Cormac's paper and the code in type.sml is probably the best we have to describe the intended type system. This is what I meant by foundational documents. Do you still feel that they are not needed? Well, neither I nor anyone on my team know SML. Nor do we know the internals of the reference implementation, what aspects of it are normative, which are implementation details, and which are considered bugs and are intended to change. Nor would I know where in the RI to look to understand how to implement particular features. For example, let binding was raised as an example of a possible early implementation feature. I don't know where in the ~40 klocs of SML in the repository I should look. I am trying to give you a guide, but am unsure if you want this form of guidance. SML is a small functional language, and we use mostly applicative style. Nothing crazy. Here is a general rule for researching an aspect of the language: start by identifying the AST node for a feature; if you can't find it by guesswork and reading ast.sml, ask someone on IRC or here. Then depending on what you want to do for that feature, read the parser.sml function that produces that node, the defn.sml function that elaborates it, the type.sml rule that checks it, or the eval.sml rule that evaluates it. It would also be good to read some of mach.sml to familiarize yourself with the runtime machine model eval.sml manipulates, if you're investigating a runtime entity. The interesting types are VAL, OBJ, VAL_TAG, MAGIC, SCOPE, REGS, PROP_STATE, TEMP_STATE, TEMPS, PROP, PROP_BINDINGS. If learning SML is really a prerequisite to being one of the early implementors, then I am not sure anyone working on WebKit/JavaScriptCore is qualified, which is a pity. If learning SML is a prerequisite to understanding the finished spec, then that will be a pretty high barrier to entry for even non-early implementors. It certainly will not be a requirement for understanding the finished spec, though depending on whether any pieces of the code make it in as informative annexes, it may help. It *is* currently a requirement for understanding the RI, which the committee agreed to work together on last year rather than drafting formal pseudo-assembly / pseudo-english language we could neither execute nor even automatically check for typos. So far some members of the committee have taken to working on it, while some have not. Are you interested in working on it, as part of Apple's active participation in the process? Or learning to read it? Or is it a completely opaque non-sequitur from Apple's perspective? The idea here -- and feel free to say it's a bad idea or you disagree! -- is that a very high level programming language carries a *different* mixture of risks than very low level natural language, and possibly a more desirable mixture. In particular, as you say, it risks overspecifying and using unfamiliar technical notation; but the risks of natural language (being logically contradictory and underspecifying) are significant too! I tried reading over some of the SML files you mentioned and without context
Re: ES4 implementation process, teams, and meetings
Maciej Stachowiak wrote: What I am asking is this: for each proposal where you'd like early implementations, before implementation commences please write down enough information about that proposal in some reasonably understandable form to represent the current shared understanding of the insiders/old-timers. Ok. Understand though that you're asking for a bit of a change to the game plan. That's a reasonable demand but the early-early imlpementations (RI, futhark, spidermonkey, esc) were all done by people who *did* have that shared understanding (through earlier participation and discussion) and *did* find the SML we were all working on reasonably understandable. So presenting yourself as a participant with neither of those supports in place, you're sort of walking into a room, kicking the legs out from under a table and asking why it's suddenly on the floor. We need to revise the strategy a bit to help for your case, if we can't rely on those. Please try to be patient if this takes some time; none of us had put write pre-impl docs in our current work-assignment schedules. We'll need to make room to do it. I can probably dedicate a month of nearly-full-time energy to this starting in the first week of march. Soon enough? (Also note that about half of the RI is written in ES4, not SML, and that is surely understandable to your team with minimal effort) I have no idea what changes aren't in trac. In the past I've asked questions on #jslang or elsewhere about particular proposals (such as the parametric types proposal) and been told that many things about it had been changed, and the person telling me wasn't sure if all these changes had trac tickets, or if so, what they were. Unfortunately parametric types are one of the cases where you are stabbing at both a terribly in-flux and terribly deep part of the language; they only make sense in the context of types in general, and the *entire* type system has been in flux for quite some time. I believe it is stable now, and I believe the way to understand it is to read Cormac's paper first and type.sml second, ignoring much of the residue in the wiki. I'm sorry about this. A consolidated write-up would be good, I agree. Part of what I was trying to get at by describing dependencies was that you don't really need to understand the entire type system to get a little miniature version of it hobbling along. The initial task to ask one of your engineers to do is give each property and each value a pointer to a runtime-defined 'type' (TBD) and run a function that checks those 'match' (TBD) when someone performs an assignment. Then start enhancing the meaning of type and match as you learn more about them: mbedthis (and the RI) started with simple nominal types (classes in a hierarchy) and worked out from there. It really seems to me like in many cases there is a shared understanding among many of the insiders that is only recorded inside people's heads. Maybe that's not right, but that is certainly the impression I've gotten every time I have asked questions about where something is documented. This depends significantly on the feature in question. For types, and discounting both Cormac's paper and the RI as something you can't extract the desired degree of meaning from, this is probably true. There's not a laid-out plan of the full type system in english alone. I don't know if you like reading LaTeX-y academic type judgment rules more than SML :) For simpler features, I think this is an over-generalization. With some simple markup about currentness, some of them are implementable as-is. But hopefully we can patch up the worst cases and/or mark the stable and easy pieces quickly, so you can make progress. Great, if some proposals are accurate and up to date enough to drive an initial implementation, then my concern is addressed for those features. But I don't know how to tell which ones those are. Is there a list of which proposals are up to date? No. You're right that we should sort them or mark their outdated-ness and revise them. This may cover features that stand as proposals. For deeper cross-language issues (say how classes work), it is not a matter of bringing a page up to date as writing one in the first place. Nobody proposed them since they were inherited from AS3. We may get to writing pre-implementation guides to these too, but it too will take time. Furthermore, this won't help when it comes time to implement proposals that *aren't* up to date. All I'm asking is that they be brought up to date first. A reasonable request; one we haven't focused energy on lately. -Graydon ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 4:34 PM, Graydon Hoare wrote: Maciej Stachowiak wrote: I don't think the sets are disjoint, but they are not identical either. Agreed. I am trying to arrive at an understanding of which camp Apple aspires to (designer, implementor or both) and in particular how you wish to enact that role. Apple is not monolithic. Some of us hope to participate more in the design process, but it's pretty likely that people who don't deeply involve themselves in the design process will do a lot of the implementation work. Those people who are implementing need to have something to follow. Any Rhino hackers (or other implementors) may also wish to chime in. It sounds to me like not having anything to do with the RI is characteristic of how you wish to participate. Is this correct? We're unlikely to have much interest in working on implementing the RI. We have a small team and our hands are full implementing our own JS engine (and the rest of the browser engine). And we think that ultimately implementing a production-quality implementation in our engine is more valuable. While we try to do our part to help with development and validation of important web standards, learning a new programming language and coding in it is a pretty high barrier to entry. Is that really the only way to meaningfully participate in the process? As for reading the RI, it seems a lot harder to understand than specs written in prose. As far as I can tell, only people who have coded significant portions understand it. I don't think foundational documents are what we need to implement specific features. What we need are rough cut but accurate specs for the features to implement. I don't think SML + trac is a form that anyone here can easily understand. Ok. Some of the features are deep and some are shallow. Many of the shallow ones are not in much flux, do not require much insight, and are well-described by the proposals on the wiki. You could readily show one to an engineer and have them implement, say, destructuring assignment, triple quotes, date and time extensions, regexp extensions, a hashcode function, getters and setters, block expressions (let expressions), enumerability control, String.prototype.trim, expression closures. Great, can we start recording this list somewhere? Perhaps part of breaking down features and figuring out the dependencies should be recording which features have an up to date and accurate wiki proposal. Others, as you've noted, require some clean-up / consolidation to be brought into line with tickets. Decimal and numerics turned out to be an ongoing source of trouble, as did tail calls. We're still discussing these. We can probably do this in parallel with your team implementing the ones that are more stable. A clear list of proposals that are clearly not ready would be valuable as well. So let's also start recording this list. The thing we don't have feature sketches for are deep, systemic issues that affect the whole language: types, fixtures and namespaces. Also the rules governing packages, classes and interfaces. These came in implicitly from AS3, and the type system in particular has been modified extensively: Cormac's paper and the code in type.sml is probably the best we have to describe the intended type system. This is what I meant by foundational documents. Do you still feel that they are not needed? Documentation will certainly be needed when it comes time to implement said features. Well, neither I nor anyone on my team know SML. Nor do we know the internals of the reference implementation, what aspects of it are normative, which are implementation details, and which are considered bugs and are intended to change. Nor would I know where in the RI to look to understand how to implement particular features. For example, let binding was raised as an example of a possible early implementation feature. I don't know where in the ~40 klocs of SML in the repository I should look. I am trying to give you a guide, but am unsure if you want this form of guidance. SML is a small functional language, and we use mostly applicative style. Nothing crazy. Here is a general rule for researching an aspect of the language: start by identifying the AST node for a feature; if you can't find it by guesswork and reading ast.sml, ask someone on IRC or here. Then depending on what you want to do for that feature, read the parser.sml function that produces that node, the defn.sml function that elaborates it, the type.sml rule that checks it, or the eval.sml rule that evaluates it. It would also be good to read some of mach.sml to familiarize yourself with the runtime machine model eval.sml manipulates, if you're investigating a runtime entity. The interesting types are VAL, OBJ, VAL_TAG, MAGIC, SCOPE, REGS, PROP_STATE,
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 5:49 PM, Maciej Stachowiak wrote: On Feb 21, 2008, at 5:03 PM, Graydon Hoare wrote: So presenting yourself as a participant with neither of those supports in place, you're sort of walking into a room, kicking the legs out from under a table and asking why it's suddenly on the floor. We need to revise the strategy a bit to help for your case, if we can't rely on those. Please try to be patient if this takes some time; none of us had put write pre-impl docs in our current work-assignment schedules. We'll need to make room to do it. I can probably dedicate a month of nearly-full-time energy to this starting in the first week of march. Soon enough? I'm totally willing to wait for documentation that is more clear and up to date. What concerned me was that it wasn't part of the plan at all, and that Brendan at least seemed unreceptive to the request. You keep misrepresenting things. I never said anything about not writing up-to-date specs. I said specs should be co-evolved with implementations to be up to date, and to the extent that they can be based on extracted RI code, they should be developed that way, so as to be mechanically checkable. This is a serious point. It deserves more than what I regard as flippant attitude about ask Graydon combined with overstatement of the value of all-prose specs. If you have to look at four files to understand let as implemented in the RI, so what? ES1-3 require multiple readings of sections 8-15 just to settle simple questions. The overview and evolutionary programming tutorial take more top-down approaches to presenting (not specifying) features. Programming languages with the kind of type system, binding rules, and compatibility constraints that ES4 has absolutely need executable semantic specifications, not merely all-prose specs. That it takes time is natural. Thanks for understanding. Now with that in mind, please re-read Jeff's post and mine. We are talking about working intensively in the next three months on both specs and implementation. Now is the time to step up. Apple was hors de combat for a long time in Ecma TC39. Kicking the legs out from under the table and pointing at the floor is not good citizenship in my book, whatever our (real) failings in keeping proto-specs up to date. Ultimately for ES4 to be a meaningful standard, it has to have a spec that is comprehensible without special insider information. That's true if you mean by comprehensible things only in people's head. I claim it's false if you mean things specified only in prose. My expectation was that at least parts of it would start to approach that point before they reached the implementation phase. I'm surprised that this wasn't the plan already, but I'm glad you are willing to be flexible. I'm not thrilled about people detouring into recapitulating in uncheckable, duplicative, buggy prose what is in the RI. I'd rather we focus on where the final spec will want to use prose anyway, not extracted, lowered RI code. Probably we will have to compromise and do both prose and RI work. How willing to be flexible are you? /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Maciej Stachowiak wrote: We're unlikely to have much interest in working on implementing the RI. Ok. I'm sorry to hear that, but I understand. As for reading the RI, it seems a lot harder to understand than specs written in prose. As far as I can tell, only people who have coded significant portions understand it. Fair enough. Comprehensibility is a good part of the measurable value of a spec, so if it the code prohibits that we are in an undesirable state. I wonder -- I do not mean to offend here -- if this is partly sticker shock at the initial barrier, which is simply that you have to digest SML, and you haven't read it before. It is not a terribly hard language to learn: it consists of value bindings, function expressions, function-application expressions, case expressions with destructuring pattern matching, if/then/else expressions, and a very small algebraic type system (function types, named types, records, disjoint sums, and sugar for lists). I also intended to do -- and have gradually been doing -- a conversion to the simplest possible syntactic forms of SML I could write, unpacking any syntactic short-hands or dense, idiomatic phrases that might have turned up during the more intense implementation stages. This is similar to rewriting english paragraphs for clarity, and is easy work for someone who speaks the language. Parallelizes easily. I believe this will help with the legibility significantly: for example, read evalCondExpr and tell me if it's illegible: evalCondExpr (regs:Mach.REGS) (cond:Ast.EXPR) (thn:Ast.EXPR) (els:Ast.EXPR) : Mach.VAL = let val v = evalExpr regs cond val b = toBoolean v in if b then evalExpr regs thn else evalExpr regs els end One of my goals -- which I have surely not achieved yet -- is for most of the RI to be distilled to this very pedestrian dialect. On the one hand, it's useful to have a reference implementation to validate what is being done, explore ideas, have something to test and compare against, etc. But yes I think it is an incredibly bad idea for the only specification to be a computer program. It's not approachable. I don't think I could quickly grok a program of this complexity even in a programming language I am familiar with. And by its nature it does not very cleanly partition separate concepts. For example, below you pointed me to 6 places in the code for let statements, and I doubt that reading those functions alone will be enough to understand it. So in practice, I don't think there is any way to understand let in detail without asking you or another expert on the RI. These are ... points I nearly agree with, but not quite, and at the risk of being terribly long-winded I'd like to air the discussion a bit in public here, if we can back off from worrying that I'm saying anything about the schedule of auxiliary-doc-generation (which I've hopefully addressed in the other email): First I want to to point out that there is no established right way to publish language specifications. Language specifications range in style and formalisms employed. People frequently need to study spec, and implementations, and formal treatments in eg. proof assistants or reduced semantic models, *and* do impl-to-impl compatibility bakeoffs. And it still sometimes takes many years, many revisions, to nail down what people actually agree on or disagree on, what's in the language or out of it. Sometimes it takes 5 or 10 years to discover a horrible unsoundness in the language (or, gasp, that you accidentally made the type system turing complete!) No one approach is proven to work. Not yet. The AS3 draft spec we were looking at two winters ago had sections containing pseudo-C++ code, as a way of describing relevant data structures. ES3 has pseudo-assembly, that has typos and nonsensical parts in addition to requiring readers to execute goto statements in their head to understand the flow of a rule. R6RS, for a different example, shipped most recently with a PLT Redex operational semantic model to accompany and illuminate it. We considered using PLT Redex too, and in fact rejected it in part out of the belief (perhaps mistaken!) that normal programmers would find a normal language like SML easier to read than one from the more academic setting of operational semantics descriptions. Possibly in the future (as the POPLMark challenge is hoping to establish) a standard metatheory will solidify for semantics such that machine-checked evaluation rules are no less common than machine-checked grammars in EBNF. But we're not there yet, so we picked something that seemed like it might help, and in at least some senses (see next point) it did. Second I want to point out that while much of the value of a spec is in informing/transmitting information from designers to
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 2:24 PM, Maciej Stachowiak wrote: On Feb 21, 2008, at 10:41 AM, Brendan Eich wrote: We'd like to be active participant. However, it seems like as newcomers/outsiders, we do not have enough information available to participate in early implementation. Neither does anyone else. That's the point of the co-evolved implementations and specs. Sorry if this is unclear. Michael O'Brien's messages have been painfully clear on where specs are lacking, and before there's an ES4 standard we need to fix those lacks. I am not asking for a finished, formal, final detailed spec. What I am asking is this: for each proposal where you'd like early implementations, before implementation commences please write down enough information about that proposal in some reasonably understandable form to represent the current shared understanding of the insiders/old-timers. That would be enough info for us relative outsiders/newcomers to participate. I don't think it's too much to ask for a rough but up-to-date and accurate first draft. I'm not sure how we are supposed participate otherwise. Maybe it is not expected that we would or should. No, we're not turning anyone away for not diving into the deep end of the pool Michael already dove into. We need specs, but they need to be informed by ongoing, well-ordered implementation work. Where we have good-enough specs already (Graydon listed many places near surface syntax), we're ready -- indeed JS1.7 included a bunch of these in order to shake out specs and provide implementation and field-testing feedback. The deeper unresolved issues to do with numbers, e.g., need more spec work, I agree. The proposals on the wiki are way out of date, it's not easy to find what trac tickets modified them, and there seem to be commonly understood planned changes that aren't even reflected in trac. That's a failure to file trac tickets -- could you please list these changes that aren't in the trac? There's no other bug system to track these planned changes, so they had better show up at http://bugs.ecmascript.org/ soon or they won't happen. I have no idea what changes aren't in trac. You wrote there seem to be commonly understood planned changes that aren't even reflected in the trac. I'm asking you what exactly you meant. In the past I've asked questions on #jslang or elsewhere about particular proposals (such as the parametric types proposal) and been told that many things about it had been changed, and the person telling me wasn't sure if all these changes had trac tickets, or if so, what they were. A Mozilla IRC channel and an anonymous respondent -- I don't know what to say. Let's just start fresh here, ok? It really seems to me like in many cases there is a shared understanding among many of the insiders that is only recorded inside people's heads. Maybe that's not right, but that is certainly the impression I've gotten every time I have asked questions about where something is documented. The only way to proceed is case by case, but I think we've already identified the RI as an informative artifact, not just in people's heads, but something you are not able to use. That is a pity, but perhaps we can work around it. Great, if some proposals are accurate and up to date enough to drive an initial implementation, then my concern is addressed for those features. But I don't know how to tell which ones those are. Is there a list of which proposals are up to date? No, probably that's the first order of business, and it should not take that much time. Furthermore, this won't help when it comes time to implement proposals that *aren't* up to date. All I'm asking is that they be brought up to date first. The preference has been to work via the trac to produce sub-specs instead of editing the old (sometimes mis-named) proposals: pages on the wiki. Whatever we do, I agree we need to get rid of the out of date docs. This has been a recurrent problem; sorry for it. Then what is proposed? If I ask an engineer on my team to implement a feature such as type annotation, how should I ask them to proceed? A type annotation is not a leaf feature like a new tag in HTML5. It involves foundational issues. The answer at this point involves reading Cormac's paper, and the RI. We can work from here toward a final spec which will entail prose, but it may also use conventional formalisms for type rules, and it will be based on checkable RI code, even if that is not directly excerpted into the spec. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. And then Hixie goes and rewrites it. I am calling b.s. here, Maciej. We implemented offline web app support early in Firefox 3, based on such WHAT-WG
Re: implementation dependencies (was Re: ES4 work)
On Feb 19, 2008, at 6:39 PM, Graydon Hoare wrote: Finally there is a category I left off the above elaboration, mostly because it is under-developed in the RI: control mechanisms. There are dependencies between tail calls, generators and stack inspection, and I can't say I fully understand the dependencies nor the impact they have on the rest of the implementation. This reminds me of something that Maciej already questioned in his initial response to the language overview: if the control inspector is optional, it's either useless on the web because not supported by all major browsers, or else de-facto non-optional: mandatory due to support in one or two top-market-share browsers. Therefore it should not be included as an optional part out of ES4. We already agreed not to make it mandatory for the sake of small-device implementations. I like the control inspector proposal (http://wiki.ecmascript.org/ doku.php?id=proposals:stack_inspection), but Maciej's argument seems decisive to me. Those of us likely to implement it in the near term could work on developing an ad-hoc spec from the proposal, to be considered for inclusion in a later edition. Generators have an implementation plan in the RI based on delimited continuations in SML, and we seem to have reached consensus recently on proper tail calls. To lighten the load and focus on these two high- value control abstractions proposed for ES4, could we think about deferring control inspector? This is discussion fodder, I'm not yet filing a trac ticket. /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: implementation dependencies (was Re: ES4 work)
Seems to me we may have some emerging agreement on the following items. Please be kind if I'm overstating the consensus, but I believe the following items start us in the right direction without being too onerous. Triage the existing proposals into those that are current and correct and those that aren't. Publish that list. Bring the out-of-date proposal pages up to date Implementers designers pitch in and write up design notes for proposals. This is then both input to the spec and immediate guidance for implementers. Some doc/comments for the RI Create a common place to store resolutions and clarifications on issues. The mailing list isn't great for this gems get lost in the volume. Perhaps the author for each design note could maintain the document and append questions on the end as clarifications in wiki style. I'll start the ball rolling with writing up some notes on Program Units, use unit and unit dependencies. Brendan/Jeff: what format would you like these notes in? Michael ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: implementation dependencies (was Re: ES4 work)
Comments below: This sounds good, but if we've accepted proposals and need detailed specs, why not write specs? This is not just a matter of wiki namespace (proposal: vs. spec:). Proposals have emphasized precedents, use-cases, and anti-use-cases, and considered alternatives. Discussion uncovered further detail, but still not enough for a spec in all cases. The reason is we could remove a few road blocks with some design notes. These won't be as complete as the spec but could from the basis of writing some of the spec prose. Also, the spec can reference the RI (not just SML but, for the standard library, the self-hosted ES4!) in a systematic way. Proposals preceded the RI. The problem here is time. I think doing the spec with the required level of rigour will take much longer than would be ideal to get implementations started. Suggest we evaluate Lars's forthcoming library spec and see what proposals pages it effectively updates. Agree we should mark those pages somehow as superseded by specs, with links to the specs. Not thrilled about leaving out-of-date proposals around, but there's clearly a conflict between the wiki, which is great for content creation, and the trac and spec, which are better for disposition and finalizable specification. I've seen an early cut of the Library spec. Is an update coming? Again I am leery of reinventing the RI in prose, duplicating its meaning with added bugs and no testability. Better to refer to the RI directly as I think you proposed earlier, or even excerpt it as was planned for the spec. The ES4 excerpts should not need lowering; Graydon's script can be used if people find SML hard to read. Fine to refer to the RI, but I think we're talking notes here. Graydon's emails contain a lot of good notes but are certainly not a spec (yet). I think we just need more of those notes. I would agree that if this can't be done simply and easily, we should just push on as forward momentum will eventually deliver the goods. Time is never a friend if we are slow or meandering. Create a common place to store resolutions and clarifications on issues. The mailing list isn't great for this gems get lost in the volume. Perhaps the author for each design note could maintain the document and append questions on the end as clarifications in wiki style. Tracking issues is a job for the trac, although there's always room for on-the-side summaries linking to tickets, if you keep editing to keep up with the primary source of truth in the trac. I think the summaries are where the gold is. That is the piece we are missing. We actually have a lot of information, but it is scattered and hard to put together in a coherent manner. I'll start the ball rolling with writing up some notes on Program Units, use unit and unit dependencies. Brendan/Jeff: what format would you like these notes in? You missed this question above. Cheers Michael ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
