RE: Exactly where is a RegularExpressionLiteral allowed?
OK, let's try to wrap up this issues. In addition to adding RegularExpressionLiteral to Literal, do we also agree to delete the third paragraph of section 7 that says: Note that contexts exist in the syntactic grammar where both a division and a RegularExpressionLiteral are permitted by the syntactic grammar; however, since the lexical grammar uses the InputElementDiv goal symbol in such cases, the opening slash is not recognised as starting a regular expression literal in such a context. As a workaround, one may enclose the regular expression literal in parentheses. The second paragraph says: The InputElementDiv symbol is used in those syntactic grammar contexts where a division (/) or division-assignment (/=) operator is permitted. Should we insert the work initial (or leading) immediately in front of division to clarify where such contexts occur? Allen ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Exactly where is a RegularExpressionLiteral allowed?
Waldemar Horwat wrote: David-Sarah Hopwood wrote: I'll repeat my argument here for convenience: A DivisionPunctuator must be preceded by an expression. A RegularExpressionLiteral is itself an expression. (This assumes that the omission of RegularExpressionLiteral from Literal is a bug.) Therefore, for there to exist syntactic contexts in which either a DivisionPunctuator or a RegularExpressionLiteral could occur, it would have to be possible for an expression to immediately follow [*] another expression with no intervening operator. The only case in which that can occur is where a semicolon is automatically inserted between the two expressions. Assume that case: then the second expression cannot begin with [*] a token whose first character is '/', because that would have been interpreted as a DivisionPunctuator, and so no semicolon insertion would have occurred (because semicolon insertion only occurs where there would otherwise have been a syntax error); contradiction. Yes, I verified when we were writing ES3 that this was the only case where the syntactic grammar permitted a / to serve as both a division (or division-assignment) and a regexp literal. The interaction of lexing and semicolon insertion would have been unclear (how can you say that the next token is invalid if you don't know how to lex it?), so we wrote the spec to explicitly resolve those in favor of division. If that is what the note is intended to clarify, I think its current wording is more confusing than helpful. It certainly confused me. Anyway, there is no case in which a regexp needs to be parenthesized to avoid lexical ambiguity. How about replacing the current wording by something that specifically discusses the semicolon insertion issue, with an example: There are two goal symbols for the lexical grammar. The InputElementDiv symbol is used in those syntactic grammar contexts where a leading division (/) or division-assignment (/=) operator is permitted. The InputElementRegExp symbol is used in other syntactic grammar contexts. NOTE There are no syntactic grammar contexts where both a leading division or division-assignment, and a leading RegularExpressionLiteral are permitted. This is not affected by semicolon insertion (section 7.9); in examples such as the following: a = b /hi/g.exec(c).map(d); where the first non-whitespace, non-comment character after a LineTerminator is '/' and the syntactic context allows division or division-assignment, no semicolon is inserted at the LineTerminator. That is, this example is interpreted in the same way as: a = b / hi / g.exec(c).map(d); -- David-Sarah Hopwood ⚥ ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Exactly where is a RegularExpressionLiteral allowed?
On Mar 23, 2009, at 11:36 AM, Allen Wirfs-Brock wrote: Pratap came up with an issue in reviewing the ES3.1 draft that I don’t have a good answer to: Sections 7 and 5.1.2 mention InputElementDiv and InputElementRegExp. 7 There are two goal symbols for the lexical grammar. The InputElementDiv symbol is used in those syntactic grammar contexts where a division (/) or division-assignment (/=) operator is permitted. The InputElementRegExp symbol is used in other syntactic grammar contexts. Where in the grammar (or elsewhere) does it define **when** these two contexts are used? There are no subsequent references to InputElementDiv or InputElementRegExp in either the grammars or the prose of the specification. Nor, is there any reference to RegularExpressionLiteral in the syntactic grammar or supporting prose. So how am I as a language implementer reading this specification supposed to know that: var rx = /a*z/; is a valid expression whose RHS is an RegualrExpressionLiteral? The best explanation I can come up with for this interpretation solely from the specification is that the production to the right of the “=” in VariableDeclaration is an AssignmentExpression that can reduce to ConditionalExpression and since ConditionExpression doesn’t describe any non-parenthesized expressions that begin with “/” or “=/” =/ is not an operator (it was in archaic, pre-ISO, last-days-of-Disco C, but long ago fixed to be /=). I should use InputElementRegExp and accept a RegularExpressionLiteral as a PrimaryExpression. (and there really isn’t anything in the specification that tells me to make that last leap. Actually, I don’t really think that the specification actually says that the above example is valid.) If you make the /= correction, there is no ambiguity. 7.1 last paragraph: Note that contexts exist in the syntactic grammar where both a division and a RegularExpressionLiteral are permitted by the syntactic grammar; however, since the lexical grammar uses the InputElementDiv goal symbol in such cases, the opening slash is not recognised as starting a regular expression literal in such a context. As a workaround, one may enclose the regular expression literal in parentheses. ASI strikes again: a = b /hi/g.exec(c).map(d); The Note takes care of this. This isn’t a new ES3.1 problem. Nothing has changed from the ES3 specification in this regard. Am I missing something obvious? Obviously this can’t be unambiguously specified in the grammar but it sure seems like there should be some sort of comprehensible specification of exactly when RegularExpressionLiteral is allowed. I think that's 7.1. /be___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
RE: Exactly where is a RegularExpressionLiteral allowed?
=/ was my typo, it's not in the specification. But see below: From: Brendan Eich [mailto:bren...@mozilla.com] Sent: Monday, March 23, 2009 11:57 AM ... If you make the /= correction, there is no ambiguity. 7.1 last paragraph: Note that contexts exist in the syntactic grammar where both a division and a RegularExpressionLiteral are permitted by the syntactic grammar; however, since the lexical grammar uses the InputElementDiv goal symbol in such cases, the opening slash is not recognised as starting a regular expression literal in such a context. As a workaround, one may enclose the regular expression literal in parentheses. It's actually in 7 not 7.1. However the problem is that RegularExpressionLiteral doesn't even occur in the syntactic grammar so strictly speaking there are no such specified contexts. ... This isn't a new ES3.1 problem. Nothing has changed from the ES3 specification in this regard. Am I missing something obvious? Obviously this can't be unambiguously specified in the grammar but it sure seems like there should be some sort of comprehensible specification of exactly when RegularExpressionLiteral is allowed. I think that's 7.1. I don't think so, although perhaps the fix is as easy as adding RegularExpressionLiteral as an alternative RHS for PrimaryExpression. As it stands right now, I still content that the specification does actually say how a RegularExpressionLiteral can be used in an expression. I think this is only discoverable by examining existing implementations. Allen ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Exactly where is a RegularExpressionLiteral allowed?
On Mar 23, 2009, at 12:38 PM, Allen Wirfs-Brock wrote: I don't think so, although perhaps the fix is as easy as adding RegularExpressionLiteral as an alternative RHS for PrimaryExpression. Oh sure -- that is the missing link. Thanks! /be ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Exactly where is a RegularExpressionLiteral allowed?
Brendan Eich wrote: [...] If you make the /= correction, there is no ambiguity. Indeed there is no ambiguity; I think Allen's point is that the spec is currently written in a way that is very unhelpful in allowing one to conclude that. 7.1 last paragraph: Note that contexts exist in the syntactic grammar where both a division and a RegularExpressionLiteral are permitted by the syntactic grammar; I believe that statement is wrong. I gave a detailed argument for why it is wrong at http://www.mail-archive.com/es-discuss@mozilla.org/msg01329.html; the only reply was by Eric Suen, and the argument in his post was incorrect. I'll repeat my argument here for convenience: A DivisionPunctuator must be preceded by an expression. A RegularExpressionLiteral is itself an expression. (This assumes that the omission of RegularExpressionLiteral from Literal is a bug.) Therefore, for there to exist syntactic contexts in which either a DivisionPunctuator or a RegularExpressionLiteral could occur, it would have to be possible for an expression to immediately follow [*] another expression with no intervening operator. The only case in which that can occur is where a semicolon is automatically inserted between the two expressions. Assume that case: then the second expression cannot begin with [*] a token whose first character is '/', because that would have been interpreted as a DivisionPunctuator, and so no semicolon insertion would have occurred (because semicolon insertion only occurs where there would otherwise have been a syntax error); contradiction. [*] Ignoring comments and whitespace. however, since the lexical grammar uses the InputElementDiv goal symbol in such cases, the opening slash is not recognised as starting a regular expression literal in such a context. As a workaround, one may enclose the regular expression literal in parentheses. ASI strikes again: a = b /hi/g.exec(c).map(d); Semicolon insertion is not possible after 'b', because the '/' following it is a valid token in that context, so there is no syntax error that could prompt semicolon insertion. The Note takes care of this. This is not a case where the note applies; it's essentially the same case as given by Eric Suen. My response to him is at http://www.mail-archive.com/es-discuss@mozilla.org/msg01331.html. -- David-Sarah Hopwood ⚥ ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Exactly where is a RegularExpressionLiteral allowed?
Brendan Eich wrote: On Mar 23, 2009, at 12:38 PM, Allen Wirfs-Brock wrote: I don't think so, although perhaps the fix is as easy as adding RegularExpressionLiteral as an alternative RHS for PrimaryExpression. Oh sure -- that is the missing link. Thanks! It should be Literal, not PrimaryExpression. There is no technical difference (since Literal is only used as one of the alternatives for PrimaryExpression), but it's just common sense that a RegularExpressionLiteral is a literal. -- David-Sarah Hopwood ⚥ ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Exactly where is a RegularExpressionLiteral allowed?
David-Sarah Hopwood wrote: I'll repeat my argument here for convenience: A DivisionPunctuator must be preceded by an expression. A RegularExpressionLiteral is itself an expression. (This assumes that the omission of RegularExpressionLiteral from Literal is a bug.) Therefore, for there to exist syntactic contexts in which either a DivisionPunctuator or a RegularExpressionLiteral could occur, it would have to be possible for an expression to immediately follow [*] another expression with no intervening operator. The only case in which that can occur is where a semicolon is automatically inserted between the two expressions. Assume that case: then the second expression cannot begin with [*] a token whose first character is '/', because that would have been interpreted as a DivisionPunctuator, and so no semicolon insertion would have occurred (because semicolon insertion only occurs where there would otherwise have been a syntax error); contradiction. Yes, I verified when we were writing ES3 that this was the only case where the syntactic grammar permitted a / to serve as both a division (or division-assignment) and a regexp literal. The interaction of lexing and semicolon insertion would have been unclear (how can you say that the next token is invalid if you don't know how to lex it?), so we wrote the spec to explicitly resolve those in favor of division. Waldemar ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Exactly where is a RegularExpressionLiteral allowed?
On Mar 23, 2009, at 4:27 PM, David-Sarah Hopwood wrote: Brendan Eich wrote: On Mar 23, 2009, at 12:38 PM, Allen Wirfs-Brock wrote: I don't think so, although perhaps the fix is as easy as adding RegularExpressionLiteral as an alternative RHS for PrimaryExpression. Oh sure -- that is the missing link. Thanks! It should be Literal, not PrimaryExpression. There is no technical difference (since Literal is only used as one of the alternatives for PrimaryExpression), but it's just common sense that a RegularExpressionLiteral is a literal. Agreed, presumably Allen agrees too. It's obvious now that you point it out ;-). Thanks for the links to your exchange with Eric Suen, I should have remembered that. /be ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
RE: Exactly where is a RegularExpressionLiteral allowed?
-Original Message- From: es-discuss-boun...@mozilla.org [mailto:es-discuss- boun...@mozilla.org] On Behalf Of Brendan Eich Sent: Monday, March 23, 2009 6:39 PM It should be Literal, not PrimaryExpression. There is no technical difference (since Literal is only used as one of the alternatives for PrimaryExpression), but it's just common sense that a RegularExpressionLiteral is a literal. Agreed, presumably Allen agrees too. It's obvious now that you point it out ;-). Homefully I'm relatively agreeable...Strictly speaking RegularExpressionLiteral is now semantically more like an ArrayLiteral or an Object literal in that one of them evaluates to a different object each time it is executed. However, I don't think it actually makes much of a difference whether it is a Literal or a PrimaryExpression and since I'm certainly not proposing that we move it from section 7 to section 11 at this point in time it probably makes more sense to say it is a Literal. Allen ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Exactly where is a RegularExpressionLiteral allowed?
On Mar 23, 2009, at 7:04 PM, Allen Wirfs-Brock wrote: -Original Message- From: es-discuss-boun...@mozilla.org [mailto:es-discuss- boun...@mozilla.org] On Behalf Of Brendan Eich Sent: Monday, March 23, 2009 6:39 PM It should be Literal, not PrimaryExpression. There is no technical difference (since Literal is only used as one of the alternatives for PrimaryExpression), but it's just common sense that a RegularExpressionLiteral is a literal. Agreed, presumably Allen agrees too. It's obvious now that you point it out ;-). Homefully I'm relatively agreeable...Strictly speaking RegularExpressionLiteral is now semantically more like an ArrayLiteral or an Object literal Those are still called Array Initialiser and Object Iniitialiser, right? Not to belabor the point, but they aren't literals in the same sense, although Object Initialiser's property names are literals (constants). These are expression forms with evaluated terms inside the bracing. in that one of them evaluates to a different object each time it is executed. That's true, but it doesn't make a literal what it is, in the grammatical sense. However, I don't think it actually makes much of a difference whether it is a Literal or a PrimaryExpression and since I'm certainly not proposing that we move it from section 7 to section 11 at this point in time it probably makes more sense to say it is a Literal. Cool. That does seem like the minimal change, and it seems worthwhile also to keep Literal meaning lexeme. /be ___ Es-discuss mailing list Es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
