I'll make various comments after each part.
On Jun 26, 2009, at 10:18 PM, The Blue Wizard wrote:
>
> I ran into a problem trying to imitate the special-case parsing logic
> that was found in an old recursive descent parser I was translating
> from.
>
> Basically, I am trying to do this:
>
> WRITE "(" write_list ")"
>
> where write_list is a sequence of either expr or STRING, separated by
> commas, and the expr is a "standard" expression made up of the usual
> binary operators (+, -, *, /), unary minus, and parentheses involving
> INTEGERs and STRINGs, where STRINGs have the semantic of the ordinal
> value of the first character and the rest are simply ignored (in
> Python code, it is just ord((STRING[:1]+'\0')[0]) ).
>
> The STRING in write_list should be treated differently from expr in
> that all characters get output, and any subsequent operators appearing
> after STRING is illegal.
>
> If you think about it, you can see how a recursive descent parser code
> would simply see a STRING as the first token, and makes a special case
> for it, then search for a comma or a closing parenthesis before going
> on.
>
> So:
> WRITE('Hello') is legal, and it prints Hello
> WRITE('Test'+9) is illegal
> WRITE(('A'+8),6+4) is legal and it prints 73 10
>
> How do I do that in Yacc (PLY flavor, naturally)?
>
I'm not exactly sure of the best way to do what you propose, but you
might be able to solve it in the lexer using two different string
tokens. Define a token for a 1-character string literal and a
separate token for a multi-character string literal. Then make sure
expressions can only contain the 1-character literal. Another
approach is to simply build more semantics into WRITE. For example,
you could allow the parsing rule for WRITE to accept any expression,
but then you add a check that looks through the expression to make
sure strings are properly handled.
> Now some remarks:
>
> 1) I notice that the document for PLY 3.2 has the blurb "version
> 3.0"...should it get updated to "version 3.2"?
>
PLY-3.1 and 3.2 are only minor bug fix releases to 3.0. No
documentation was changed.
> 2) Is it OK to access lexstate? I have a custom lexer that is a
> wrapper class for ply.lex; itactually manages the token outputting and
> so I need to know its state (it works beautifully, by the way). The
> document doesn't say anything about accessing the state, so I browsed
> the lex.py source code to find out how :)
>
Probably okay to access as long as you go through the appropriate
"lexer" attribute on objects.
> 3) While coding up that same custom lexer class, I discovered that
> yacc doesn't seem to retrieve the 'tokens' keyword from the lexer
> class that I pass to (e.g. yacc.parser(lexer=mylexer)), so I
> eventually figured out a hack: move the tokens to outside the lexer
> class, then insert a 'tokens = tokens' line inside that class. It
> works...but it seems wrong somehow to me. Thoughts?
>
Yacc gets its token list from the tokens specification that appears in
the same location as all of the grammar rules.
> 4) When I set t.type to a literal (e.g. '%') in the lexer, Yacc (or
> was it Lex?) complains it doesn't recognize that token. How do I fix
> that?
>
You need to have a literals specification in the lexer for single
character literals to work. Also, literals have to be enclosed in
quotes in the grammar rules. No idea why % wouldn't work
specifically, unless it's getting mixed up with some Python string
formatting someplace.
> 5) I discover that yacc.py has enumerate(), which is a Python 2.3
> thing. The document says that PLY should work on Python 2.2. I guess
> it's time to bump up the version number, hmm?
>
Yep.
> 6) While coding up a generator written using PLY, I got a subtle and
> spurious "warning" bug re: p_error(). I studied the yacc.py code and
> determined that since it uses a simple minded text scanning, it
> flagged the p_error being embedded in a string. I thought to make a
> note of it for everyone. I elect to ignore this harmless warning.
>
Can you show me a test case or example. If it's a bug related to the
yacc implementation, I can try to fix it.
Cheers,
Dave
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"ply-hack" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en
-~----------~----~----~----~------~----~------~--~---
