Patches item #1070218, was opened at 2004-11-20 22:58 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1070218&group_id=5470
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Grant Olson (logistix) Assigned to: Jeremy Hylton (jhylton) Summary: Add BLANK_LINE to token.py Initial Comment: The parser module generates an ast node number 54 for an entirely blank line. These lines are apparently ignored by the real parser, and don't have a real definition in include\token.h. Presumably because of that they don't have an entry in token.py. However, this broke an app where I hardcoded the number and recent language features have changed the mapping of numbers to token names. I think there should be a BLANK_LINE defined in token.py so that it can be referred to by name, even if it's not used in the real parser. While dealing with parsermodule generated ast-trees, "symbol.sym_name[node] or token.tok_name[node]" should always be true. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2006-11-19 09:33 Message: Logged In: YES user_id=21627 Originator: NO As you found yourself, the patch is wrong. This token type is already exposed from tokenize as tokenize.NL; if you use tokenize, you should use its list of token types, not the one of token.py. Closing this patch as rejected. ---------------------------------------------------------------------- Comment By: Grant Olson (logistix) Date: 2004-11-22 01:41 Message: Logged In: YES user_id=699438 A few clarifications: On further review, the node number 54 comes from tokenize.py, not the parsermodule. I also meant to say that "symbol.sym_name.has_key(node) or token.tok_name.has_key(node)" should always be true. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1070218&group_id=5470 _______________________________________________ Patches mailing list Patches@python.org http://mail.python.org/mailman/listinfo/patches
