[issue16121] shlex.shlex.error_leader() reports incorrect line number
Birk Nilson added the comment: The implementation incremented the line number immediately when a newline was detected, even before the token had been processed completely - causing the issue Arfrever posted. This also caused the unexpected behavior of a tokens line number including the amount of lines within the token itself. In other words '"a \n b \n c" \n d' would result in the token "a \n b \n c" to have the line number #3, followed by "d" being on the expected line #4. In my patch, the expected behavior of seeing the first token on line #1 and the second on #4 is introduced. I also added the testLineNumbers method in the test suite - included in the patch. -- keywords: +patch nosy: +birknilson Added file: http://bugs.python.org/file29207/issue16121.patch ___ Python tracker <http://bugs.python.org/issue16121> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16121] shlex.shlex.error_leader() reports incorrect line number
Birk Nilson added the comment: Sorry about that. Rookie mistake. I'm investigating it now and will get back with a revised & fully tested patch. -- ___ Python tracker <http://bugs.python.org/issue16121> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16121] shlex.shlex.error_leader() reports incorrect line number
Birk Nilson added the comment: After investigating the issue I have a couple of proposals. Although a bit vague, the documentation of shlex.lineo seems to suggest that it should be incremented immediately on finding a newline character. Changing this to allow wrapped lines within a token without incrementing the line number changes the existing shlex API. Something I believe should be avoided since netrc relies on the existing behavior and third-party modules might too. Instead I recommend the following steps. Step #1: Fix the immediate issue of getting different line numbers for the same input depending on whether posix=(True|False), but keep the current - greedy - behavior of shlex.lineo. Step #2: A separate patch introduces shlex.wrapped_lineo which does not increment the lineno immediately, but prior to reading the next token - as introduced in my previous patch. Step #2 should arguably be introduced in a separate issue - if at all - since it is a new feature to the shlex API. I will provide a patch for #1 within the next day or two along with one for #2 if you guys think it is a good idea. -- ___ Python tracker <http://bugs.python.org/issue16121> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
