[issue25324] Importing tokenize modifies token

2017-05-16 Thread Albert-Jan Nijburg
Changes by Albert-Jan Nijburg <albert...@trinket.io>: -- pull_requests: +1699 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python

[issue30377] Unnecessary complexity in tokenize.py around handling of comments and newlines

2017-05-17 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: I did yesterday, should be coming through today right? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/i

[issue30377] Unnecessary complexity in tokenize.py around handling of comments and newlines

2017-05-16 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: Oh yes you're right! I've updated the code on github. Even cleaner this way :). -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/i

[issue30377] Unnecessary complexity in tokenize.py around handling of comments and newlines

2017-05-19 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: Still no CLA, I checked my username on the pdf, and it's correct, hope someone looks at it soon :) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/i

[issue25324] Importing tokenize modifies token

2017-05-24 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: I've updated the PR and added the tokenize tokens to token.h and their names to tokenizer.c. This way they'll show up when you run token.py. The names will always be in tok_name and tokenizer.py will use those. Not breaking the public api and no longer

[issue30377] Unnecessary complexity in tokenize.py around handling of comments and newlines

2017-05-16 Thread Albert-Jan Nijburg
New submission from Albert-Jan Nijburg: While porting tokenize.py to javascript I stumbled upon this. The bit of code that checks if it's a newline or a comment, checks for comment twice. These can be split up, this way the code is a bit more readable. https://github.com/python/cpython/blob

[issue25324] Importing tokenize modifies token

2017-05-16 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: > I prefer to add tokenize tokens directly in Lib/token.py, and then get > COMMENT, NL and ENCODING using tok_name.index(). That would make more sense from a breaking change perspective, but we would step on the toes of anyone adding `COMMENT

[issue25324] Importing tokenize modifies token

2017-05-16 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: lib2to3 appears to have it's own token.py as well with NL and COMMENT withtout ENCODING... Lib/lib2to3/pgen2/token.py -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/i

[issue30455] Generate C code from token.py and not vice versa

2017-05-30 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: I think this covers all the changes from PR #1608. Looks a lot nicer too, building it every time from the make file. You may want to add to the docs that token.py is now the source of the tokens. -- ___ Python

[issue25324] Importing tokenize modifies token

2017-05-31 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: I've updated token.rst and Misc/NEWS. Let me know if the wording is correct. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/i

[issue25324] Importing tokenize modifies token

2017-06-01 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: Aah! Oops I can fix later today. On Thu, 1 Jun 2017 at 18:08, STINNER Victor <rep...@bugs.python.org> wrote: > > STINNER Victor added the comment: > > We got a bug report from Coverity: > > *** CID 1411801: Incorrect expression (

[issue25324] Importing tokenize modifies token

2017-06-01 Thread Albert-Jan Nijburg
Changes by Albert-Jan Nijburg <albert...@trinket.io>: -- pull_requests: +1990 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python

[issue25324] Importing tokenize modifies token

2017-05-30 Thread Albert-Jan Nijburg
Albert-Jan Nijburg added the comment: Let me know if you want me to add/change anything about my PR :) I'm happy to do so. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/i

[issue7167] Smarter FTP passive mode

2017-10-06 Thread Albert-Jan Nijburg
Albert-Jan Nijburg <albert...@trinket.io> added the comment: I understand the standpoint that the server is configured incorrectly, and I see why this might not be the best solution to the problem. But not everyone owns the ftp server they're connecting to. And the `EPSV` command