[issue24119] Carry comments with the AST

2020-03-12 Thread Guido van Rossum 


Guido van Rossum  added the comment:

I propose to close this issue, since (as of Python 3.8) we now have 
ast.parse(source, type_comments=True).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2020-03-12 Thread Batuhan Taskaya 


Change by Batuhan Taskaya :


--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2019-01-19 Thread Guido van Rossum 


Guido van Rossum  added the comment:

See also issue35766.

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2019-01-12 Thread Raymond Hettinger 


Raymond Hettinger  added the comment:

Since the AST carries the module/lineno attributes isn't there already a way 
trace back into the token stream to recover comments?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2019-01-12 Thread Raymond Hettinger 


Raymond Hettinger  added the comment:

Possible superceder:  https://bugs.python.org/issue7

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2019-01-12 Thread kernc 


Change by kernc :


--
nosy: +kernc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2017-05-01 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

Where would comment be attached in the following case?

a = ('start' # comment 1
 'continuation') # comment 2

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2017-05-01 Thread Brett Cannon 

Brett Cannon added the comment:

There's potentially some usefulness from other tools, but Raymond is right that 
the main motivation is definitely gone long-term. Dropping this down to "low" 
priority simply because others have asked for this kind of support before.

--
priority: normal -> low

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2017-05-01 Thread Raymond Hettinger 

Raymond Hettinger added the comment:

After PEP 526, the need for this proposal may have evaporated.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2017-05-01 Thread Louie Lu 

Louie Lu added the comment:

Brett, which implement method will you prefer?

If we want to carry comment at builtin_compile_impl, it will need to change the 
grammar since tokenize just drop the comment when dealing with source code.

But if just using regex, will it be more easy with just combine the exists 
tokenize and ast module?

--
nosy: +louielu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2017-03-15 Thread Brett Cannon 

Brett Cannon added the comment:

The type annotation is already in the AST so there's nothing to carry over from 
typed_ast (we only care about the latest Python version while typed_ast tries 
to be version-agnostic).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2017-03-15 Thread Mateusz Bysiek 

Mateusz Bysiek added the comment:

For some time now, there's an alternate ast implementation 
https://github.com/python/typed_ast that carries PEP 484 type comments with the 
AST as attributes of certain nodes.

Their approach is described here: 
https://github.com/python/typed_ast/blob/master/typed_ast/ast3.py#L5

If type comments become mainstream in Python, could this approach maybe be 
adopted as official Python AST at some point?

--
nosy: +mbdevpl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2015-05-05 Thread Antoine Pitrou 

Antoine Pitrou added the comment:

Or a separate AST node - comment mapping.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24119
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2015-05-05 Thread Brett Cannon 

Brett Cannon added the comment:

Another option is to provide a tool in 'tokenize' or 'ast' which will take the 
source and some comment regex and then attach the found comment metadata to the 
AST.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24119
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2015-05-04 Thread Brett Cannon 

Brett Cannon added the comment:

Normally I would agree comments don't belong there, but if we are going to 
start giving them semantic meaning then I don't think it's not so clear to me 
anymore.

As to where to attach, simple place is off of the Module node. Another is to 
have it be fundamental like lineno and only attach it when it is a 
line-trailing comment.

Yes, the tokenize module will give you the comments as well, but it is 
unfortunate you have to parse the code twice in order to get the comments and 
the AST.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24119
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2015-05-04 Thread Christian Heimes 

Changes by Christian Heimes li...@cheimes.de:


--
nosy: +christian.heimes

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24119
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2015-05-03 Thread Brett Cannon 

New submission from Brett Cannon:

One thing about https://www.python.org/dev/peps/pep-0484/ is that it makes 
comments potentially semantically meaningful. Unfortunately the AST doesn't 
carry comments with it in any way, making it difficult to build a tool to 
implement a linter for PEP 484 using purely the ast module. Even if comments 
were carried along side-band and could do correlation by line number would be 
useful in this scenario.

I thought an issue had previously existed for this topic but I could find it.

--
components: Library (Lib)
messages: 242485
nosy: brett.cannon
priority: normal
severity: normal
stage: test needed
status: open
title: Carry comments with the AST
type: enhancement
versions: Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24119
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24119] Carry comments with the AST

2015-05-03 Thread Mark Shannon 

Mark Shannon added the comment:

Comments don't belong on the AST. Where would you attach them?

The tokenizer module provides all information about comments. Tools can get the 
information quite easily if they need it.

--
nosy: +Mark.Shannon

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24119
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com