[issue39686] add dump_json to ast module

[Cheryl Sabella]

Change by Cheryl Sabella :


--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Richard K]

Richard K  added the comment:

> I don't think the clang argument holds because clang is a command-line tool 
> after all and it makes sense that it can produce several outputs while the 
> ast module is exposes APIs that you can further process inside the language. 
> Having json from the clang output will require more than one tool if clang 
> does not support it while doing it in Python only requires Python.

I see what you mean. I was just trying to illustrate that such a feature is 
desired by some. 

Perhaps 'Python only requires Python' means that Python _could_ be the first 
widely used language with such a superior meta-programming feature with respect 
to AST analysis/code generation. 


> > it appears that they do so in non-standard ways.

> Can you clarify what do you mean with that? 

By non-standard I mean that the resulting json does not follow the structure of 
the tree explicitly. For example with ast2json, '"_type": "Print"' includes a 
(somewhat misleading) key that is not represented in the actual AST. 

Example of ast2json output (example found here, 
https://github.com/YoloSwagTeam/ast2json#example),

{
"body": [
{
"_type": "Print",
"nl": true,
"col_offset": 0,
"dest": null,
"values": [
{
"s": "Hello World!",
"_type": "Str",
"lineno": 1,
"col_offset": 6
}
],
"lineno": 1
}
],
"_type": "Module"
}


> Just to clarify: ast.dump *will* fail with a more deph object as well, I am 
> not claiming that ast.dump will parse everything because of course suffers 
> the same problem.

Makes sense. As you mentioned, these are edge cases which I assume will not be 
an issue for those seeking to gain the benefits of 'ast.dump_json'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Just to clarify: ast.dump *will* fail with a more deph object as well, I am not 
claiming that ast.dump will parse everything because of course suffers the same 
problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> There seems to be movement towards a general usage. For instance, take a look 
> at clang, in particular the flag '-ast-dump=json'.

I don't think the clang argument holds because clang is a command-line tool 
after all and it makes sense that it can produce several outputs while the ast 
module is exposes APIs that you can further process inside the language. Having 
json from the clang output will require more than one tool if clang does not 
support it while doing it in Python only requires Python.

> it appears that they do so in non-standard ways.

Can you clarify what do you mean with that? 

> The implementation of ast.dump also uses recursion. I have tested 
> ast.dump_json on sufficiently large source files and have not run into 
> recursion depth exceeded issues.

This is not the primary argumet as by itself is weaker because this is an edge 
case but for instance, here is an example of ast.dump succeeding and your tool 
failing:


>>> x = ast.List()
>>> for _ in range(1010):
...: x = ast.List(x)
...:

>>> ast.dump(x)
'List(elts=List(elts=List(elts=List(elts=List(elts=L..

>>> dump_json(x)
---
RecursionErrorTraceback (most recent call last)
 in 
RecursionError: maximum recursion depth exceeded while calling a Python object

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Richard K]

Richard K  added the comment:

Batuhan & Pablo thank you for your thoughts! Just wanted to reply to a few of 
the comments to clarify my position on the issue.


> IMHO this is not a feature that has a general usage. If you want, as far as I 
> can see, there are some packages for doing that in PyPI already. Also, the 
> patch looks small so you can just add this to the required project.


There seems to be movement towards a general usage. For instance, take a look 
at clang, in particular the flag '-ast-dump=json'.

$ clang -cc1 -ast-dump=json foo.cc


> ast.dump now can dump in pretty-printed way.

Indeed however, there is not much one can do further with the output of 
ast.dump. With ast.dump_json one would benefit from programmer-centric 
functionality.

-- 

> Thanks, Richard for your proposal. I concur with Batuhan: I am -1 as well on 
> this addition. Echoing some of the same ideas, I think this is specialized 
> enough that does not make sense to have it in the standard library, 
> especially if a Pypi package already exists. 


After just browsing the the pypi package/s you may be referring to, it appears 
that they do so in non-standard ways.


> Additionally, this is straightforward to implement for very simple cases but 
> PR18558 will fail for very generic ASTs if they are deep enough (it uses 
> recursion).


The implementation of ast.dump also uses recursion. I have tested ast.dump_json 
on sufficiently large source files and have not run into recursion depth 
exceeded issues.


Thanks again for your perspective!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Thanks, Richard for your proposal. I concur with Batuhan: I am -1 as well on 
this addition. Echoing some of the same ideas, I think this is specialized 
enough that does not make sense to have it in the standard library, especially 
if a Pypi package already exists. Additionally, this is straightforward to 
implement for very simple cases but PR18558 will fail for very generic ASTs if 
they are deep enough (it uses recursion).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

> The proposed enhancement would provide a complementary function, `dump_json` 
> as in a json representation of the ast. 

IMHO this is not a feature that has a general usage. If you want, as far as I 
can see, there are some packages for doing that in PyPI already. Also, the 
patch looks small so you can just add this to the required project.

> This would be useful for those who would like to benefit from the utilities 
> of the json module for formatting, pretty-printing, and the like.  

ast.dump now can dump in pretty-printed way.

> It would also be useful for those who want to serialize the AST or export it 
> in a form that can be consumed in an other programming language.

As I said, serializing/exporting the AST isn't a common use case. It is a very 
specific goal.

In general, I'm -1 on this.

--
nosy: +benjamin.peterson, pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Richard K]

Change by Richard K :


--
keywords: +patch
pull_requests: +17938
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18558

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39686] add dump_json to ast module

[Richard K]

New submission from Richard K :

Currently within the ast module, `dump` generates a string representation of 
the AST for example,

>>> ast.dump(node)
'Module(body=[], type_ignores=[])'


The proposed enhancement would provide a complementary function, `dump_json` as 
in a json representation of the ast. 
This would be useful for those who would like to benefit from the utilities of 
the json module for formatting, pretty-printing, and the like.  
It would also be useful for those who want to serialize the AST or export it in 
a form that can be consumed in an other programming language.
A simplified example, 


>>> import ast
>>> node = ast.parse('')
>>> ast.dump_json(node)
{'Module': {'body': [], 'type_ignores': []}}


A simplified example of using `ast.dump_json` with the json module,

>>> import json
>>> json.dumps(ast.dump_json(node))
'{"Module": {"body": [], "type_ignores": []}}'

--
components: Library (Lib)
messages: 362256
nosy: sparverius
priority: normal
severity: normal
status: open
title: add dump_json to ast module
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com