[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

These were all the cleanups on my checklist, so can close the issue now. I'll 
probably create other issues if something additional comes up

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:


New changeset 107a2c59c91b3911bdd6dfdb83271c588c506a5a by Batuhan Taskaya in 
branch 'main':
bpo-40528: fix is_simple(sum)s behavior for attributes (GH-26918)
https://github.com/python/cpython/commit/107a2c59c91b3911bdd6dfdb83271c588c506a5a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +25491
pull_request: https://github.com/python/cpython/pull/26918

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:


New changeset 6c76df2b86d742cc294beae7f9b6bafabb946ad5 by Batuhan Taskaya in 
branch 'main':
bpo-40528: move asdl identifier collection to the new metadata system (GH-26858)
https://github.com/python/cpython/commit/6c76df2b86d742cc294beae7f9b6bafabb946ad5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +25435
pull_request: https://github.com/python/cpython/pull/26858

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:


New changeset 35ad425866d591c33d7f2be2b9da8bce2bff9523 by Batuhan Taskaya in 
branch 'main':
bpo-40528: Implement a metadata system for ASDL Generator (GH-20193)
https://github.com/python/cpython/commit/35ad425866d591c33d7f2be2b9da8bce2bff9523


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +19494
pull_request: https://github.com/python/cpython/pull/20193

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 63b8e0cba3d43e53a8dd8878ee1443c8427f462d by Batuhan Taskaya in 
branch 'master':
bpo-40528: Improve AST generation script to do builds simultaneously (GH-19968)
https://github.com/python/cpython/commit/63b8e0cba3d43e53a8dd8878ee1443c8427f462d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

Next is using asdl_int_seq for all simple sum types, and not for only cmpop. 
Looks like it is hardcoded to use it, and I plan to refactor this in a way that 
might save some time in python implementation of classes (also it will solve 
this bug). For not conflicting, I'll wait GH 19968.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +19283
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19968

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 091951a67c832db83c60f4eb22f1fb474b70e635 by Batuhan Taskaya in 
branch 'master':
bpo-40528: Improve and clear several aspects of the ASDL definition code for 
the AST (GH-19952)
https://github.com/python/cpython/commit/091951a67c832db83c60f4eb22f1fb474b70e635


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

A real world example would be tools like `unimport`, that try to remove a 
certain part of import by looking start/end column offsets. Before (lib2to3), 
it was using tokens to manipulate source, and what I can tell is that having 
position information on multi line from imports would be a life saver for tool 
authors.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

Oh, I confuse with it identifier. IMHO replacing identifier with Constant would 
infer position in some cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> For which of them a line number is meaningful and useful?

Apologies, I was thinking of 'identifier', which is basically a PyObject 
representing a string. For instance, when parsing function calls like

x = ast.parse("f(x=435)")

the 'arg' attribute of the keyword is just the string 'x', without any metadata 
about it (like column offset and such).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

"string" is only used for

* type_comment in a number of statements
* kind in Constant
* tag in type_ignore

For which of them a line number is meaningful and useful?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

> I was also thinking about getting rid of "string" nodes and just use Constant 
> for those. Having the strings raw make the AST inconsistent because nodes 
> that are 'strings' do not have line numbers and other metadata, making tools 
> that need the source to report stuff more complex than it should.

I'm +1 on that (especially for finding import aliases, as we discussed 
earlier), if necessary precautions happen to ensure minimum breakage (from what 
I understand, it is something that will break everyone's code. Maybe we can 
implement custom Constant.__str__? and other different things to ensure it 
breaks small).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I was also thinking about getting rid of "string" nodes and just use Constant 
for those. Having the strings raw make the AST inconsistent because nodes that 
are 'strings' do not have line numbers and other metadata, making tools that 
need the source to report stuff more complex than it should.

What are your opinions on this @serhiy.storchaka and  @BTaskaya?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

Currently working on implementing some items from this TODO 
https://github.com/eliben/asdl_parser/blob/master/TODO

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

> You just read my thoughts. That's what I was going to do tonight. ;)

:) Awesome. Well I'll plan to continue this; @pablogsal and I was discussing 
the idea of extracting _ast.AST subclasses to python level (which would save a 
lot of time for new additions). So my primary goal is clearing bunch of old 
stuff / making it extendable and clean.

--
nosy: +pablogsal
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

You just read my thoughts. That's what I was going to do tonight. ;)

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
keywords: +patch
pull_requests: +19268
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19952

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40528] Improve / Clear ASDL generator

[Batuhan Taskaya]

New submission from Batuhan Taskaya :

- Better error messages with punctuation
- Py_UNREACHABLE() for unreachable states
- Removal of several 'unused' and 'undocumented (in Zephyr ASDL spec) built-in 
types.

--
components: Interpreter Core
messages: 368235
nosy: BTaskaya
priority: normal
severity: normal
status: open
title: Improve / Clear ASDL generator
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com