In the theory of operation: "The notion of a *token order traversal* of a parse tree is the foundation of this project"
In contrast, what traversal order do parse trees provide? How is token order different/better? What does it allow me to do that I can't otherwise do with parse trees? "This help is essential, because the following two source files generate identical parse trees!" This implies to me that with a parse tree you can't do a round trip from source code, to a parse tree and back to the original source code. Is that correct? If so is this just one externally visible benefit to your library or is it the main benefit? If it is the main benefit, then I think it should be made much more clear earlier in the document like in the Overview near here: "These links promise to collapse the complexity of any code that changes text, including the asttokens <https://pypi.org/project/asttokens/>, fstringify <https://pypi.org/project/fstringify/>, and black <https://pypi.org/project/black/> projects." something like: "These links allow portions of python source code to be transformed while leaving the rest untouched. Much of the complexity of the asttokens <https://pypi.org/project/asttokens/>, fstringify <https://pypi.org/project/fstringify/>, and black <https://pypi.org/project/black/> projects comes from not being able to link between the text and structure of the Python code. Using the code of this library can collapse the complexity of these and any projects which change Python text" On Tue, Jan 14, 2020 at 6:10 PM Brian Theado <brian.the...@gmail.com> wrote: > Ok, now I see in your proposed email response in the other thread you do > mention the asttokens project. And also later in #1440 you said "The > asttokens <https://asttokens.readthedocs.io/en/latest/> project assigns > tokens to ast nodes, but only for statement-level ast nodes." That's useful > to know. > > One thing I really like about the front page of that project ( > https://github.com/gristlabs/asttokens) is the front-and-center example > which shows in a few lines of code how to call the code using concrete > inputs and shows exactly what the outputs are. > > I find that very valuable. Examples are usually the first thing I want to > see. Can you have something similar in #1440? > > On Tue, Jan 14, 2020 at 5:59 PM Brian Theado <brian.the...@gmail.com> > wrote: > >> I just followed the stackoverflow link ( >> https://stackoverflow.com/questions/16748029/how-to-get-source-corresponding-to-a-python-ast-node#) >> and someone posted that they created >> https://github.com/gristlabs/asttokens which "Annotates Python AST trees >> with source text and token information". Is that different from what >> your code does? >> >> On Tue, Jan 14, 2020 at 8:02 AM Edward K. Ream <edream...@gmail.com> >> wrote: >> >>> In a day or three I'll be announcing #1440 >>> <https://github.com/leo-editor/leo-editor/issues/1440>, the unification >>> of parse trees and tokens, to python's core developers. I'll do that by >>> making a comment in this related python issue >>> <https://bugs.python.org/issue33337>. >>> >>> In preparation for that announcement, I have revised/created the first >>> three comments of #1440: >>> >>> - The first (main) comment >>> <https://github.com/leo-editor/leo-editor/issues/1440> gives an >>> overview of the project and its status. >>> - The second comment >>> <https://github.com/leo-editor/leo-editor/issues/1440#issuecomment-573661883> >>> is the Theory of Operation. >>> - The third comment >>> <https://github.com/leo-editor/leo-editor/issues/1440#issuecomment-574145510> >>> contains the Project's history. >>> >>> The announcement itself will be fairly brief, and will refer readers to >>> #1440 for more details. So I want these three comments to be "perfect". >>> >>> Please read and critique all three comments. I am interested in your >>> comments and suggestions. >>> >>> Edward >>> >>> P. S. There are similar discussions in LeoDocs.leo, but I'll be >>> referring to #1440 in the upcoming public announcements. The links above >>> are good looking and are available to anyone, including those with no >>> access to Leo. >>> >>> EKR >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "leo-editor" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to leo-editor+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/leo-editor/5c6da99b-360f-4ab2-bae9-f9a4d589e860%40googlegroups.com >>> <https://groups.google.com/d/msgid/leo-editor/5c6da99b-360f-4ab2-bae9-f9a4d589e860%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "leo-editor" group. To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/CAO5X8CwyH3t_e_Zto-QeJFXQ1CTUUdidAovYYVweyS50t26%2BoA%40mail.gmail.com.
