On Wed, Jan 15, 2020 at 3:57 AM Edward K. Ream <[email protected]> wrote:
> On Tue, Jan 14, 2020 at 7:06 PM Brian Theado <[email protected]> > wrote: > [...] > The ast module is particularly deficient in this regard. The documentation > for ast.walk <https://docs.python.org/3/library/ast.html#ast.walk> is: > > " Recursively yield all descendant nodes in the tree starting at *node* > (including *node* itself), in no specified order. This is useful if you > only want to modify nodes in place and don’t care about the context." > > Hmm. This could be one motivating example. The TOG class inserts > parent/child links, and TOT.traverse(tree) *is* the token order > traversal. So a much more valuable version of ast.walk would be: > > tog = TokenOrderGenerator() > tot = TokenOrderTraverser() > contents, encoding, tokens, tree = tog.init_from_file(filename) > tot.traverse(tree) > But in its current form, this example doesn't do anything user visible. It doesn't return anything and it doesn't display anything. IMO, examples should as much as possible be standalone and do something the user can see. That way the user can paste the exact code into a REPL and see something happen. Or better, the documentation showing the example can also show the results. As it is now, the user would have to define a subclass of TOT in order to see anything. This is what I was complaining about in another thread. In order to be usable, at a minimum a 5 line subclass needs to be written. I proposed that you move the logic from that class into a function which yields each result. That function could be used both by the TOT class and by simple for loops (perfect for demo and explanations). But your response was "You have little chance of changing my mind" :-). Maybe my suggestion isn't the right approach, but I still think there is something missing regarding ease-of-use. [...] > The injected links will be useful for any tool that wants to modify python > source code. fstringify and black are two most prominent examples. Now we > come to the real motivation. This is the "hole" in the documentation I have > been talking about. > > *Any* tool that wants to modify python text will benefit from having > *both* a token-level view of the text *and* a parse-tree level view of > the text. The asttokens package provides this dual view, but only for > top-level python statements. In effect, the TOG class is a much simpler > implementation of the asttokens package. > > This suggests that some wrapper functions, similar/identical to those in > the asttokens package, would be useful. > If this means you will be able to write simple examples in the form of the one on the front page of the asttokens package, then it sounds useful. [...] Thanks for all the other explanation. -- You received this message because you are subscribed to the Google Groups "leo-editor" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/CAO5X8CzfYaG-a3xjN%3DJw_4Az9jSo63BWRe9uk753h%2BxErgOeiQ%40mail.gmail.com.
