I have started work on #2541 <https://github.com/leo-editor/leo-editor/issues/2541>, which will add support for new ast nodes in Python 3.10.
After looking at leoAst.py with fresh eyes, the following Aha's emerged: 1. Using python generators in the TokenOrderGenerator borders on the absurd. Generators complicate the code patterns while providing no real benefits. Instead, it would be better to use recursive descent, as #2555 <https://github.com/leo-editor/leo-editor/issues/2555> suggests. There is little chance of exceeding python's stack limit. Anyway, sys.setrecursionlimit increases that limit. 2. After years of wondering, I have discovered a way to traverse any tree with neither recursion nor generators. Each visitor returns a list of (handler, argument) tuples, which the main loop then executes in the desired order. This scheme won't take the world by storm. The (handler, argument) tuples correspond to the closures that python creates behind the scenes for generators. Still, it solves a puzzle elegantly. See the postscript. *Summary* I will soon rewrite much of the code in leoAst.py. The legacy (generator-based) code was particularly stupid because the generators never returned anything! I may do a full implementation of the iterative tree traversal algorithm, both as a proof of concept and as a benchmark for comparison with other approaches. Edward P.S. Here is a slightly condensed version of the iterative main loop: def main_loop(self, node): func = getattr(self, 'do_' + node.__class__.__name__) exec_list = [(func, node)] while exec_list: func, arg = self.exec_list.pop(0) result = func(arg) if result: # Prepend the result, a list of tuples. self.exec_list[:0] = result For example, here is the visitor for the walrus operator: # NamedExpr(expr target, expr value) def do_NamedExpr(self, node): return [ (self.visit, node.target), (self.op, ':='), (self.visit, node.value), ] *Notice*: none of the visitors are recursive. They merely *represent* recursion by returning tuples whose handler is self.visit. The maximum stack depth will be about 5, independent of the ast tree. EKR -- You received this message because you are subscribed to the Google Groups "leo-editor" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/dd216465-7ce1-4b67-b340-d7c88378b37dn%40googlegroups.com.
