The Orange class is based on tokens. This token-based approach work
perfectly, except for the placement of spaces of around colons in slices.
Here are some examples, as recommend by pep 8
<https://www.python.org/dev/peps/pep-0008/> and as rendered by black, and
now Leo's Orange class:
a[1::]
a{1:2:]
a[1 : 1 + 2]
a[lower:upper:]
a[lower + offset : upper + offset]
a[:: step_fn(x)]
a[: upper_fn(x) : step_fn(x)]
a[: upper_fn(x) : 2 + 1 :]
As you can see, there should be spaces around all colons, with two
constraints:
Constraint 1: Put no spaces around colons if *all *parts of the slice
consists only of names and constants.
Constraint 2: Never put a space between a colon and a parenthesis.
This is a great example of the value of having access both to the token
list and the parse tree. Indeed, constraint 1 requires access to the parse
tree. It would be extremely clumsy to deduce this constraint from the token
list. Otoh, constraint 2 requires access to the token list. It would be
extremely clumsy to deduce this constraint from the parse tree.
Folks, this example was the motivation for the entire TokenOrderTraverser
class.
Here is Orange.colon, the code that handles incoming colon tokens:
def colon(self, val):
"""Handle a colon."""
node = self.token.node
self.clean('blank')
if not isinstance(node, ast.Slice):
self.add_token('op', val)
self.blank()
return
# A slice.
lower = getattr(node, 'lower', None)
upper = getattr(node, 'upper', None)
step = getattr(node, 'step', None)
expressions = (ast.BinOp, ast.Call, ast.IfExp, ast.UnaryOp)
if any(isinstance(z, expressions) for z in (lower, upper, step)):
prev = self.code_list[-1]
if prev.value not in '[:':
self.blank()
self.add_token('op', val)
self.blank()
else:
self.add_token('op-no-blanks', val)
The *only* things you have to know about are the following:
1. self.token.node is the parse tree associated with the incoming colon
token.
2. The colon is part of a slice if and only if isinstance(node, ast.Slice).
3. Within a slice, the code puts spaces around colons if and only if *any*
part of the slice is a non-trivial expression.
Getting 3 right is tricky. Suffice it to say that details are handled at
the token level, as usual in the Orange class.
*Summary*
The value of the TOG class lies in its power, not in easy-to-understand
snippets. For most people asttokens will suffice. But for ambitious
programs like fstringify and orange, the full power of the TOG class is
simplifies the code enormously.
Since the day I first conceived of token order traversals I have known that
the colon method would the acid test of the Orange class. Until yesterday
I had only a vague idea of what Orange.colon would look like. Clearly, it
is the simplest thing that could possibly work.
Dozens individual test cases in TestOrange.test_one_line_pet_peeves cover
this crucial method.
Edward
--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/4f25ed19-c18a-4725-94e5-92a7127d459d%40googlegroups.com.
