Documentation often primes the mental pump. So it was yesterday. In this
post <https://groups.google.com/d/msg/leo-editor/PsB8dTr2Wac/Yh49dHh6BAAJ>
I said:
QQQ
- The results list exists *only* to hold data to be transferred to the
tokens list.
- Syncing the tokens and results lists is the *only* way make this transfer!
- ...significant (non-optional) tokens appear in exactly the same order in
the two lists.
QQQ
Any time I assert that there is only one way of doing something my
subconscious starts screaming at me ;-) Eventually, I realized that the
notion of significant tokens could collapse the problem. Just synchronize
the two token lists on their significant tokens! Instantly everything
becomes dead easy.
Here are the three main methods of the Linker class, shown in simplified
form for clarity:
def assign_links(self, results, tokens, tree):
# Create the lists of significant tokens and results.
sig_tokens = list(filter(self.is_significant, tokens))
sig_results = list(filter(self.is_significant, results))
# Raise an exception if the two lists are not compatible.
self.check(sig_results, sig_tokens)
# Make two-way links between tokens and results.
for r, t in zip(sig_results, sig_tokens):
self.set_links(r, t, tokens)
tx = 0 # The index of the last patched token.
def set_links(self, r, t, tokens):
# Patch all previous assignable tokens.
while self.tx <= t.index:
token = tokens[self.tx]
if self.should_be_assigned(token, r.node):
# Patch the token.
token.node = r.node
# Add the token to r.node.token_list.
token_list = getattr(r.node, 'token_list', [])
r.node.token_list = token_list + [token]
self.tx += 1
def check(self, results, tokens):
n1, n2 = len(results), len(tokens)
assert n1 == n2, (n1, n2)
for r, t in zip(results, tokens):
assert r.kind == t.kind, (repr(r), repr(t))
assert self.compare_values(r, t), (repr(r), repr(t))
Subclasses may override the remaining three helper methods, shown here w/o
docstrings:
def is_significant(self, token):
return (
token.kind in ('name', 'number') or
token.kind == 'op' and token.value != ';')
def compare_values(self, r, t):
if t.kind == 'string':
val = True # to do. Must be more lenient.
else:
val = t.value == r.value
return val
def should_be_assigned(self, token, node):
return token.kind not in ('encoding', 'endmarker', 'ws')
That's all!
*Summary*
The linker is now the simplest thing that could possibly work.
It's also more flexible. Subclasses can define the three helper methods as
they like.
This is the way it is written in The Book.
Edward
--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/b85e78b9-d8bd-4b05-8c2f-946b46969cd1%40googlegroups.com.
