I've been saying that the new @auto project solves a problem that I've been
working on for almost 20 years, namely, how to represent structure without
using sentinels. Alas, this is only partially true. As explained below,
the new @auto project does not solve serious, fundamental problems with
@auto.
The new @auto project does indeed introduce important new technology. It
uses unl's (text descriptions of tree structure) to associate data with the
nodes created by the @auto importers. As a result, the new importer can
reliably do the following:
1. Add organizer nodes to the imported nodes.
2. Add clone links from imported nodes to other nodes in the outline.
3. Associate uA's with imported nodes.
4. Change the headlines of imported nodes to "custom" headlines.
These features will make Leo much more attractive to use in collaborative
environments. Alas, the following serious problems with @auto remain:
1. Importers are more complex, error-prone, and slower than Leo's @file
read code.
Leo's @file read code knows nothing about any programming language; the
parser for each importer is language specific. Although far from trivial,
the @file read code is fundamentally simpler than any importer code--the
@file read code just handles each sentinel as it appears in the source file.
A line-based parsing scheme for importers merits consideration. It
promises to simplify and accelerate the importer code, but it is not a
cure-all. Surprisingly, the code generators in the base importer class are
likely to remain unpleasantly complex. As a result, it seems unlikely that
it will ever be possible to do without perfect import checks.
2. Importers, can not, in principle, create outlines equivalent to the
original source file.
This is most apparent in languages like html and JavaScript, where the
"units" of text can start anywhere on a line. The perfect import checks
for such languages must be **lenient**: they must allow for inserted
newlines between html tags or JavaScript functions. This can't be helped,
but it does mean that round-tripping an external file through Leo may
insert extra newlines. I see no easy (non-heroic) way of avoiding such
changes.
3. Perfect import checks are slow.
Worse, with the new @auto code the perfect-import checks must be done
twice: once by the importers themselves, and once by the new import code.
The first three points deal with "behind-the-scenes" issues. The next
three points affect users more directly.
4. Importers never generate section references.
It's surprisingly annoying to do without section references. I have said
many times that I consider it bad style to replace functions/methods with
sections, but sections have another use, namely to indicate that some nodes
must precede others in the outline. In Python, for example, definitions of
base classes must preceded definitions of subclasses. And of course, it's
often useful to use sections to group Python imports.
5. Importers can not know where to put **interstitial** code, code that
appears between defs or classes.
This may be the most serious problem of all. For example, consider the
following Python snippet.
anIvar= False
def spam(self):
'''A method using anIvar.'''
foo = spam
Clearly, we want all this code to appear in the same node. No problem for
@file, but a *big* problem for Leo's importers. Yes, we could imagine
creating a convention (presumably involving comments) that would guide the
importers, but once again this assumes stealth sentinels of some kind.
Furthermore, code from other projects will not follow such conventions.
There is no obvious solution. Yes, one could imagine yet another kind of
data contained in an @views tree that would specify that the code above
should appear in a single code. But this will be *content-centric* data.
Unlike unl-centric data presently used, such data will inevitably break
when contents changes. It doesn't seem like a viable option. BTW, @shadow
does solve this particular problem, but I see no way at present to meld the
best features of @shadow and @auto.
This problem could be fatal. In the example above, we expect one of the
assignments (to anIvar or foo) to end up in the wrong node. But this means
that reorganizing the @auto tree may break the program by causing an
undefined variable at runtime. Yes, this has always been a problem with
@auto. It's difficult to accept that the whole project could founder on
such a detail, but I see no way forward.
6. Importers can not organize leading declarations.
At present, importers put all code appearing before the first class or
function in a node call "declarations for x". Lacking sentinels, this is
the best the importers can do. Importers could create some kind of
"default" organization, but once again the user has no say in it.
===== Summary
Before writing this post, I was not aware just how problematic @auto always
has been. This is extremely disappointing. Despite spectacular new
technology, @auto seems destined to remain far less useful than @file. As
a result, I see no way to use @auto for Leo's sources. Maybe some new
insight will breath new life into the new @auto project, but I don't see
how that can happen. Boo hoo.
My plan is to wrap up the new @auto project and move on to the vim project.
Your comments please, Amigos.
Edward
--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/groups/opt_out.
