I'm appealing to the Leo gurus out there to think outside the box --
outside the Python world -- to come up with a LeoBridge II-like
application that =supports clones=.
It's OK to tell me if the ideas below are beyond Leo's current level of
integration capability, i.e., it just can't be done. Or, maybe it's
possible, but just too involved and would take a team of Python+Leo
experts months or years to implement. I would (reluctantly) accept
either of these answers and just give up on the idea of using Leo for
the kind of project management that I think it would excel at.
I've put a lot of time and thought into the high-level design outlined
below; I just can't make it happen because I don't know enough about
Python and even less about Leo's internal workings. I'm not opposed to
learning, and I =have= made several attempts to figure this out on my
own in the last several months, but the learning curve for Python+Leo
combined is just too steep for me. I don't work in Python every day like
other (most?) Leo users. I don't have a need to use Leo for Python app
development so Leo's Python-specific features aren't a benefit to me. I
can't even make use of Leo's fantastic IDE capabilities because there's
no way to integrate it to the programming language I use every day. Even
if I did know Python better, I have read several times about the
delicacy of some of Leo's internals -- that tells me I'm really not the
person to go messing with Leo's code.
I've lost count of the number of times I've started a new project and
thought, "Gee, this would be incredibly easier to manage with Leo's
powerful clone feature, if I could just figure out an easy/simple way to
get everything I needed from different sources =into= Leo." Project
management is what is driving my request to find someone to work with me
on this.
Design discussion:
======================
1) Support for clones.
======================
1a) It would be my responsibility to come up with a unique identifier
that is =meaningful to me= (for my work flow, this would be a six to
eight digit alphanumeric reference#), and that would become a node's
headline. The body associated with this node would likely contain text.
1b) It would be my responsibility to keep track of when this uniquely
identifiable node is first sent to Leo, because the first time is when
the body would also be sent.
1c) It would be my responsibility to tell Leo -- in some way -- that
this node is to be used later for cloning. I can live with the fact that
I may need to add some special character or word to the unique
identifier's headline that indicates this, if no other method seems
practical (like the "node type" method described below), but if possible
I'd like Leo to strip this special character/word when it creates the
headline.
1d) As I continue to stream data to Leo, when I want a cloned node to be
made I would pass only the unique identifier and Leo would create a
cloned node with the appropriate gnx.
1e) Leo needs a way to associate the gnx with the unique identifier.
Possible ways I can think of:
- When Leo sees "'" or "@clone" (or whatever method ends up being used,
I'll use "@clone" for the rest of this discussion), it creates/updates a
special "@clone-list" node in the outline so Leo has a specific place to
look, rather than having to search the entire outline each time looking
for a match. At the start of the import/data streaming process, Leo
would just need to search the entire outline only once to find the
"@clone list", and could then limit searching to only this node. When I
pass a new unique identifier in the data stream, Leo would create a new
child node in "@clone-list".
- OR, at the start of the import/data streaming process, Leo builds a
headline-and-gnx table in memory by reading through the entire outline,
for all headlines. When @clone headlines are encountered in the data
stream, it searches this memory table to find the matching gnx. If it
doesn't find one, it assigns a gnx for this new node as normal, and adds
it to the memory table so it'll "know" about it the next time it
encounters @clone in the data stream.
- OR, less efficiently, when @clone is encountered in the data stream,
it looks through the entire outline (headlines only) each time to find a
match.
=======================================================================
2) Support data streams passed to Leo using other programming languages
(i.e., non-Python) and other methods external to Leo.
=======================================================================
For example, maybe I have a C program, or a Perl script, or a VB
program, or a tclsh or bash script that I want to pass the data as
arguments to Leo. LeoBridge II doesn't need a UI, just a way to "listen"
for data.
Or maybe there's a little LeoBridge II server listening on a =network=
socket. Maybe nc (netcat) is the client passing data to LeoBridge II.
Or maybe LeoBridge II has a way to listen on its stdin.
Maybe, for the Windows environment, I could call a leobridge.dll.
Perhaps the format of the data stream passed to Leo could have these
four fields/arguments:
- node type:
------------
R=regular node
U=node headline defines a unique identifier to be used for cloning
C=node headline is a unique identifier and Leo needs to create it as
a cloned node
- node level:
-------------
absolute (0=root, 1=first level, 2=second level, etc.)
or
relative
"+0" or "S" = sibling, new node at same level as previous node
"+1" or "P" = promote
"-1" or "D" = demote
- node headline text
--------------------
- node body text (ignored for node type "C")
----------------
Perhaps the above fields are passed to LeoBridge II tab delimited, or
binary zero terminated strings if I'm calling LeoBridge II from C, or
maybe as a CSV. (Whatever makes the most sense for the data being passed
-- something to handle quotes, double quotes, commas, etc. in headlines,
and also handle newlines in body text.)
Here's an example using absolute node levels:
R,0,"Fruit","Scientific/Latin name"
U,1,"Apple","Malus pumila"
U,1,"Pineapple","Ananas comosus"
U,1,"Kiwi","Actinidia deliciosa"
U,1,"Peach","Prunus perscia"
U,1,"Lime","Citrus aurantiifolia"
R,0,"Fruits I want to keep stocked in the house",""
C,1,"Apple",""
C,1,"Peach",""
C,1,"Kiwi",""
R,0,"Shopping list for tomorrow's trip to the store",""
R,1,"Bread",""
R,1,"Milk",""
C,1,"Apple",""
C,1,"Kiwi",""
R,1,"Butter",""
R,1,"Diet Coke","it's on sale this week!"
Results in a Leo outline that looks like this (ignoring body text):
Fruit
*Apple
*Pineapple
*Kiwi
*Peach
*Lime
Fruits I want to keep stocked in the house
*Apple
*Peach
*Kiwi
Shopping list for tomorrow's trip to the store
Bread
Milk
*Apple
*Kiwi
Butter
Diet Coke
Where "*" indicates Leo's red "cloned node" icon.
Another way I've thought of is if LeoBridge II was able to acknowledge
acceptance of each node-worth of data I send it, by telling me the gnx
it assigned, then I could take on the responsibility of keeping track of
the gnx's for the clones. This would require LeoBridge II to be
interacting with an application, instead of just reading the data stream
in one gulp. The "C" node type above could then have the gnx in the node
headline field/argument.
=============================================
3) Support adding to an existing Leo outline.
=============================================
For my work flow, I would start by importing a list of the unique
identifiers and their associated body text, because they are all in one
file.
A second file has different regular nodes. It also has the unique
identifiers from file one sprinkled throughout that are placeholders for
where I want clones to be created.
A third file would have another set of textual elements, created from a
different data dump, that would also have unique identifiers for
creating clones.
There's a need for flexibility in building up the Leo outline from
various source data and at various times through the life of the project.
==============
Other comments
==============
The rest of this message contains responses to prior posts on this
thread where people tried to help me. Originally I was asking for help
to modify leoImport.py's MORE parsing routines:
> Rather than create a new file format, and complex code to parse same,
> my inclination would be to create a python script that would import
> the information directly into a Leo outline. It that works it will be
> much simpler than the way you are proposing.
>
> In other words, creating an intermediate file which you then import
> creates a very complex unnecessary step. If you have information you
> want to import, it will probably be much easier to open that
> information in a Leo script, which then creates nodes using
> p.insertAsNthChild or p.insertAfter.
I agree it would be simpler, and I've heard what you said and understand
the concept, it's just that this is beyond my Python+Leo knowledge
level. But what I can do is easily create an intermediate file using the
programming language I use every day, probably with very few lines of code.
> HTH. Please feel free to continue this conversation. It's
> interesting.
I hope there are more than just the three of us (Satheesh, me, maybe
Ken?) who can see the benefit in having a LeoBridge II-like capability. :D
> To see how to create entire tree from e.g. imported data, see
> implementation of createOutlineFromCacheList.
I've seen it. I've studied it as best I can. WAY over my head.
> Check source code of leoremote.py for an example. Terry also did a
> bunch of examples, perhaps he should push them to contrib branch.
I've looked into using leoremote.py, leobridge.py and ipython.py. It
appears these are only usable with another Python program.
I've yet to see an example of non-Python integration to Leo.
> I would just generate an input file (xml or "objtree" syntax used by
> the cacher, see treecaching.txt), and use leoremote to import it.
I'm looking for something simpler than an intermediate XML file because
I don't have tools to generate XML from the different data sources I
want to import into Leo. Treecaching.txt was an interesting read, but
WAY over my head.
