Re: #561: Please explain why making Leo a jupyter client is important

[Jose Gonzalez]

Reading your responses I noticed my own mistake:

> - Separate code from outputs. One of the main strengths for Leo, 
combining code and results into the same document, is also one of its main 
limitations. ​

I meant " One of the main strengths for **Jupyter**, combining code and 
results into the same document, is also one of its main limitations."

I guess I was suggesting exploring the use of clones to allow saving 
vanilla code/documentation in one file and output for a particular 
execution (possibly with sentinels) in another file, intermingling them in 
the same "notebook" tree during edition/visualization time.

Presenting them together is convenient for delivering the notebook as a 
whole reproducible document, but storing them in the same file produces 
code fragmentation and other issues mentioned.

Thanks for your comments

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: #561: Please explain why making Leo a jupyter client is important

[Jose Gonzalez]

I think it's also about building on Jupyter concepts as reproducible 
literate programming platform, not just cloning its current functionality

See it as an alternative to the JupyterLab application 
(https://channel9.msdn.com/Events/PyData/Seattle2017/BRK11), reusing the 
same capabilities from Jupyter notebooks in terms of computing power 
(multilanguage kernels), document integration (markdown + code cells in the 
same entity), and ecosystem for all sort of data visualization (ipywidgets)

Things that "leopyter" could do better in my opinion would come from better 
underlying data structures: 
 - Hierarchical document structure would result in more organized notebooks 
(example https://app.notablemind.org/tutorial/)
 - Causality: Establishing dependencies between cells execution does not 
come out of the box in Jupyter, it requires extensions 
(https://github.com/stitchfix/nodebook or 
https://www.youtube.com/watch?v=5MmXs6m3JcE) . This is essential for 
 repetitivity and maybe leo graph structure could accommodate this 
 - Links: being able to navigate across multiple notebooks in the same leo 
document, which could be useful for keeping a knowledge base within a 
notebook repository
 - Separate code from outputs. One of the main strengths for Leo, combining 
code and results into the same document, is also one of its main 
limitations. The workflow for many people like me is to use the notebook 
first to "play with a problem" while documenting/visualizing results and 
then mature the code, isolating it to a function/library for broader use. 
Having results and code in the same json document makes this process a 
little bit more complicated, in many cases forcing the programmer to 
refactor the notebook to extract the code (e.g. 
http://opiateforthemass.es/articles/why-i-dont-like-jupyter-fka-ipython-notebook/)
 
Also, it you want to keep your code under version control, you need to 
somehow strip out the results before committing it (e.g. 
https://github.com/kynan/nbstripout).  I believe leo cloning and three way 
merging mechanism introduced for @clean could probably help in separating 
notebooks into different physical store files that get together in edition 
time
 - Metadata associated with nodes can facilitate multiple views of the same 
tree document. This is something I believe is in earlier stages of use for 
Jupyter: e.g. offering multiple views of the same document using cell 
annotations (e.g. cells as slides https://github.com/damianavila/RISE) or 
filtering information
- Broader experience injecting different data-formats into the document 
model (ReText, code, ...) than jupyter (with rare cases of use outside its 
json format, like https://github.com/rossant/ipymd)

On the other side, while I certainly like python and use it for many 
purposes, it has been a while since I gave up to the idea that python 
interfaces would be more solid than web UIs (javascript,...) And I think 
leo deep roots on python can be a factor slowing down the adoption of some 
of these technologies for the front-end. For literate programming 
interfaces there are things that modern javascript and web technologies do 
very well:
* Moving to cooperative environments is relatively easier since you can 
reuse much on the code in your desktop based application (e.g. 
https://github.com/nteract/nteract) to deploy a multiuser server 
application (https://github.com/nteract/commuter, https://cocalc.com/). 
* Html rendering technologies  really play well with advanced visualization 
for all sort of multimedia information, from maps 
(https://github.com/OpenGeoscience/geonotebook) to scientific data plots 
(https://plot.ly/python/getting-started/)
* Web technologies favored frameworks with clear separation of Model-View 
to facilitate content production (CSS, etc). This, although being 
incorporated to desktop technologies like QML, can facilitate offering 
multiple perspectives of the same document as mentioned before (mindmap, 
slides, kanban board, etc e.g https://jaredforsyth.com/treed/)  
* Having its roots in the web, javascript has very interesting capabilities 
for concurrency, asynchronous operation, etc. Its a good playground for 
technologies like functional programming, reactive interfaces, declarative 
interfaces, etc. And it keeps growing in popularity, with many available 
libraries and potential contributors

Fortunately, it seems python+javascript have the potential to coexist and 
complement each other well, with some examples like dash 
https://medium.com/@plotlygraphs/introducing-dash-5ecf7191b503

Just my 2 cents
   

On Friday, October 27, 2017 at 10:48:13 AM UTC-4, Edward K. Ream wrote:
>
>
>
> On Thu, Oct 26, 2017 at 8:51 AM, Terry Brown  > wrote:
>
>>
>> > Does this vision match yours? Here are some initial thoughts.
>>
>> My thoughts, which include a lot of groundless assumptions about how
>> Jupyter works, would be that we have a new node type:
>>
> ​[big snip]
>
>
> 

Re: sqlite-format and settings - ideas

[Jose Gonzalez]

I'm following this thread and new wave of ideas for leo with great attention. I 
really like the goal of making leo a more interactive platform for 
keeping/editing knowledge bases... Great job everybody!

Just my 2 cents:
Maybe sqllite is just a tool for a greater purpose, a step toward a greater 
goal. Let me explain myself:

IMO, generating diffs truly representative of the evolution of the node 
hierarchy in a leo document would require an API that explicitly models that 
hierarchal structure. 

That made me remember CoreObject. 
https://github.com/etoile/CoreObject
http://coreobject.org/technotes/

Explicitly modelling the hierarchical structure of a document and its mutations 
in time enables really useful applications, like reactive interfaces, 
cooperative tools and distributed edition of these documents 

And it's also based on SQLite :)
"
CoreObject is a pragmatic OODB with a homegrown DVCS, leveraging SQLite for 
transactional storage and indexing"

Too bad my objective C skills are not existant to be able to explore the 
concepts in their implementation and their applicability to leo

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: Time travel in Leo

[Jose Gonzalez]

A couple of links related to "versioning"  just in case they serve as an 
inspiration:

http://coreobject.org/technotes/ 
(https://www.youtube.com/watch?v=UmpaAunTEQ0)
https://github.com/mirage/irmin 
(http://roscidus.com/blog/blog/2015/04/28/cuekeeper-gitting-things-done-in-the-browser/)
 

This is what it came to my mind when the discussion about "multithread" 
graphs started but not till recently I found real implementations of that 
idea. Too bad none of them is written in python. 



On Wednesday, June 1, 2016 at 9:46:37 AM UTC-4, Kent Tenney wrote:
>
> Only tangentially relevant: 
>
> The versioning idea I had working briefly and am still aiming for, 
> I consider 'spatial' as apposed to a time travel notion. 
>
> I had buttons labeled 'Left', 'Right', 'Up', and 'Down' 
> The idea is the different versions are next to each other instead 
> of before and after. A database held each version. 
> This is at the node level. 
>
> The usage is like: 
> - I want to try a variation of the code in this node. 
>
> - I press 'Right' and, if there is nothing to the right of the current 
> node, 
>   a copy is made. If I'd previously created a node to the 'Right', it 
> is retrieved. 
>
> - edit, edit, test ... press 'Left' 
>
> - now I see the original node, on it's right is the variation. Same for 
> any 
>   number of locations in any direction. 
>
> - all versions exist in the DB backend, only one in the Leo file. 
>
> I have good intentions of implementing this in a more robust way than 
> the proof of concept I had working at one time. I found it very simple 
> and convenient during the bit of testing I did, don't know if it would 
> really hold up long term. 
>
> Thanks, 
> Kent 
>
> On Wed, Jun 1, 2016 at 7:45 AM, Edward K. Ream  > wrote: 
> > I'll be working on pyflakes for at least several more days. 
> > 
> > As I do so, I am struck by how important the clone-find-all (cff, cfa, 
> cffm) 
> > command are: they provide exactly the kind of search-related views that 
> Leo 
> > needs.  For me, cff is the workhorse.  To my knowledge, these commands 
> exist 
> > in no other environment. 
> > 
> > As I consider how to present my pyflakes work, it occurs to me that a 
> > relatively easy script might be almost as useful as the clone-find 
> commands. 
> > Kent has asked for a long time for node history.  The idea is to take 
> > history to the "limit", by allow a slideshow-like view of an entire Leo 
> > outline based on git commits.  Hit the back button: go back in time to 
> see 
> > what the *entire* file was like at the previous commit.  Hit the forward 
> > button, go forward in time. 
> > 
> > Imo, this is probably the only way to show what has really changed. 
>  Diffs 
> > are far too scattered.  Preferably, the time travel would not modify the 
> git 
> > repo.  Instead, it would be better to have the script do a "lightweight" 
> > computation of the various versions of the outline. 
> > 
> > We'll see how feasible this would be.  We must make it difficult to 
> revert 
> > to a previous time by accident... 
> > 
> > In short, I want a way of describing, for instance to the pyflakes 
> people, 
> > the steps I took getting from A to B.  Diffs have no chance of doing 
> what I 
> > want. A "movie" of how pyflakes_study.leo has changed seems like the 
> only 
> > way.  More generally, such a feature might be a killer feature, just 
> like 
> > cff. 
> > 
> > Edward 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "leo-editor" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to leo-editor+...@googlegroups.com . 
> > To post to this group, send email to leo-e...@googlegroups.com 
> . 
> > Visit this group at https://groups.google.com/group/leo-editor. 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: @auto x.json and @auto-json now work

[Jose Gonzalez]

A possibility could be to specify a schema for a given json file indicating how 
to structure the document. That would expand the possibilities for Leo to 
handle flat-file hierarchical datasets for applications like bookmarks manager, 
bibliographies, etc. A similar concept is implemented in 
http://treeline.bellz.org, I believe using (less human readable) XML files. 
Just an idea...

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: @auto x.json and @auto-json now work

[Jose Gonzalez]

A possibility could be to specify a schema for a given json file indicating how 
to structure the document. That would expand the possibilities for Leo to 
handle flat-file hierarchical datasets for applications like bookmarks manager, 
bibliographies, etc. A similar concept is implemented in 
http://treeline.bellz.org, I believe using (less human readable) XML files. 
Just an idea...

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: @auto x.json and @auto-json now work

[Jose Gonzalez]

A possibility could be to specify a schema for a given json file indicating how 
to structure the document. That would expand the possibilities for Leo to 
handle flat-file hierarchical datasets for applications like bookmarks manager, 
bibliographies, etc. A similar concept is implemented in 
http://treeline.bellz.org, I believe using (less human readable) XML files. 
Just an idea...

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: Import/export to Jupyter (IPython) notebooks

[Jose Gonzalez]

Also, there is a young project named nteract (github.com/nteract/nteract) that 
seems to be an interesting option as a lightweight and modular reusable 
alternative for rendering notebook cells in a web view.

BTW, the container application in this case is electron (using node.js), which 
uses libchromium. There are some Python bindings for this html renderer (e.g. 
https://github.com/cztomczak/cefpython), so it could be another option to 
explore for Leo.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: Import/export to Jupyter (IPython) notebooks

[Jose Gonzalez]

I tried a similar solution before using (https://github.com/damianavila/vIPer) 
and I don't remember it being that slow... I'll recheck again

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: Leo reimagined: a tree/dag/view for every attribute!!!

[Jose Gonzalez]

Could this "multi-graph" structure be a mechanism to implement version 
control in Leo? As you know, the fundamental data structure for Git and 
other version control systems is also a DAG. If, in addition to the 
existing tree representing the document structure, there is another tree 
defining the evolution in time of the document/each node, then Leo would be 
able to track the whole history of a project.

This could be a good foundation for collaborative document edition (merging 
modifications on the same Leo document by multiple users), which would make 
Leo even more powerful as a distributed tool for knowledge base management. 

If the functionality can be implemented in such a way that the new "history 
tree" can be fully mapped on top of an existing version control system such 
as git (i.e. mapping each edge -representing the transition from a 
particular version of a node to a newer one- to individual blobs in git 
commits), then Leo could probably reuse many of the existing tools/concepts 
from version control systems: diff visualization, synchronization between 
users (i.e. pull requests, remotes), tree management, etc

Just my 2 cents
Jose

On Thursday, February 11, 2016 at 6:12:20 PM UTC-5, Edward K. Ream wrote:
>
> John Lunzer's recent remarks has triggered one of the biggest Aha's in 
> Leo's history, one that was totally unexpected until this afternoon.
>
> Leo's fundamental data structure is a tree of nodes.  More accurately, it 
> is a Directed Acyclic Graph or dag.  Links in such a graph can point "down" 
> the tree, but never up, so there are no cycles.  Leo's clones are well 
> defined just exactly because there are no cycles.  Given a node, we know 
> precisely *and* intuitively which nodes are descendants. Because of 
> clones, a node may have more than one parent.
>
> Here's the Aha.  Imagine this data structure as being linked with *black* 
> thread.  Now imagine another *completely independent* set of *red* 
> threads.  The only requirement is that the links defined by the red threads 
> must *also* be a tree/dag. That is, there is *absolutely no* necessary 
> relationship between the red and black threads.
>
> Now imagine that every Leo attribute creates its *own* sets of 
> links/threads.  *We have a new view for every attribute.* Let's call such 
> a data structure a *multi-threaded tree/dag*. Trees might actually be 
> enough, but I'm not sure so I'll refer to tree/dag (tree or dag). 
> *Important*: some nodes may not have any thread running them for some 
> colored threads (attributes).  That's just fine.
>
> Alas, the term "multi-threaded data structure" is commonly used for data 
> structures that can be accessed by independently running threads. That's 
> not at all what I am talking about.
>
> I am not aware of anything like this in graph theory. Could this be a 
> truly new idea?  I really have no idea.
>
> Of course, you could call the resulting structure a general graph, and 
> technically it is, but how the data structure will be used inside Leo is 
> *very 
> *different. Instead of a single, unintuitive mess, we have a collection 
> of easily traversed trees/dags. *We instantly have an arbitrary 
> dimensioned graph*! Each color creates a new dimension.  Of course, we 
> only *see* one dimension at a time, but attributes may be the 
> *composition* of other attributes (unions, intersections or other 
> relations), so this is no real restriction.
>
> And listen to this.  *It is astoundingly easy to create this multi-dag in 
> Leo*.  At present, each vnode has a parents list and a children list.  
> All we have to do is replace these two lists by dictionaries whose keys are 
> attributes and whose values are the parents or children lists for those 
> attributes.  Finally, we add Python properties shield the dictionary lookup 
> from the rest of Leo.  We are talking about maybe 50 lines of code! As far 
> as the rest of Leo is concerned, vnodes *still* have parents and children 
> lists.  The switcheroo happens totally invisibly, when the user wants to 
> see another view.  The attribute changes and that changes what the vnode 
> dictionary/properties "deliver" to the rest of Leo. 
>
> To recap: What we see depends only on the attribute that defines the 
> view.  This attribute determines what parents and children lists become 
> active.  So easy.  So very powerful.
>
> Each *current view* will show only the nodes with the given attribute. To 
> repeat, attributes can be composed of other attributes, so in effect each 
> view can be associated with an *arbitrary set* of attributes.  This is an 
> *astoundingly* general data structure.
>
> The real challenge lies in assigning different colored links to nodes. I'm 
> still trying to wrap my thoughts around all of this.
>
> As always with truly huge Ahas, it's already impossible to know exactly 
> what thoughts lead to it.  As soon as the Aha happened, the world changed 
> so much that the past becomes hard to