Re: How good Leo handles settings?

[vitalije]

On Thursday, June 29, 2017 at 4:16:38 PM UTC+2, Terry Brown wrote:
>
> Hey Vitalije, what are you working on exactly?  Not wanting to 
> duplicate any effort and wondering if you're working on the sqlite 
> settings idea or something else, and if you have any thoughts on the 
> sqlite approach separating file specific settings from the file they 
> apply to, vs. the current system's inclusion of those settings in the 
> same file? 
>
> Cheers -Terr 
>

I am sorry for the delayed answer. I worked last night and came to the 
state where I have a script that can turn any open Leo document into sqlite 
database with the same name except the extension (for example: LeoPyRef.leo 
turns into LeoPyRef.db ). I have changed the isLeoFile so that it now 
accepts extensions .leo, .db and zip files. I have changed openLeoOrZipFile 
so that it checks if it is a .db file that needs to be opened and if so it 
opens a sqlite connection and returns it instead of open file. Returned 
value from this method goes unchanged through some helper methods and 
finally arrives in readSaxFile. There I have simulated the parsing of xml 
nodes to tree of SaxNodeClass objects and switched the call to 
fc.parse_leo_file which returns SaxNodeClass root node with the call to 
retrieve the tree of SaxNodeClass nodes from database. It allowed normal 
start of Leo and opening LeoPyRef.db as it were ordinary LeoPyRef.leo. 
Later I have discovered that it would be better to intercept the normal 
reading procedure not in readSaxFile but higher up in call chain in method 
getLeoFileHelper. Because with the first variant the values stored in 
database were lower level so that they represent tree of SaxNodeClass nodes 
which would then be transformed into v-nodes. Instead it is easier to put 
v-nodes directly in database and then to retrieve higher level v-nodes tree 
and set it to c.hiddenRootNode in getLeoFileHelper.

That was the end of my yesterday work. Today I couldn't touch it until now 
and now I am going to finish the prototype that is able to open any Leo 
file stored in sqlitedb and to read settings from the good old 
leoSettings.leo and myLeoSettings.leo files. 

Next I plan to make a script that would analyze all settings files and to 
put all settings into database table like in the previous experiment I did. 
The only thing missing in that experiment I believe was additional field in 
database table which should contain level of settings source. For example 
default value should be on level zero, leoSettings (if it is something 
different than default I am not sure) on level one, myLeoSettings on level 
2 and finally settings in leo document would have level 3. Then querying 
database about any settings value can be performed like this:

select value from settings where key=? order by level desc limit 1;
Which could be explained like give me the value of given key with the 
highest level.

The idea is that every Leo document would contain all settings defaults, 
leoSettings and myLeoSettings and also any additional settings for 
particular document. When user changes myLeoSettings settings, earlier 
created documents will have outdated values. In that case executing simple 
command would update all values inside db leo document.

To summarize:

   1. user can keep all present settings unchanged
   2. user still can change settings in myLeoSettings.leo
   3. every new created file would have those settings embedded
   4. user can choose to execute simple command to update settings in any 
   Leo db document or perhaps Leo could automatically check the modification 
   time of myLeoSettings.leo and offer a user to update settings if they are 
   outdated

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Thu, Jun 29, 2017 at 8:47 AM, Kent Tenney  wrote:

> >setting: long - primary key for settings table
> >name: text - setting name
> >body: text - text data for setting for docs. and @data settings
> >type: long - link to types table (@bool, @int, @data, etc.)
> >parent: long - link to settings table for hierarchy
> >outline: long - link to outline table
>

​Can you provide a link to the original message?  I don't see it.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

Hey Vitalije, what are you working on exactly?  Not wanting to
duplicate any effort and wondering if you're working on the sqlite
settings idea or something else, and if you have any thoughts on the
sqlite approach separating file specific settings from the file they
apply to, vs. the current system's inclusion of those settings in the
same file?

Cheers -Terr

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

On Thu, 29 Jun 2017 08:47:01 -0500
Kent Tenney  wrote:

> >setting: long - primary key for settings table
> >name: text - setting name
> >body: text - text data for setting for docs. and @data settings
> >type: long - link to types table (@bool, @int, @data, etc.)
> >parent: long - link to settings table for hierarchy
> >outline: long - link to outline table
> 
> Hows about a 'scope' field: one of
> default, global, user, curdir, fname

I think outline will cover that - if an outline is specified, the
setting applies to that outline.  If not, the search order for DB files
will be similar to that for (my)leoSettings.leo files.

But yes, the obvious implementation allows settings for files without
touching the file.  Trouble is I'm still not sure how to handle the
"file specific settings no longer in .leo file" issue - on the one hand
it seems no big deal that copying a .leo file to another system leaves
behind $HOME/myLeoSettings.leo, but having any settings that were in
the copied file (under the current system) not follow along seems like a
regression.

Cheers -Terry

> which are listed by increasing weight ...
> the 'fname' one would allow specifying settings for a file without
> touching it,
> which strikes me as kind of cool.
> 
> Maybe that's what is accomplished by the 'parent' and 'outline'
> fields, if so, please ignore this.
> 
> On Wed, Jun 28, 2017 at 3:55 PM, Terry Brown 
> wrote:
> 
> > On Wed, 28 Jun 2017 12:48:06 -0700 (PDT)
> > vitalije  wrote:
> >
> > > > ​I would be happy with that, provided that a) settings can be
> > > > organized
> > > >> and b) per-document (.leo file) settings are possible.​ ​ The
> > > >> best way of organizing anything is in a .leo file.​
> > > >
> > > As I wrote in the earlier post today, it is trivial in sqlite to
> > > open connection to several different databases (files). So it is
> > > easy to have one database file for settings machine wide, another
> > > one for settings per user, and yet another one for settings per
> > > folder (if I am not mistaken there used to be possible for user
> > > to define settings per project/folder, although I have never used
> > > that possibility). It is easy to make script to dump all relevant
> > > (leo, myLeo) settings-files in separate databases.
> >
> > The docs. on settings location are here:
> > http://leoeditor.com/customizing.html#id3
> >
> > not 100% sure but I assume `homeDir` is ~/.leo or $HOME\.leo and
> > `configDir` is .../leo/config/, and `localDir` is the folder
> > containing the .leo file.
> >
> > Hmm, I'd forgotten about the machine-name variation, although it
> > can be very useful when the same filesystem is accessed from
> > different hardware setups.
> >
> > Essential: 1a, 2b, 3
> > Consider dropping: 1b, 1c, 2a - really when would you use these?
> > Useful: 2c, 2d - 2d when you want .leo files in a folder to share
> >  settings
> >
> > So there's a question of when to use separate DB files vs. info. in
> > the DB to separate contexts.
> >
> > 1a and 2b would obviously be their own DB files, with a search that
> > fails in 2b being repeated in 1a, trusting for now that that won't
> > be a performance issue.
> >
> > 3 is trickier.  We're going to (ok, proposing to ;-) separate the
> > settings data from the rest of the .leo file(*).  Managing two files
> > side by side seems obnoxious, from the users point of view.  I
> > think it would be better to put the outline specific info. in the
> > same DB as used for 2b (e.g. ~/.leo/myLeoSettings.sqlite3).  So the
> > setting table might look like:
> >
> > setting: long - primary key for settings table
> > name: text - setting name
> > body: text - text data for setting for docs. and @data settings
> > type: long - link to types table (@bool, @int, @data, etc.)
> > parent: long - link to settings table for hierarchy
> > outline: long - link to outline table
> >
> > We'd start by looking at c.db to see how the outline table might
> > manage outline identities.
> >
> > (*) this separation is a high cost thing.  It means the user can't
> > just move the .leo file to another system and have that file's local
> > settings follow along.  This is consistent with myLeoSettings.leo
> > also not following along, but still seems like a regression.
> >
> > Which suggests that at least for file local settings we might want
> > to continue to support .leo file @settings trees.  Hmm.
> >
> > That leads to wanting to make .leo file @settings trees more
> > writable than they are now, which leads to the possibility of using
> > Leo outlines rather than sqlite as a storage format.  Before this
> > is interpreted as scraping the idea copletely, I'll point out the
> > leosax.py module which reads .leo files into a light weight dict
> > representation of an outline many many times faster than Leo's
> > regular read code.  It ignores external files, and is read only.
> >
> > Alternatives are
> 

Re: How good Leo handles settings?

[Kent Tenney]

>setting: long - primary key for settings table
>name: text - setting name
>body: text - text data for setting for docs. and @data settings
>type: long - link to types table (@bool, @int, @data, etc.)
>parent: long - link to settings table for hierarchy
>outline: long - link to outline table

Hows about a 'scope' field: one of
default, global, user, curdir, fname

which are listed by increasing weight ...
the 'fname' one would allow specifying settings for a file without touching
it,
which strikes me as kind of cool.

Maybe that's what is accomplished by the 'parent' and 'outline' fields, if
so, please ignore this.





On Wed, Jun 28, 2017 at 3:55 PM, Terry Brown  wrote:

> On Wed, 28 Jun 2017 12:48:06 -0700 (PDT)
> vitalije  wrote:
>
> > > ​I would be happy with that, provided that a) settings can be
> > > organized
> > >> and b) per-document (.leo file) settings are possible.​ ​ The best
> > >> way of organizing anything is in a .leo file.​
> > >
> > As I wrote in the earlier post today, it is trivial in sqlite to
> > open connection to several different databases (files). So it is easy
> > to have one database file for settings machine wide, another one for
> > settings per user, and yet another one for settings per folder (if I
> > am not mistaken there used to be possible for user to define settings
> > per project/folder, although I have never used that possibility). It
> > is easy to make script to dump all relevant (leo,
> > myLeo) settings-files in separate databases.
>
> The docs. on settings location are here:
> http://leoeditor.com/customizing.html#id3
>
> not 100% sure but I assume `homeDir` is ~/.leo or $HOME\.leo and
> `configDir` is .../leo/config/, and `localDir` is the folder containing
> the .leo file.
>
> Hmm, I'd forgotten about the machine-name variation, although it can be
> very useful when the same filesystem is accessed from different
> hardware setups.
>
> Essential: 1a, 2b, 3
> Consider dropping: 1b, 1c, 2a - really when would you use these?
> Useful: 2c, 2d - 2d when you want .leo files in a folder to share
>  settings
>
> So there's a question of when to use separate DB files vs. info. in the
> DB to separate contexts.
>
> 1a and 2b would obviously be their own DB files, with a search that
> fails in 2b being repeated in 1a, trusting for now that that won't be a
> performance issue.
>
> 3 is trickier.  We're going to (ok, proposing to ;-) separate the
> settings data from the rest of the .leo file(*).  Managing two files
> side by side seems obnoxious, from the users point of view.  I think it
> would be better to put the outline specific info. in the same DB as
> used for 2b (e.g. ~/.leo/myLeoSettings.sqlite3).  So the setting table
> might look like:
>
> setting: long - primary key for settings table
> name: text - setting name
> body: text - text data for setting for docs. and @data settings
> type: long - link to types table (@bool, @int, @data, etc.)
> parent: long - link to settings table for hierarchy
> outline: long - link to outline table
>
> We'd start by looking at c.db to see how the outline table might manage
> outline identities.
>
> (*) this separation is a high cost thing.  It means the user can't just
> move the .leo file to another system and have that file's local
> settings follow along.  This is consistent with myLeoSettings.leo also
> not following along, but still seems like a regression.
>
> Which suggests that at least for file local settings we might want to
> continue to support .leo file @settings trees.  Hmm.
>
> That leads to wanting to make .leo file @settings trees more writable
> than they are now, which leads to the possibility of using Leo outlines
> rather than sqlite as a storage format.  Before this is interpreted as
> scraping the idea completely, I'll point out the leosax.py module which
> reads .leo files into a light weight dict representation of an outline
> many many times faster than Leo's regular read code.  It ignores
> external files, and is read only.
>
> Alternatives are
>
> (a) keeping the .leo and settings DBs side by side,
> something.leo and something.leo.settings.sqlite?
> (b) embedding the sqlite data in a the .leo XML file -
> that seems like a horrible idea.
>
> So I think the question of the cost of separating file-local settings
> from the .leo file is an issue here.  It sort of runs contrary to the
> idea of XML, which should allow everything to be in one file.
>
> But anything that deviates from simple vanilla use of sqlite DB files
> starts to lose the simplicity gain.  Hmm.
>
> Cheers -Terry
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "leo-editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leo-editor+unsubscr...@googlegroups.com.
> To post to this group, send email to leo-editor@googlegroups.com.
> Visit this group at 

Re: How good Leo handles settings?

[Edward K. Ream]

On Thursday, June 29, 2017 at 5:43:08 AM UTC-5, Edward K. Ream wrote:

> Or Leo could cache leoSettings.leo and myLeoSettings.leo, pretty much as 
described in #511 .
> I've just added caching to the list of items to be handled by the renamed 
> #510: 
improve how Leo handles settings 
.

Actually, Leo reads settings files only once, no matter how many outlines 
are loaded.  Yes, It would be slightly faster to cache settings files 
*between* invocations of Leo, but this would be a fairly minor speed 
improvement.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Thursday, June 29, 2017 at 4:57:04 AM UTC-5, Edward K. Ream wrote:
>
> On Wednesday, June 28, 2017 at 11:53:42 AM UTC-5, vitalije wrote:
>

So Leo opens *two* documents (the settings files) before opening ekr.leo, 
> and then Leo reads ekr.leo twice.
>
> Is it really necessary? It is if we insist on present scheme for dealing 
>> with settings. OTOH if Leo document was database it could contain all 
>> settings in one place and even if we want to have settings per machine/per 
>> user/ per folder and per document it can be done with no effort. We simply 
>> need to open and attach several databases. Sqlite allows that out of the 
>> box.
>>
>
> Or Leo could cache leoSettings.leo and myLeoSettings.leo, pretty much as 
> described in #511 . 
> The cache could even be a sqlite db.
>

I've just added caching to the list of items to be handled by the renamed #510: 
improve how Leo handles settings 
.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wednesday, June 28, 2017 at 11:53:42 AM UTC-5, vitalije wrote:

So Leo actually reads four Leo documents before it starts opening your 
> document and it read it twice. 
>

When I open ekr.leo, I see the following:

reading settings in C:\leo.repo\leo-editor\leo\config\leoSettings.leo
reading settings in C:\Users\edreamleo\.leo\myLeoSettings.leo
reading settings in C:\Users\edreamleo\ekr.leo

So Leo opens *two* documents (the settings files) before opening ekr.leo, 
and then Leo reads ekr.leo twice.

Is it really necessary? It is if we insist on present scheme for dealing 
> with settings. OTOH if Leo document was database it could contain all 
> settings in one place and even if we want to have settings per machine/per 
> user/ per folder and per document it can be done with no effort. We simply 
> need to open and attach several databases. Sqlite allows that out of the 
> box.
>

Or Leo could cache leoSettings.leo and myLeoSettings.leo, pretty much as 
described in #511 . 
The cache could even be a sqlite db.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 6:09 PM, Edward K. Ream  wrote:

@Edward, I have another question. Where is the point where the actual
> reading of Leo document ends and data is transferred in the tree.
>
>  To be more concrete I have found in method getLeoFileHelper of
> FileCommands class the following lines:
>
> c.setRootVnode(v)
>
> fc.rootVnode = v
>
> I thought that is what I was looking for, but on my surprise
> c.setRootVnode is empty method. It does nothing.
>
> It looks like *both* the statements you show can be eliminated. Not only
> does c.setRootVnode do nothing, but fc.rootVnode is never used! I'll clean
> this up soon.
>

​Done at 259ae3e in master:

- Added a comment that readSaxFile sets c.hiddenRootNode.
- Replaced the two statements with a "pass".
- Removed the fc.rootVnode ivar entirely.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 2:48 PM, vitalije  wrote:

@Edward, I have another question. Where is the point where the actual
reading of Leo document ends and data is transferred in the tree.

 To be more concrete I have found in method getLeoFileHelper of
FileCommands class the following lines:

c.setRootVnode(v)

fc.rootVnode = v

I thought that is what I was looking for, but on my surprise c.setRootVnode
is empty method. It does nothing.

​You have found the spot. At this point, the sax code has set c.hiddenRootNode.


It looks like *both* the statements you show can be eliminated. Not only
does c.setRootVnode do nothing, but fc.rootVnode is never used! I'll clean
this up soon. My apologies for the confusion.

So, how c commander gets the result of loading Leo document?

​Using c.hiddenRootNode, set by the sax read code.​
​ ​
Here is c.rootPosition
​:

def rootPosition(self):

c = self
if c.hiddenRootNode.children:
v = c.hiddenRootNode.children[0]
return leoNodes.Position(v, childIndex=0, stack=None)
else:
return None

If I knew that my prototype will be finished very soon.

​Oh good. We can have a race :-)

I think I know how to make Leo reloadable, without changing Leo's core much.

The timing of this discussion is extremely lucky, as I have just spent lots
of time delving into caching. The plan is as follows:

1. A new *reload-leo* command will write every open Leo outline to a *cache
list*, using a variation of cacher.makeCacheList:

def makeCacheList(self, p):
return [
p.h, p.b, p.gnx,
[self.makeCacheList(p2) for p2 in p.children()]]

The command will also write leoSettings.leo and myLeoSettings.leo
to their own cache lists, if they are not already open.

2. The reload-leo will then do the following:

- set a g.app.reloading flag
- reload all Leo modules
- re-execute all of Leo's startup code, for each previously open outline,
including leoSettings.leo and myLeoSettings.leo.

If g.app.reloading is True, fc.getLeoFile will get *everything* from the
proper complete cache list via a single (recursive) call to
cacher.createOutlineFromCacheList.

Boom. No calls to sax. No need to reload external files. This should be
almost instantaneous.
Because all modules have already been reloaded, all newly-created data
reflect changes to, say, the VNode, Position or Commander classes.

There will likely be other tests against the g.app.reloading flag. The easy
way would be to have Leo re-parse the cached settings file. Later, the
settings themselves (the settings dicts) could be saved and restored.

It's conceivable that this could be done in just a few days...

*Summary*

Caching all open outlines, plus settings files, will allow Leo to
re-execute its *complete* startup logic after reloading all affected
modules. This will allow Leo to update *all* its data structures
*quickly *without
special cases. This is a fascinating possibility.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

On Wed, 28 Jun 2017 12:48:06 -0700 (PDT)
vitalije  wrote:

> > ​I would be happy with that, provided that a) settings can be
> > organized 
> >> and b) per-document (.leo file) settings are possible.​ ​ The best
> >> way of organizing anything is in a .leo file.​
> >
> As I wrote in the earlier post today, it is trivial in sqlite to
> open connection to several different databases (files). So it is easy
> to have one database file for settings machine wide, another one for
> settings per user, and yet another one for settings per folder (if I
> am not mistaken there used to be possible for user to define settings
> per project/folder, although I have never used that possibility). It
> is easy to make script to dump all relevant (leo,
> myLeo) settings-files in separate databases. 

The docs. on settings location are here:
http://leoeditor.com/customizing.html#id3

not 100% sure but I assume `homeDir` is ~/.leo or $HOME\.leo and
`configDir` is .../leo/config/, and `localDir` is the folder containing
the .leo file.

Hmm, I'd forgotten about the machine-name variation, although it can be
very useful when the same filesystem is accessed from different
hardware setups.

Essential: 1a, 2b, 3
Consider dropping: 1b, 1c, 2a - really when would you use these?
Useful: 2c, 2d - 2d when you want .leo files in a folder to share
 settings

So there's a question of when to use separate DB files vs. info. in the
DB to separate contexts.

1a and 2b would obviously be their own DB files, with a search that
fails in 2b being repeated in 1a, trusting for now that that won't be a
performance issue.

3 is trickier.  We're going to (ok, proposing to ;-) separate the
settings data from the rest of the .leo file(*).  Managing two files
side by side seems obnoxious, from the users point of view.  I think it
would be better to put the outline specific info. in the same DB as
used for 2b (e.g. ~/.leo/myLeoSettings.sqlite3).  So the setting table
might look like:

setting: long - primary key for settings table
name: text - setting name
body: text - text data for setting for docs. and @data settings
type: long - link to types table (@bool, @int, @data, etc.)
parent: long - link to settings table for hierarchy
outline: long - link to outline table

We'd start by looking at c.db to see how the outline table might manage
outline identities.

(*) this separation is a high cost thing.  It means the user can't just
move the .leo file to another system and have that file's local
settings follow along.  This is consistent with myLeoSettings.leo also
not following along, but still seems like a regression.

Which suggests that at least for file local settings we might want to
continue to support .leo file @settings trees.  Hmm.

That leads to wanting to make .leo file @settings trees more writable
than they are now, which leads to the possibility of using Leo outlines
rather than sqlite as a storage format.  Before this is interpreted as
scraping the idea completely, I'll point out the leosax.py module which
reads .leo files into a light weight dict representation of an outline
many many times faster than Leo's regular read code.  It ignores
external files, and is read only.

Alternatives are

(a) keeping the .leo and settings DBs side by side,
something.leo and something.leo.settings.sqlite?
(b) embedding the sqlite data in a the .leo XML file - 
that seems like a horrible idea.

So I think the question of the cost of separating file-local settings
from the .leo file is an issue here.  It sort of runs contrary to the
idea of XML, which should allow everything to be in one file.

But anything that deviates from simple vanilla use of sqlite DB files
starts to lose the simplicity gain.  Hmm.

Cheers -Terry


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[vitalije]

>
> ​I would be happy with that, provided that a) settings can be organized 
>> and b) per-document (.leo file) settings are possible.​ ​ The best way of 
>> organizing anything is in a .leo file.​
>
> As I wrote in the earlier post today, it is trivial in sqlite to open 
connection to several different databases (files). So it is easy to have 
one database file for settings machine wide, another one for settings per 
user, and yet another one for settings per folder (if I am not mistaken 
there used to be possible for user to define settings per project/folder, 
although I have never used that possibility). It is easy to make script to 
dump all relevant (leo, myLeo)settings-files in separate databases. 

@Edward, I have another question. Where is the point where the actual 
reading of Leo document ends and data is transferred in the tree. To be 
more concrete I have found in method getLeoFileHelper of FileCommands class 
the following lines:

c.setRootVnode(v)

fc.rootVnode = v

I thought that is what I was looking for, but on my surprise c.setRootVnode 
is empty method. It does nothing. 
So, how c commander gets the result of loading Leo document? If I knew that 
my prototype will be finished very soon. I have already made possible for 
launchLeo.py to accept '.db' as extension for Leo documents in sqlite 
format and it passes sqlite connection as theFile argument to 
getLeoFileHelper. It could be easily intercepted even before in 
LoadManager.openFileByName 
method and control transferred to the caller with complete new outline read 
from database. I just don't know where to transfer the tree restored from 
database and built in memory. 
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

On Wed, 28 Jun 2017 13:25:39 -0500
"Edward K. Ream"  wrote:

> On Wed, Jun 28, 2017 at 12:05 PM, Terry Brown 
> wrote:
> 
> > On Wed, 28 Jun 2017 06:39:41 -0700 (PDT)
> 
> The success of the current
> > ​ ​
> > code in the context of 1. is questionable, seeing new and even
> > experienced users struggle with settings management.
> 
> ​Imo few (none?) of those problems arise because Leo represents
> settings in .leo files.​

Well, no, I guess using .leo files for storage isn't a problem from the
user point of view.  I think having to edit an outline and use text
markup and decide which file to edit and how to express values
are challenges for users, depending on their backgrounds.  Using a DB
for setting storage would, to me anyway, make it much easier to write a
more friendly interface for updating settings.

> ​I object to changing the UI on the grounds that it would simplify
> code. I don't want people to have to change their settings files!
> Maybe a prototype will convince me that settings files are dumb
> idea.  Until then, I remain skeptical.

Of course.  I think simplifying code has value in its own right,
although it may not always be a priority.  And in this case simplifying
code may make it easier to simplify part of the user experience.
But degrading or even perturbing the user experience simply to simplify
code would not make sense.

> Incremental proposals
> would likely be easier to understand and evaluate.

Hmm, I suppose a settings DB could be made to shadow the existing
settings and then used as a basis to demo alternative settings
editing.

Cheers -Terry


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 12:05 PM, Terry Brown  wrote:

> On Wed, 28 Jun 2017 06:39:41 -0700 (PDT)
>

The success of the current
> ​ ​
> code in the context of 1. is questionable, seeing new and even
> experienced users struggle with settings management.
>

​Imo few (none?) of those problems arise because Leo represents settings in
.leo files.​



> 2. GUI wise Leo has discarded GUI's in the past, and now has two.  I
> think what you're saying is that setting management *must* remain
> something that can be done within Leo using the tree and body panes.
> I guess that's an understandable desire, although to date it doesn't
> help resolve the settings management challenges.
>

​Yes, I think that's what I meant ;-)​


I'm not sure if you meant to apply the scripting api clause to this
> discussion, but it doesn't seem relevant.  There would be no change to
> the signature of g|c.config().


​Good.  I wasn't sure about this point.
​


> > For example, yesterday I discovered a settings table that inited
> > several commander ivars.  My first thought was, maybe this table
> > could go away. But no.  These ivars appear all over Leo.  It would be
> > horrible to use c.config.getInt or c.config.getBool in their stead.
>
> If you're referring to ivarsData, I don't think that's relevant - the
> issue of objects "caching" config. vars. as ivars is orthogonal to the
> proposed alternative g|c.config() implementation.
>

​I was referring to ivarsData.  ​I didn't mean to imply that this was
directly relevant to the discussion, although Vitalije's suggestion to
replace these ivars with python properties is interesting.

>
> I think a confusing part of this discussion is that it's unclear
> whether it's about simplifying code or improving the settings ui.  I
> think it's both, i.e. a proposal to use a DB as a key value store that
> would hugely simplify loading of settings, and also reduce the
> technical barriers to implementing a less challenging settings UI.
>

​I object to changing the UI on the grounds that it would simplify code. I
don't want people to have to change their settings files!  Maybe a
prototype will convince me that settings files are dumb idea.  Until then,
I remain skeptical.
​


> But as long as there's a requirement for Leo settings to be editable as
> Leo outlines, the DB implementation is probably not useful.
>

​That's the rub.  But maybe I'll get really excited by the prototype.​


With respect to the settings struggles
> ​ [big snip] ​
> I think it's clear that the current settings
> management options will always be a barrier for a significant number of
> users.


​I don't have that sense, but if it's true I would welcome proposals to
simplify matters. A proposal that says, in effect, I have a great idea for
throwing away all of Leo's settings machinery is going to be a hard sell,
but possibly not impossible sell.  Incremental proposals would likely be
easier to understand and evaluate.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 11:01 AM, vitalije  wrote:

By transferring to ClojureScript I have discovered some very cool projects
>> that I could not imagine were possible at all. One of the coolest things is
>> writing reloadable code. I am amazed how cool it is to make small change in
>> your code and in a few milliseconds browser shows the effect of changed
>> code. It is impossible to express in words the feeling. Once you try it and
>> feel it for yourself you can never tolerate again any tool that requires
>> you to reload or to wait more than a second for the effect of your action
>> to be visible.
>>
> ​
I keep coming back to this in my thoughts. Making Leo reloadable would
instantly make Leo a killer app.

How to do a prototype in the least amount of time? The way forward is
murky. Yes, Python has imp.reload, but it's not enough to reload code. All
of Leo's objects must be reloaded.

Prototypes can sweep all kinds of inconvenient details under the rug.  All
hacks are allowed, including hacks to avoid reloading settings files,
plugins, whatever. But hacks are not enough.  A strategy is needed.

The situation is akin to the time before Leo's first breakthrough, namely
the Aha that I could use MORE as a prototype for Leo.  What kind of
prototype could steer us in the direction of fully reloadable python code?
What kind of clever scaffolding is needed?

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Offray Vladimir Luna Cárdenas]

Yes, Smalltalk has liveness build in the core experience since early
releases. Is something that you can not find almost anywhere the common
experience of programming and computing in general. Recently I was
reading about how this liveness impressed Steve Jobs and his team and
inspired much of the computing after that (despite GUI, paradoxically
hid the live coding idea). Here are the details:

https://www.quora.com/What-was-it-like-to-be-at-Xerox-PARC-when-Steve-Jobs-visited/answer/Alan-Kay-11

With Grafoscopio, I want to leave outlining for writing and exploratory
computing in data storytelling, but also use Pharo browsers and code
inspectors when fits the problem better. I want to stretch the outline
metaphor as much as I can, while recognizing other ways of dealing with
coding and user experience.

Cheers,

Offray

On 28/06/17 12:22, Terry Brown wrote:
> On Wed, 28 Jun 2017 12:10:27 -0500
> Offray Vladimir Luna Cárdenas  wrote:
>
>> Using Pharo Smalltalk I have experienced a similar feeling about a
>> direct live system where changes are reflected immediately without
>> any reload.
> Years ago someone showed me the Squeak Smalltalk environment.  It
> seemed that it had "liveness" built in from the lowest levels.  As in
> open an inspector / mutator dialog for the desktop background color
> object and watch it change dynamically as you change the value.  If
> Pharo's similar, I think it has more liveness intrinsically.
>
> But there is certainly potential for making some things more live in
> Leo.  We just have to work within PyQt etc. and how they work.
>
> Cheers -Terry
>


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 11:01 AM, vitalije  wrote:

Every ivar that caches settings can be turned into property that gets its
>> value from c.config.get<
>> ​something>.
>>
>
​Excellent idea. There is a performance penalty, but it is not likely
significant.

It wouldn't be so hard to even make a script that would search and find
> every occurrence in Leos code of such ivars and turned them into properties.
>
​
Yes.
​


> By transferring to ClojureScript I have discovered some very cool projects
> that I could not imagine were possible at all. One of the coolest things is
> writing reloadable code.
>


> I am amazed how cool it is to make small change in your code and in a few
> milliseconds browser shows the effect of changed code. It is impossible to
> express in words the feeling. Once you try it and feel it for yourself you
> can never tolerate again any tool that requires you to reload or to wait
> more than a second for the effect of your action to be visible. Maybe I
> become a bit spoiled after experiencing all that and now I am expecting
> from each tool I use to have such feature.
>

​Very cool.  This is purpose-level thinking.
​


> @Edward, you have mention the Aha moment when @clean file become possible.
> I had proposed that few years earlier but you didn't understand my idea. I
> don't know is it possible to find those messages on this list but I
> remember that I proposed for shadow files that were used to be kept in
> hidden folder as ordinary files that their content could be stored inside
> Leo document itself. That is precisely what @clean nodes do today. Isn't it?
>

​Close enough. The "diff" is between the external file (without sentinels)
and and the results of simulating writes from the @clean tree to two strings*,
with and without* sentinels.

Well if I were able to explain that idea better @clean feature could have
> been implemented few years earlier.
>

I see.  My biggest regret is not realizing @shadow was sound (ambiguities
don't permanently matter) before Bernhard Mulder died.  He may have thought
the theorem was trivial, so didn't bother to state and prove it.
​

> In Leo's site there is paragraph where Edward explains why he likes Python
> (here )
>
> Before using Python I never fully realized how difficult and dangerous
> memory allocation is in C++. Try doing:
>
> aList[i:j] = list(aString)
>
> in C. You will write about 20 lines of C code. Any error in this code will
> create a memory allocation crash or leak.
>
> Now I am trying to say the same thing about settings code. All settings
> could be present in just about two lines of code: sqlite3.connect(...) and
> conn.execute('select ...').fetchone().
>

​That would be great internally. Can you init​

​sqlite from leoSettings.leo, myLeoSettings.leo and the local .leo files?
​

> Surely, Leo has proven that it is possible to keep settings in the same
> file format in which Leo documents are kept. But it was achieved by writing
> a lot lines of code and that means lot of possible places for bugs to hide.
> It would not make a memory leak or hard crash, but nevertheless it can
> cause a lot of frustration to the user (especially to spoiled one) . What
> is more important it can prevent (and if I may say so it does prevent)
> achieving such a cool feature as reloadability. Who knows what other
> features could be discovered once the reloadability is achieved.
>

​Reloadability is a separate issue.​

​ Leo could cache settings so that reload time is minimized.​

>
> Present scheme for keeping settings allows hierarchical management. Well
> it does. But honestly speaking how big is that benefit? I for example don't
> remember when I touched myLeoSettings.leo file last time. But I do remember
> that it was painful every time I did. Being able to see my settings
> in hierarchical view isn't that useful if you must read a ton of
> documentation to figure out what is the proper way to set something.
>

​Leo has hundreds of settings.  I change them in myLeoSettings.leo or a
local .leo file all the time​

​for testing.  There have been few complaints about @bool, @int, etc. The
confusion comes from knowing which settings are in effect.

I suppose we could say that *all* settings come only from
myLeoSettings.leo.  This would mean a) copying settings from
leoSettings.leo and b) foregoing the possibility of per-file settings.
This does not seem attractive to me.
​

> If you are lucky and everything work as advertised you have to open
> settings file make change and then reopen all your documents.
>

​The full reload-settings command is supposed to do this without reloading
files.
​


> But if it doesn't work you have to go through the documentation to figure
> where you went wrong and why your new setting is not accepted. If you are
> very unlucky, there can be a bug somewhere that prevents your settings to
> take place and there would be a lot of combinations to try (as 

Re: How good Leo handles settings?

[Terry Brown]

On Wed, 28 Jun 2017 12:10:27 -0500
Offray Vladimir Luna Cárdenas  wrote:

> Using Pharo Smalltalk I have experienced a similar feeling about a
> direct live system where changes are reflected immediately without
> any reload.

Years ago someone showed me the Squeak Smalltalk environment.  It
seemed that it had "liveness" built in from the lowest levels.  As in
open an inspector / mutator dialog for the desktop background color
object and watch it change dynamically as you change the value.  If
Pharo's similar, I think it has more liveness intrinsically.

But there is certainly potential for making some things more live in
Leo.  We just have to work within PyQt etc. and how they work.

Cheers -Terry

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Offray Vladimir Luna Cárdenas]

Hi,

Using Pharo Smalltalk I have experienced a similar feeling about a
direct live system where changes are reflected immediately without any
reload. Also I have shared that dealing with traversing the settings
three could be cumbersome and you need to read a lot before having the
change you want, if everything goes well. I would like something like
using the minibuffer or some interactive console to make such changes
possible and direct and despite using Leo less (I'm focusing on using
and improving Grafoscopio [1]) I advocate the exploration of improving
liveness in the Leo environment, so Sqlite seems a good candidate for that.

[1] http://mutabit.com/grafoscopio/index.en.html

As I've told, Leo (tool and community) is an important source of
inspiration and all this criticisms are directed towards making it
better and not on authors or their pretty good engineering decisions in
general.

Cheers,

Offray

On 28/06/17 11:01, vitalije wrote:
>
>
> For example, yesterday I discovered a settings table that inited
> several commander ivars.  My first thought was, maybe this table
> could go away.  But no.  These ivars appear all over Leo.  It
> would be horrible to use c.config.getInt or c.config.getBool in
> their stead.  So this table creates a major convenience in Leo's
> code, even while it complicates the startup code and (eventually)
> will complicate the reload-settings command.  It a choice, and I
> think the tradeoff is a good one.
>
> Edward
>
>
> Every ivar that caches settings can be turned into property that gets
> its value from c.config.get. It wouldn't be so hard to even
> make a script that would search and find every occurrence in Leos code
> of such ivars and turned them into properties. That way Leo would have
> single source of settings data and every change in the source would be
> visible every where instantaneously. 
>
> Last few months I have been studying ClojureScript and experimenting
> with it. Maybe last 5 years or so, my programming was mostly oriented
> towards client scripting in web browsers. Until recently I was using
> almost exclusively  CoffeeScript for all programming tasks. By
> transferring to ClojureScript I have discovered some very cool
> projects that I could not imagine were possible at all. One of the
> coolest things is writing reloadable code. I am amazed how cool it is
> to make small change in your code and in a few milliseconds browser
> shows the effect of changed code. It is impossible to express in words
> the feeling. Once you try it and feel it for yourself you can never
> tolerate again any tool that requires you to reload or to wait more
> than a second for the effect of your action to be visible. Maybe I
> become a bit spoiled after experiencing all that and now I am
> expecting from each tool I use to have such feature. 
>
> For all my computer related work I use Leo as editor, as a database as
> scripting environment, for processing all sorts of files, for all
> types of scripting. And I am very grateful to Edward and whole
> community for all the work they all put in this project. I am no
> newbie in Leo world nor I am newbie in python or for that matter
> almost to any programming language from Z80 and 6502 assembly language
> to C, C++, Java, Scala, Groovy, Javascript ActionScript, ... up to
> Clojure and ClojureScript.
>
> @Edward, you have mention the Aha moment when @clean file become
> possible. I had proposed that few years earlier but you didn't
> understand my idea. I don't know is it possible to find those messages
> on this list but I remember that I proposed for shadow files that were
> used to be kept in hidden folder as ordinary files that their content
> could be stored inside Leo document itself. That is precisely what
> @clean nodes do today. Isn't it?
> Well if I were able to explain that idea better @clean feature could
> have been implemented few years earlier.
> But at that time my suggestion was refused and I didn't care too much
> because I work almost exclusively alone and I have never had problems
> with synchronizing files nor with the sentinel lines. 
>
> In Leo's site there is paragraph where Edward explains why he likes
> Python (here ) 
>
> Before using Python I never fully realized how difficult and dangerous
> memory allocation is in C++. Try doing:
>
> aList[i:j] = list(aString)
>
> in C. You will write about 20 lines of C code. Any error in this code
> will create a memory allocation crash or leak.
>
>
> Now I am trying to say the same thing about settings code. All
> settings could be present in just about two lines of code:
> sqlite3.connect(...) and conn.execute('select ...').fetchone().
> Surely, Leo has proven that it is possible to keep settings in the
> same file format in which Leo documents are kept. But it was achieved
> by writing a lot lines of code and that means lot of possible places
> for bugs to hide. It would not 

Re: How good Leo handles settings?

[Terry Brown]

On Wed, 28 Jun 2017 06:39:41 -0700 (PDT)
"Edward K. Ream"  wrote:

> 1. Clarity and simplicity of *purpose*.
> 2. Stability and simplicity of *user interface*, including gui and 
> scripting api.
> 3. Simplicity of *code*.

[snip]

> In this discussion, the first two priorities demand that Leo continue
> to use outlines to represent the hierarchy of settings. 

I don't really think that follows in any particularly clear manner.

1. is a general goal no one can argue with.  The success of the current
code in the context of 1. is questionable, seeing new and even
experienced users struggle with settings management.

2. GUI wise Leo has discarded GUI's in the past, and now has two.  I
think what you're saying is that setting management *must* remain
something that can be done within Leo using the tree and body panes.
I guess that's an understandable desire, although to date it doesn't
help resolve the settings management challenges.

I'm not sure if you meant to apply the scripting api clause to this
discussion, but it doesn't seem relevant.  There would be no change to
the signature of g|c.config().  If you mean that it would no longer be
possible to edit settings by editing settings .leo files with Leo's
API, that would be true, but I would argue that no one does that
anyway.

[snip]

> For example, yesterday I discovered a settings table that inited
> several commander ivars.  My first thought was, maybe this table
> could go away. But no.  These ivars appear all over Leo.  It would be
> horrible to use c.config.getInt or c.config.getBool in their stead.

If you're referring to ivarsData, I don't think that's relevant - the
issue of objects "caching" config. vars. as ivars is orthogonal to the
proposed alternative g|c.config() implementation.

I think a confusing part of this discussion is that it's unclear
whether it's about simplifying code or improving the settings ui.  I
think it's both, i.e. a proposal to use a DB as a key value store that
would hugely simplify loading of settings, and also reduce the
technical barriers to implementing a less challenging settings UI.

But as long as there's a requirement for Leo settings to be editable as
Leo outlines, the DB implementation is probably not useful.

With respect to the settings struggles I think it's important to
remember these numbers, which I'm about to make up:

0% - percent of Leo users whose primary use of Leo is developing Leo,
rounded to the nearest percent ;-)

10%?, 20%?, 50%? - percent of Leo users whose primary use of Leo is
developing in Python (I have no idea what this number is)

10%?, 20%?, 50%? - percent of Leo users who do not use Leo for coding
at all (I have no idea what this number is either)

Given those numbers, I think it's clear that the current settings
management options will always be a barrier for a significant number of
users.  Even, hard though it is to believe, with cff :-)

Cheers -Terry



-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[vitalije]

Someone asked how this proposition can help users. I wasn't clear about 
that. It won't help that much unless reloading is also addressed. But there 
are some settings that aren't so obvious when represented as a text. 
Colors, shortcuts for example. Now it is required that a user writes them 
as a text and there is possibility of spelling mistakes. Add to it need to 
open another Leo file, edit settings, save it, close it and then reopen 
your document again to accept new changes. Then you realize that the color 
is not so pleasing and you have to go again through the whole process. Very 
soon I gave up adjusting colors. I think that settings like colors, fonts, 
shortcuts and similar should not be edited as a text but as something that 
user can see and feel without understanding any particular syntax. They 
certainly can be kept as a text, but user should be allowed to edit them in 
more natural way.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[vitalije]

>
>
> For example, yesterday I discovered a settings table that inited several 
> commander ivars.  My first thought was, maybe this table could go away.  
> But no.  These ivars appear all over Leo.  It would be horrible to use 
> c.config.getInt or c.config.getBool in their stead.  So this table creates 
> a major convenience in Leo's code, even while it complicates the startup 
> code and (eventually) will complicate the reload-settings command.  It a 
> choice, and I think the tradeoff is a good one.
>
> Edward
>

Every ivar that caches settings can be turned into property that gets its 
value from c.config.get. It wouldn't be so hard to even make a 
script that would search and find every occurrence in Leos code of such 
ivars and turned them into properties. That way Leo would have single 
source of settings data and every change in the source would be visible 
every where instantaneously. 

Last few months I have been studying ClojureScript and experimenting with 
it. Maybe last 5 years or so, my programming was mostly oriented towards 
client scripting in web browsers. Until recently I was using almost 
exclusively  CoffeeScript for all programming tasks. By transferring to 
ClojureScript I have discovered some very cool projects that I could not 
imagine were possible at all. One of the coolest things is writing 
reloadable code. I am amazed how cool it is to make small change in your 
code and in a few milliseconds browser shows the effect of changed code. It 
is impossible to express in words the feeling. Once you try it and feel it 
for yourself you can never tolerate again any tool that requires you to 
reload or to wait more than a second for the effect of your action to be 
visible. Maybe I become a bit spoiled after experiencing all that and now I 
am expecting from each tool I use to have such feature. 

For all my computer related work I use Leo as editor, as a database as 
scripting environment, for processing all sorts of files, for all types of 
scripting. And I am very grateful to Edward and whole community for all the 
work they all put in this project. I am no newbie in Leo world nor I am 
newbie in python or for that matter almost to any programming language from 
Z80 and 6502 assembly language to C, C++, Java, Scala, Groovy, Javascript 
ActionScript, ... up to Clojure and ClojureScript.

@Edward, you have mention the Aha moment when @clean file become possible. 
I had proposed that few years earlier but you didn't understand my idea. I 
don't know is it possible to find those messages on this list but I 
remember that I proposed for shadow files that were used to be kept in 
hidden folder as ordinary files that their content could be stored inside 
Leo document itself. That is precisely what @clean nodes do today. Isn't it?
Well if I were able to explain that idea better @clean feature could have 
been implemented few years earlier.
But at that time my suggestion was refused and I didn't care too much 
because I work almost exclusively alone and I have never had problems with 
synchronizing files nor with the sentinel lines. 

In Leo's site there is paragraph where Edward explains why he likes Python (
here ) 

Before using Python I never fully realized how difficult and dangerous 
memory allocation is in C++. Try doing:

aList[i:j] = list(aString)

in C. You will write about 20 lines of C code. Any error in this code will 
create a memory allocation crash or leak.

Now I am trying to say the same thing about settings code. All settings 
could be present in just about two lines of code: sqlite3.connect(...) and 
conn.execute('select ...').fetchone(). Surely, Leo has proven that it is 
possible to keep settings in the same file format in which Leo documents 
are kept. But it was achieved by writing a lot lines of code and that means 
lot of possible places for bugs to hide. It would not make a memory leak or 
hard crash, but nevertheless it can cause a lot of frustration to the user 
(especially to spoiled one) . What is more important it can prevent (and if 
I may say so it does prevent) achieving such a cool feature as 
reloadability. Who knows what other features could be discovered once the 
reloadability is achieved. 

Present scheme for keeping settings allows hierarchical management. Well it 
does. But honestly speaking how big is that benefit? I for example don't 
remember when I touched myLeoSettings.leo file last time. But I do remember 
that it was painful every time I did. Being able to see my settings 
in hierarchical view isn't that useful if you must read a ton of 
documentation to figure out what is the proper way to set something. If you 
are lucky and everything work as advertised you have to open settings file 
make change and then reopen all your documents. But if it doesn't work you 
have to go through the documentation to figure where you went wrong and why 
your new setting is not accepted. If you are very 

Re: How good Leo handles settings?

[Edward K. Ream]

On Tuesday, June 27, 2017 at 12:24:24 PM UTC-5, Terry Brown wrote:

> On Tue, 27 Jun 2017 11:27:27 -0500 
> "Edward K. Ream"  wrote: 
>
 

> > ​How does that help new users?​ 
>
> That's the problem with this proposal, it's so overloaded with benefits 
> it's hard to keep them categorized ;-)  


> This doesn't help new users directly, it just simplifies Leo's code 
> base and init.  It does make it easier to write a @settings GUI 
> that's simpler for new users, but that's a separate benefit. 
>

I was up in the middle of the night thinking about this, and afterwords had 
a vivid dream about leadership.

In this post, I want to make sure that you, Terry, understand how and why I 
make the choices I do.

As project leader, I have several goals that are in tension:

- to encourage newbies to dive in and try new things.
- to encourage experimentation, improvement and simplification where 
possible.
- to keep Leo's user interface and scripting api as stable as possible.

But I do not want my tolerance for ideas to imply that I am willing to 
sacrifice my fundamental priorities for Leo:

1. Clarity and simplicity of *purpose*.
2. Stability and simplicity of *user interface*, including gui and 
scripting api.
3. Simplicity of *code*.

Sometimes, rarely, simplicity of purpose and code go together.  The 
Mulder-Ream update algorithm is the most important example.  Simplified 
code was an essential part of doing @clean.

Much more commonly, simplicity of purpose results in difficult, complex 
code behind the scenes. In that case, the *only *proper course is to *prefer 
clarity and simplicity of purpose over code simplicity*. It is naive to 
think that code complexity is a symptom that something is wrong.  I am 
willing to endure this misunderstanding among newbies, but I want you, 
Terry, to understand that code complexity is *not *a reliable guide to 
evaluating proposals.

In this discussion, the first two priorities demand that Leo continue to 
use outlines to represent the hierarchy of settings. Yes, the code is 
complex. However, it mirrors inherent, *unexpected-but-unavoidable*, 
behind-the-scenes complexity. This is *not* going to change, even if, 
heaven forbid, someone decides to rewrite the startup code.

This a "teachable moment" relating to other topics:

1. From very early days, simplicity of purpose demanded that Leo's core be 
independent of gui. This creates a small amount of complexity in the core 
itself, and more code complexity in wrapper classes of various kinds.  

*Without this code complexity it would have not been possible to create the 
npyscreen gui.*Early on, I often muttered to myself about the npyscreen 
code. The phrase that kept running through my head was 'a twisty little 
maze of rooms, all alike'.  But I didn't waste *any *time complaining to 
the npyscreen people about their code. The npyscreen folks are *never *going 
to change their code. *They can't*, because it would break their users' 
existing applications. Creators of npyscreen apps must suck up the 
complexity and carry on.

2. Many years ago now, Leo moved to the "one node world" in which all data 
was held in vnodes, rather than a combination of (extinct) tnodes and 
vnodes.  An essential part of the project was a transition plan so that 
existing Leonine scripts would continue to work.  That transition went so 
well that only a few remnants of the old world exist.

A major Aha allowed the *same *vnode to be expanded/contracted *independently 
*in different positions of the outline. New, complex, code created the 
valuable *illusion* that cloned (v)nodes have independent expansion bits. 
Similarly, root @ nodes contain complex data that create the 
*illusion* that (v)nodes may be marked independently.

3. Leo's settings-handling code is remarkably complex, and sometimes buggy. 
But it would be a huge mistake to think that this complexity is for 
nothing, and that somehow newbies' first ideas about settings can be taken 
at face value.  I can not allow you, Terry, to make this mistake.

*Summary*

There is no Elysian realm in which all code complexities disappear. Simple 
user interface is an *illusion*, often created by very complex code. Using 
cff, git, pylint and pyflakes, devs can *discover* the complex, surprising, 
and unavoidable reasons why code is as it is.

For example, yesterday I discovered a settings table that inited several 
commander ivars.  My first thought was, maybe this table could go away.  
But no.  These ivars appear all over Leo.  It would be horrible to use 
c.config.getInt or c.config.getBool in their stead.  So this table creates 
a major convenience in Leo's code, even while it complicates the startup 
code and (eventually) will complicate the reload-settings command.  It a 
choice, and I think the tradeoff is a good one.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop 

Re: How good Leo handles settings?

[Terry Brown]

On Tue, 27 Jun 2017 11:27:27 -0500
"Edward K. Ream"  wrote:

> > But a large part of the proposal's benefit is in the non-UI
> > simplification of @settings loading.
> 
> ​How does that help new users?​

That's the problem with this proposal, it's so overloaded with benefits
it's hard to keep them categorized ;-)

This doesn't help new users directly, it just simplifies Leo's code
base and init.  It does make it easier to write a @settings GUI
that's simpler for new users, but that's a separate benefit.

Cheers -Terry

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Tue, Jun 27, 2017 at 9:53 AM, Terry Brown  wrote:

But a large part of the proposal's benefit is in the non-UI
> simplification of @settings loading.
>

​How does that help new users?​

import leo.core.leoGlobals as g
> config = g.config_for_outline("/path/to/somefile.leo")
>
> could create `config` which would be c.config before c is created.  And
> without loading somefile.leo or mySettings.leo or any other Leo
> outline.  The path to file parameter would only be used to get the
> outline's identity, needed for outline specific settings.  The best way
> to represent outline identity is still unclear, start from what c.db
> does I guess, but that's minor.
>

​An interesting idea.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

On Tue, 27 Jun 2017 06:54:30 -0700 (PDT)
"Edward K. Ream"  wrote:

> On Tuesday, June 27, 2017 at 7:58:02 AM UTC-5, Terry Brown wrote:
> 
> My interpretation of the proposal was a replacement of all @settings 
> > trees, not simply caching.
> 
> That's a much harder sell, but I won't veto it until I know a lot
> more.

Ok - thought you'd already said there was no way of changing your mind
on this ;-)

> I guess a demo project could demonstrate the possible user interface 
> 
> A very hard sell. We already have a working user interface for
> settings that supports of all of Leo's features, including cff,
> clones, scripting API (generators, p.b, p.h, p.gnx, etc.).

Which are mostly tools and concepts of no help to new users, who are
most likely to want to be changing settings.
 
> Let's start with the easiest, quickest prototype that could possibly
> work. This will typically *not* involve programming.  A brief
> description along with some sketches would be a quicker and more
> informative.  That way, everyone, including myself, could more
> clearly understand the potential benefits.

I agree, wireframes (https://en.wikipedia.org/wiki/Website_wireframe)
are a good place to start UI wise.

But a large part of the proposal's benefit is in the non-UI
simplification of @settings loading.

import leo.core.leoGlobals as g
config = g.config_for_outline("/path/to/somefile.leo")

could create `config` which would be c.config before c is created.  And
without loading somefile.leo or mySettings.leo or any other Leo
outline.  The path to file parameter would only be used to get the
outline's identity, needed for outline specific settings.  The best way
to represent outline identity is still unclear, start from what c.db
does I guess, but that's minor.

Cheers -Terry

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Tuesday, June 27, 2017 at 7:58:02 AM UTC-5, Terry Brown wrote:

My interpretation of the proposal was a replacement of all @settings 
> trees, not simply caching.


That's a much harder sell, but I won't veto it until I know a lot more.

Sqlite is a standard Python library, so no installation issue. 
>

Oh.  I didn't know that.
 

> DB's can represent hierarchy, so that could be handled too.


I'm not so interested in the back room.

I guess a demo project could demonstrate the possible user interface 


A very hard sell. We already have a working user interface for settings 
that supports of all of Leo's features, including cff, clones, scripting 
API (generators, p.b, p.h, p.gnx, etc.).

Let's start with the easiest, quickest prototype that could possibly work. 
This will typically *not* involve programming.  A brief description along 
with some sketches would be a quicker and more informative.  That way, 
everyone, including myself, could more clearly understand the potential 
benefits.

Edward 

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

On Tue, 27 Jun 2017 05:33:15 -0500
"Edward K. Ream"  wrote:

> > On Mon, Jun 26, 2017 at 11:21 AM, vitalije 
> > wrote: 
> > What would be the benefits if Leo configuration code use database to
> > store and retrieve settings​[?]​
> 
> ​An interesting question. As I understand you, you are, in effect,
> suggesting that Leo cache the contents of leoSettings.leo and
> myLeoSettings.leo.  That has the potential to speed the loading of
> Leo significantly.

My interpretation of the proposal was a replacement of all @settings
trees, not simply caching.  I'm not sure how much of Leo's startup time
is used processing settings, but I'm guessing it's not enough to
warrant additional complexity by adding caching.

Sqlite is a standard Python library, so no installation issue.
DB's can represent hierarchy, so that could be handled too.  I think
the code simplification alone would be valuable, any startup time
decrease is just an added benefit.

But if it's vetoed, it's vetoed.  I guess a demo project could
demonstrate the possible user interface and API (although post init.
API wise I don't think there'd be much to demonstrate, as little would
change).

Cheers -Terry

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Mon, Jun 26, 2017 at 12:09 PM, Terry Brown  wrote:

> On Mon, 26 Jun 2017 09:21:46 -0700 (PDT)
> vitalije  wrote:
>
> > As I have mention in other thread I have some ideas about how can be
> > improved Leo code that deals with the configuration and user settings.
>
> Very interesting analysis.  I've had similar experiences in the init.
> code, that is, for example, why the QDock interface to replace
> free_layout is kind of stalled.
>

​I'll be happy to help you init your QDock code :-)

I'm actually pretty happy with Leo's init code.  The complexity could be
said to arise from the "has a" pattern of object relationships rather than
the "is a" pattern.

That is, all (or almost all) of Leo's objects *contain* other objects.
They do not *inherit *other object classes. This was a conscious choice,
and I have never regretted it, no matter what the problems.  The
alternative is endless worries that ivars of one class conflict with ivars
of other classes.  Python's _ and __ conventions do not perfectly solve the
problem.

In cases where Leo classes are subclasses of non-Leo classes, Leo uniformly
(I think) uses the "leo_" prefix to ensure that Leonine ivars do not
conflict with ivars of the base class.

Seems like you've proved a SQLite based config. store is possible, and
> the idea of having an editor for that outside of Leo is interesting -
> it wouldn't be hard to code a PyQt editor but using a web front end as
> you suggest would work in PyQt free environments - with the lynx
> browser it could even work in a console only setting.
>

​I think a key/value cache is conceivable.  But it must merge seamlessly
with Leo as it is now.  I'm not going to allow massive changes to Leo's
startup code, or abandon myLeoSettings.leo and leoSettings.leo.  There is
no chance of changing my mind about this.​


On the other hand it must also be true that Python and Leo are up to the
> task of managing config. - the current system works, it's only
> drawbacks are the init. complexity you've mentioned.  And the
> difficulty people have in changing settings.
>

​These difficulties arise either from bugs (fairly easy to fix) or from the
inherent complexities of individual settings.
​


> What you have maybe not covered (yet) is outline specific settings.  I
> think an external settings store could manage these by mapping some
> outline identity to DB content.


​Imo, caching myLeoSettings.leo and leoSettings.leo is enough.
​


> And also that managing settings in Leo allows hierarchical management,
> which has advantages.
>

​These advantages must remain.​


We will see what Edward thinks, but the opportunity to toss a lot of
> c.config init. complexity might be tempting.
>

​Init complexity is inherent, as is read complexity. This is not ever going
to change.

Caching might improve performance, but it will *increase*, not decrease
code complexity.

The general principle is straightforward.  Simple user interfaces may imply
lots of complexity behind the scenes.  MacOS code is hugely more complex
than DOS code.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Mon, Jun 26, 2017 at 11:21 AM, vitalije  wrote:


> What would be the benefits if Leo configuration code use database to store
> and retrieve settings
> ​[?]​
>

​An interesting question. As I understand you, you are, in effect,
suggesting that Leo cache the contents of leoSettings.leo and
myLeoSettings.leo.  That has the potential to speed the loading of Leo
significantly.


> I believe that Leo startup code could be greatly simplified if the
> settings are available just by opening a database connection.
>

​Let's be absolutely clear about the following:

1. Caching *never* simplifies code.  It always adds more complexity.

That complexity can be justified, as it is in Leo's atFile read code​. I've
just spent a week improving that code.

It may well be that caching myLeoSettings.leo and LeoSettings.leo could
significantly speed up Leo's startup time.

2. I shall not approve of any scheme that changes the *users' *view of
Leo's settings. That is, myLeoSettings.leo and leoSettings.leo must
remain *exactly
*as they are.

3. Using sqlite is a dubious choice because it would add another
installation requirement to Leo (or your plugin).  Instead, you can
probably use the PickleShareDB class in leoCache.py to maintain key/values
pairs.

I would suggest doing your experiments in a plugin if at all possible.


> There is no need to postpone any initialization because settings haven't
> been fully read yet. I have tried several times to understand what exactly
> is going on during initialization but I gave up every time because it was
> nightmare of indirect calls, halfway initialization,
>

​Init code is inherently difficult, because init order matters a lot.  Imo,
there is no simple solution to this problem.

The basic init pattern is relatively simple.  I learned it from careful
study of Apple's YellowBox code.  The finishCreate methods complete inits
that could not be completed earlier because not all data, and *especially
not all objects*, exist.

... Almost all important members are initialized twice or more times. Once
> with the simple or dummy value, and later reinitialized with the correct
> one. Because of that there are everywhere checks if some dict is properly
> initialized or not. Arguably all that stuff could be written much simpler
> if c.config was present at the very beginning of the execution process.
>

​It's the *objects *that don't exist, not just settings.​

​Defining settings early doesn't help all that much.​


Attached to this message is sqlite3-config-benchmark.leo. Executing the
> script in its first node, performs all the steps I wrote about and prints
> in Log pane times for retrieving value of every setting 1000 times. On my
> computer it take about 6.5 s for c.config and 5.5 s for sqlite3 to give all
> those values.
>

​As I said earlier, caching settings may be a reasonable thing to do.​



> I am not so good in programming PyQt gui, but if all settings are kept in
> sqlite database, I could easily make single page web application and
> minimal REST server for accessing/adjusting all settings in web browser.
>

​Interesting.  This scheme could coexist with myLeoSettings.leo. Or rather,
it *must* coexist with myLeoSettings.leo
​


> Leo would need just one simple command to open local url in users default
> browser and user would be able to tweak any and all of the settings in
> browser.
>

​That would be good, provided that the server would update
myLeoSettings.leo.
​


> Server could be implemented in python and may actually run in the same Leo
> process or in other separate process. Leo would be aware of any such change
> in settings immediately.
>

​Good.
​


> It would not be necessary to restart Leo to see changes. Maybe some of the
> changes would require restart, but most of them IMHO would not.
>

​At present, many settings are cached in object ivars, so changing what
c.config.getX returns may not make Leo fully "aware" of the new settings. #417:
Complete reload-settings
 is intended to fix
this problem.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

On Mon, 26 Jun 2017 09:21:46 -0700 (PDT)
vitalije  wrote:

> As I have mention in other thread I have some ideas about how can be 
> improved Leo code that deals with the configuration and user settings.

Very interesting analysis.  I've had similar experiences in the init.
code, that is, for example, why the QDock interface to replace
free_layout is kind of stalled.

Seems like you've proved a SQLite based config. store is possible, and
the idea of having an editor for that outside of Leo is interesting -
it wouldn't be hard to code a PyQt editor but using a web front end as
you suggest would work in PyQt free environments - with the lynx
browser it could even work in a console only setting.

On the other hand it must also be true that Python and Leo are up to the
task of managing config. - the current system works, it's only
drawbacks are the init. complexity you've mentioned.  And the
difficulty people have in changing settings.

What you have maybe not covered (yet) is outline specific settings.  I
think an external settings store could manage these by mapping some
outline identity to DB content.  The only concern there is that that's
what c.db does currently and it seems more fragile than it should be -
certainly more fragile than @settings in the Leo file for a
particular outline.  I think it uses a hash of the filename to store
values for a particular outline, but (a) that breaks with different
paths that evaluate to the same thing, and (b) I think the code
generates different hashes in 2 and 3, although these are things that
could be fixed.

I was wondering about the root node gnx, but looking at my main outline
now I see tbrown.20170418151203.1 and I'm sure it's older than
2017-04-18.

And also that managing settings in Leo allows hierarchical management,
which has advantages.

We will see what Edward thinks, but the opportunity to toss a lot of
c.config init. complexity might be tempting.

Cheers -Terry

> Using small python script I have collected all settings from 
> leoSettings.leo and created a dictionary whose keys are types of
> settings (int, ratio, float, string, color, bool and data) and values
> are lists of setting names.
> 
> Then I made list of all possible pairs (type, setting-name) and for
> each pair I put the current value in sqlite3 database table.
> 
> For each pair I have checked that the value returned by corresponding 
> c.config.get(name) method is the same one that is returned by
> sql query for given settings name.
> 
> Then I have tested using timeit module, both methods of reading
> settings and the result is that using sqlite to retrieve settings
> value is actually faster then using plain c.config module. Speed gain
> is small, just about 12.4%, but at least experiment proved that using
> sqlite for dealing with configuration is possible. 
> 
> What would be the benefits if Leo configuration code use database to
> store and retrieve settings. I believe that Leo startup code could be
> greatly simplified if the settings are available just by opening a
> database connection. There is no need to postpone any initialization
> because settings haven't been fully read yet. I have tried several
> times to understand what exactly is going on during initialization
> but I gave up every time because it was nightmare of indirect calls,
> halfway initialization, ... Almost all important members are
> initialized twice or more times. Once with the simple or dummy value,
> and later reinitialized with the correct one. Because of that there
> are everywhere checks if some dict is properly initialized or not.
> Arguably all that stuff could be written much simpler if c.config was
> present at the very beginning of the execution process. 
> 
> Attached to this message is sqlite3-config-benchmark.leo. Executing
> the script in its first node, performs all the steps I wrote about
> and prints in Log pane times for retrieving value of every setting
> 1000 times. On my computer it take about 6.5 s for c.config and 5.5 s
> for sqlite3 to give all those values. 
> 
> I am not so good in programming PyQt gui, but if all settings are
> kept in sqlite database, I could easily make single page web
> application and minimal REST server for accessing/adjusting all
> settings in web browser. Leo would need just one simple command to
> open local url in users default browser and user would be able to
> tweak any and all of the settings in browser. Server could be
> implemented in python and may actually run in the same Leo process or
> in other separate process. Leo would be aware of any such change in
> settings immediately. It would not be necessary to restart Leo to see
> changes. Maybe some of the changes would require restart, but most of
> them IMHO would not. Vitalije
> 

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

How good Leo handles settings?

[vitalije]

As I have mention in other thread I have some ideas about how can be 
improved Leo code that deals with the configuration and user settings.

Using small python script I have collected all settings from 
leoSettings.leo and created a dictionary whose keys are types of settings 
(int, ratio, float, string, color, bool and data) and values are lists of 
setting names.

Then I made list of all possible pairs (type, setting-name) and for each 
pair I put the current value in sqlite3 database table.

For each pair I have checked that the value returned by corresponding 
c.config.get(name) method is the same one that is returned by sql 
query for given settings name.

Then I have tested using timeit module, both methods of reading settings 
and the result is that using sqlite to retrieve settings value is actually 
faster then using plain c.config module. Speed gain is small, just about 
12.4%, but at least experiment proved that using sqlite for dealing with 
configuration is possible. 

What would be the benefits if Leo configuration code use database to store 
and retrieve settings. I believe that Leo startup code could be greatly 
simplified if the settings are available just by opening a database 
connection. There is no need to postpone any initialization because 
settings haven't been fully read yet. I have tried several times to 
understand what exactly is going on during initialization but I gave up 
every time because it was nightmare of indirect calls, halfway 
initialization, ... Almost all important members are initialized twice or 
more times. Once with the simple or dummy value, and later reinitialized 
with the correct one. Because of that there are everywhere checks if some 
dict is properly initialized or not. Arguably all that stuff could be 
written much simpler if c.config was present at the very beginning of the 
execution process. 

Attached to this message is sqlite3-config-benchmark.leo. Executing the 
script in its first node, performs all the steps I wrote about and prints 
in Log pane times for retrieving value of every setting 1000 times. On my 
computer it take about 6.5 s for c.config and 5.5 s for sqlite3 to give all 
those values. 

I am not so good in programming PyQt gui, but if all settings are kept in 
sqlite database, I could easily make single page web application and 
minimal REST server for accessing/adjusting all settings in web browser. 
Leo would need just one simple command to open local url in users default 
browser and user would be able to tweak any and all of the settings in 
browser. Server could be implemented in python and may actually run in the 
same Leo process or in other separate process. Leo would be aware of any 
such change in settings immediately. It would not be necessary to restart 
Leo to see changes. Maybe some of the changes would require restart, but 
most of them IMHO would not.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.


sqlite3-config-benchmark.leo
Description: Binary data