from:"vitalije"

And the ENB is in another thread.

On Wednesday, December 8, 2021 at 9:21:15 PM UTC+1 vitalije wrote:

> I've just merged PR 2353 in the devel
>
> On Wednesday, December 8, 2021 at 9:17:16 PM UTC+1 Edward K. Ream wrote:
>
>> On Wed, Dec 8, 2021 at 2:14 PM vitalije  wrote:
>>
>>> Done at 117a3bb52.
>>>
>>> It adds support for new configuration value `@bool 
>>> put-class-in-imported-headlines`
>>> if this flag is True, then all imported class nodes have prepended 
>>> `class ` to their
>>> headline.
>>>
>>
>> Many thanks for this. I have just approved the PR.
>>
>> Edward
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/6446a29d-2c0d-45b4-a4a3-071528068563n%40googlegroups.com.

Re: For Vitalije: About unit tests for the python importer

I've just merged PR 2353 in the devel

On Wednesday, December 8, 2021 at 9:17:16 PM UTC+1 Edward K. Ream wrote:

> On Wed, Dec 8, 2021 at 2:14 PM vitalije  wrote:
>
>> Done at 117a3bb52.
>>
>> It adds support for new configuration value `@bool 
>> put-class-in-imported-headlines`
>> if this flag is True, then all imported class nodes have prepended `class 
>> ` to their
>> headline.
>>
>
> Many thanks for this. I have just approved the PR.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/eed76d01-cd2f-4918-8f97-b81591ebe62dn%40googlegroups.com.

ENB: About new python importer

The import has two distinct phases. In the first phase, necessary data is 
being calculated
and in the second phase, outline is built using calculated data.

First of all we have a list of source code lines as an input argument and 
it is
visible to all inner functions. These lines are tokenized using tokenize 
module
and a list of all tokens is created which is also available to all inner 
functions.
Then the tokens are separated in groups according to the line number where 
the
token originates from.

Then using all this lists a new list is created. For each definition in the 
file
(i.e. each function/method and each class) a tuple is created with the 
several
key values.



First value is starting column of this definition. Then follows the number 
of
the first line that should go in the node with this definition. For example
if we have decorator on the function, or some comments written just above 
the
function, those lines should be included in the node. In the given example 
it
is line number 163 which contains a decorator.

Then follows a number of the line where the declaration line ends. In the 
given
example, the argument list is long so it is divided in two lines. So, the
keyword def is on the line 164, but the colon is at the end of line 165.
When we later build body for this node, those lines should not be indented.
Their indentation in the file is the same as the indentation of the 
at-others
directive in the parent node.

Then we have two string values: kind (def or class) and the name of the 
function,
class or method which will used to set the node headline. The name is 
followed
by two indentation numbers. The first is the indentation of the function 
body.
The other is the longest common leading white space for the entire function 
body.
This value is the indentation of at-others that we would put in this node.
In case of under indented comments this value is less than the indentation 
of the
function body. When there are no under indented comments these two values 
are
equal.

The last element of this tuple is the line number of the first line that 
comes
after this node. In the example it is 185.

The getdefn function calculates these tuples for each definition in the 
file.
All these tuples are gathered in the list `definitions`.

Finally after the `definitions` list has been calculated, we can start to 
make
outline. The `mknode` function handles this task. It starts with filtering
the definitions list. Only the definitions that start on the certain column
are used to generate the first level children. For each child, this function
makes another selection of definitions which start on the same column as the
function body of this node. Then it calls mknode recursively if there are
inner definitions and if the total number of lines is higher than the 
certain
threshold. If there are no inner definitions, or if the total number of 
lines
is small, node body is set immediately and process continues with the next
sibling definition.

If there are some gap in between two consecutive definitions, a declarations
node is inserted containing missed lines.

I hope this explains well how function achieve its goals. As a help for 
hacking
function returns calculated list of the definitions. I've dumped this list 
many
times while tweaking and fixing issues with the import.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/8824f218-699f-40a6-9ce8-85c3eb79b2e4n%40googlegroups.com.

Re: For Vitalije: About unit tests for the python importer

Done at 117a3bb52.

It adds support for new configuration value `@bool 
put-class-in-imported-headlines`
if this flag is True, then all imported class nodes have prepended `class ` 
to their
headline.

On Wednesday, December 8, 2021 at 8:10:48 PM UTC+1 Edward K. Ream wrote:

> On Wednesday, December 8, 2021 at 10:52:38 AM UTC-6 vitalije wrote:
>
>> Today, I've completely rewritten split_root function. This was necessary 
>> to allow importing files from mypy repository.
>> Rev ee7ff9c51f8a7 successfully imports all python files from mypy 
>> repository except one. 
>>
>
> I've just imported all the files. The code looks excellent!!
>
> There is only one easy change I would like to see.  I would like headlines 
> for the top-level class node to start with 'class '.
>
> Everything else looks perfect.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/527aa62d-4d3d-4799-8b97-1048e23dc23fn%40googlegroups.com.

Re: For Vitalije: About unit tests for the python importer

Today, I've completely rewritten split_root function. This was necessary to 
allow importing files from mypy repository.
Rev ee7ff9c51f8a7 successfully imports all python files from mypy 
repository except one. One of the files (misc/upload-pypi.py) has a line in 
the comments that looks like section reference and Leo can't produce same 
output as is the input, because the section is missing.

Regarding your condition that new importer must use vnode_info, IMHO that 
would be just a make-work. There is no need for any post pass because new 
importer deals with decorators and comment lines as well as blank lines, 
and it adds at-language and at-tabwidth directives. 

I plan only to clean up a little this new version and to add some comments 
and explanations. In case you find some other input that new importer can't 
import, I'll try my best to fix the issue. But I don't plan to use 
vnode_info. If you really wish it, you can add it yourself. It would be 
easy to reverse engineer vnode_info from the resulting outline, but I have 
zero interest doing that.

I'll write an ENB on the subject when I finish cleaning and documenting 
code.
On Wednesday, December 8, 2021 at 3:56:08 PM UTC+1 Edward K. Ream wrote:

> On Wednesday, December 8, 2021 at 8:43:45 AM UTC-6 Edward K. Ream wrote:
>
> At rev 7b109931aa, the importer crashes for 81 mypy files.  There is no 
> indication elsewhere that the crash has occurred. There probably should be.
>
> All 81 failures have the same underlying cause, namely calls to 
> tokenenize.generate_tokens. There are two such calls in the new importer. 
> Depending on the source code, the calls raise either IndentationError or 
> tokenize.TokenError.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/20a6946f-d633-4305-bd93-cc3ae89d0721n%40googlegroups.com.

Re: For Vitalije: About unit tests for the python importer

2021-12-07 Thread vitalije

I've added more comments in the code and have created PR to merge importers 
into devel branch. 

Vitalije

On Tuesday, December 7, 2021 at 7:09:40 PM UTC+1 vitalije wrote:

> > A few more tests may be needed to get to 100% coverage.
>
> Done at 6282e76a62 in importers branch. Tests for python importer cover 
> 100% code, and all succeed.
> Vitalije
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/6e168440-8542-4965-bf36-8262ea40cbd5n%40googlegroups.com.

Re: For Vitalije: About unit tests for the python importer

2021-12-07 Thread vitalije

> A few more tests may be needed to get to 100% coverage.

Done at 6282e76a62 in importers branch. Tests for python importer cover 
100% code, and all succeed.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/5a564edf-69ac-467c-831e-353b3a852b49n%40googlegroups.com.

Re: For Vitalije: About unit tests for the python importer

2021-12-07 Thread vitalije

On Tuesday, December 7, 2021 at 9:03:33 AM UTC+1 Edward K. Ream wrote:

>
>  The assert* methods of unittest.TestCase provide better reports than 
> python's assert statement.
>  
>
Leo's existing test framework works well with the assert* methods, so the 
> conversion should be easy.
>
>
Have you tried using pytest as runner. The report it generates on a simple 
python assert statement with somewhat complex expressions is astonishing. 
Much better than any report TestCase.assert would produce. For example try 
this:
def test_dummy():
   a = [1, 2, ['a', 'b', 'f', 'd', 'e', 'c'], 4, 6]
   b = [1, 2, ['a', 'b', 'c', 'd', 'e', 'f'], 4, 5] 

   assert a == b

Running pytest -v on this function I get the following report:

def test_dummy():
a = [1, 2, ['a', 'b', 'f', 'd', 'e', 'c'], 4, 6]
b = [1, 2, ['a', 'b', 'c', 'd', 'e', 'f'], 4, 5]

>   assert a == b
E   AssertionError: assert [1, 2, ['a', ...', 'c'], 4, 6] == [1, 2, 
['a', ...', 'f'], 4, 5]
E At index 2 diff: ['a', 'b', 'f', 'd', 'e', 'c'] != ['a', 'b', 
'c', 'd', 'e', 'f']
E Full diff:
E - [1, 2, ['a', 'b', 'c', 'd', 'e', 'f'], 4, 5]
E ?^  ^   ^
E + [1, 2, ['a', 'b', 'f', 'd', 'e', 'c'], 4, 6]
E ?^  ^   ^

It makes it easy to spot where the difference exactly is without the need 
of writing special formatting code by hand. In another thread I've read 
that you don't like using pytest because it generates too much output. And 
yes I've tried to run all unit tests using pytest and there were huge 
amount of failures. But most of them were caused by an assertion inside Leo 
core, during the test setup phase. I think this is caused by executing 
`TestSyntax.test_that_leo_starts`. When I excluded this test with 
self.skipTest(), pytest finds exactly the same number of failures as 
unittest. In fact it has found one more. In test_leoImport.TestCython 
class, there was no at-others and the only method was not part of the 
class, so it never run.

Commit 2492a13f1 fixes both of this.

The methods compare_outlines and create_expected_outline rely on vnode_info 
which split_root doesn't populate.

> P.S. I see that the new python importer refers to linescanner.Target and 
> defines a Python_ScanState class. Eventually, we would like both to go 
> away. I'll experiment with making them optional.
>
>
There is another thing that bothers me with the present code. All importers 
are forced to export a scanner class, even if they don't use one. Those 
exported classes are gathered in g.app.atAutoDict and 
g.app.classDispatchDict. But then, later methods app.body_parser_for_ext, 
app.scanner_for_ext and app.scanner_for_at_auto each of them create a 
wrapper function that uses this importer class and returns it as a simple 
function.

By generating those wrappers in the leoApp, unnecessary coupling exists 
between importer modules and LeoApp. LeoApp must now about internal affairs 
of importer and also importers must create class and even override some 
unused methods even if they don't need a class. It would be much cleaner if 
importers export a function and not a class. That would much simpler (in 
fact a minimal) API. Every importer must export just one function f(p, txt) 
which should create outline from given text in the given position. How each 
of the importers achieve this is entirely up to them. Of course I don't 
intend to rewrite all importers, just to change what they export and how 
LeoApp uses that what is exported.

 in LeoApp #
# instead of
def get_importer_for(key):
aClass = mydict.get(key)
def f(s, parent):
return aClass().run(s, parent)
return f
# we could simply use
def get_importer_for(key):
return mydict.get(key)

# in X_Importer 
# instead of
importer_dict = {
'class': X_Importer,
'extensions': [...],
}
# we could use
importer_dict = { 'func': X_Importer.do_import, 'extensions':[...] }

### and in the base scanner class
...
@staticmethod
def do_import(aClass):
def f(s, parent):
return aClass().run(s, parent)
return f

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/c9bcb9cd-f005-4105-b60e-3034aaea4d7an%40googlegroups.com.

Re: For Vitalije: About unit tests for the python importer

2021-12-06 Thread vitalije

Sorry, I've missed your two posts.

I've been working on new branch `importers` today.

While looking at the importer pipeline, I've noticed several conversion 
text to lines and backwards. First split is in checking tabs and spaces, if 
there are mixed tabs and spaces, again text is turned into lines, and back 
to text. Finally in gen_lines the same input text is once again divided in 
lines. So, I've changed all methods to accept lines as input argument 
instead of text. There were several places to change, in almost all 
importers that override gen_lines, but changes were minimal.

All unit tests passed except two related to leoAst that fail.

After that I've marked all test methods in test_leoImport.TestPython for 
skipping and I've started to add my own tests.
The old tests were checking only on headlines. Using new utility method for 
checking outline:

def check_outline(self, p, nodes):
it = iter(nodes)
zlev = p.level()
for p1 in p.self_and_subtree():
lev, h, b = next(it)
assert p1.level()-zlev == lev, f'lev:{p1.level()-zlev} != {lev}'
if lev > 0:
assert p1.h == h, f'"{p1.h}" != "{h}"'
assert p1.b == b, f'\n{repr(p1.b)} !=\n{repr(b)}'
try:
next(it)
return False, 'extra nodes'
except StopIteration:
return True, 'ok'

Now, both structure and content are checked.
After I've added 6 new tests with some corner cases that I've found to be 
interesting, I've deleted all other old tests in TestPython class.

You can check the tests and suggest any new test case. If you just provide 
Leo outline for interesting cases, I can easily turn them into test methods.
>From outline it is easy to get source text, and expected node structure. 
All new tests have same form:

def test_something(self):
txt = '''sourcecode'''
exp_nodes = [(lev, h, b),...]
p = self.run_test(txt)
ok, msg = self.check_outline(p, exp_nodes)
assert ok, msg

Look at the branch `importers` and we can discus what I've accomplished so 
far. I don't think there will be any serious problems in the future. 
Function split_root, should have some more comments and perhaps better var 
names, before PR is ready, but all remaining tweaks should be easy.

Let me know what do you think about the results of new python importer.
Vitalije
On Monday, December 6, 2021 at 10:01:02 PM UTC+1 Edward K. Ream wrote:

> On Monday, December 6, 2021 at 10:35:03 AM UTC-6 Edward K. Ream wrote:
>
> > I shall soon convert all tests so that they use check_headlines rather 
> than the more verbose equivalent.
>
> Done in devel at rev c65459. In the tests I have marked headlines that 
> should not be generated, or have poor headlines.  Vitalije, feel free to 
> adjust the arguments so the call passes.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/f54984cc-86c5-47c3-a4a1-b377beca05acn%40googlegroups.com.

Re: A new possible approach to importing python source files

2021-12-06 Thread vitalije

I am trying to catch up with new testing and your work on importers so far. 
What is the proper way to run tests in Leo, now?


On Monday, December 6, 2021 at 12:15:47 AM UTC+1 Edward K. Ream wrote:

> On Sunday, December 5, 2021 at 5:09:30 PM UTC-6 Edward K. Ream wrote:
>
> You are in complete charge.  Just remember that all (still applicable) 
>> unit tests should pass.
>>
>
> One more thing. The post-pass is a kind of peephole optimizer.  Almost 
> certainly the new importer should provide similar functionality. For 
> example, it's tricky to place class and def decorators where they belong in 
> the main importer phase. The present post-pass does so.  That's why I 
> suggest that you use most of the pipeline.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/ec920677-6c42-49e4-90d7-db3f3d10ab4cn%40googlegroups.com.

Re: A new possible approach to importing python source files

2021-12-05 Thread vitalije

Ok. I'll try to make PR.

On Sunday, December 5, 2021 at 11:27:02 PM UTC+1 Edward K. Ream wrote:

> On Sunday, December 5, 2021 at 2:09:05 PM UTC-6 Edward K. Ream wrote:'
>
> >  I have begun my study.
>
> Oh joy, replacing difflib (as the test file) with the "strange 
> indentation" unit test just works! My modified version of import_one_level 
> reports "same 1".
>
> Note: there was a bug in my version of import_one_level: it must return 
> root, not None.
>
> *Summary*
>
> I hereby "sign off" on the prototype python importer.  Vitalije, if you're 
> willing, I would like a PR from you that integrates your test script into 
> Leo's importer framework.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/bded719c-5530-48c0-b2c8-59ccf1f7ce1fn%40googlegroups.com.

Re: A new possible approach to importing python source files

2021-12-05 Thread vitalije

Yes, None when used as index gives the same effect as if it was omitted. 
aList[i:None] is the same as aList[i:]. The same is correct if None is the 
first index.

> Substantial work would be required to fold your script into the overall 
infrastructure created by the base Importer class. I would be happy to let 
you take it from here, but I'm also willing to help.  What would you like 
to do?

I don't see what substantial work do you mean is required. It is just a 
simple function, that can be added to any module, and used to import python 
source file instead of whatever is used now. It would be enough to use:

class MyPyScanner:
   def __init__(self, *args, **kwargs):
   pass
   def run(self, s, root):
   root.deleteAllChildren()
   root.b = s
   split_root(root)
   return True

instead of currently used Py_Importer. And register this class as a scanner 
class for python extension.
Or in Py_Importer class, just override run method to use split_root as 
shown above. This suggestion was made in accordance to the Leo version 
96f282672d 
of 2020-09-14 which is my current version. You may have changed this system 
recently, but the basic idea remains the same. Just do not call pipeline 
but instead call split_root and return True.


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/725800fb-9a8c-4003-8f98-07c4d607471cn%40googlegroups.com.

Re: A new possible approach to importing python source files

2021-12-03 Thread vitalije

I've attached the wrong version of script. Here is the correct one.

On Friday, December 3, 2021 at 4:09:17 PM UTC+1 vitalije wrote:

>
>> Clearly, you think differently than I do, and that's a very good thing 
>> :-) I'd like to get a feel for exactly how the code works, by "seeing it in 
>> action", so to speak.  I'll then attempt a theory of operation for my own 
>> benefit.
>>
>>
> I think that is a good idea. I would like to explain a little more how my 
> code works to make that easier.
>
> The function find_node_borders uses python tokenizer to search for 
> suitable node starting lines (code lines starting with token.NAME whose 
> string value is either def or class. For each found line it appends a list 
> [start_row, start_col, None, line_txt] to the list of possible results. The 
> third element which at this time is None, should be later changed to the 
> end line for this definition. The end of the definition might be noticed in 
> the generated tokens in two different cases. The first case is when we have 
> non comment python statement following the last line. In that case, 
> tokenize will emit token.DEDENT token. However, the unindented comment 
> lines will not trigger this DEDENT token, so we need to also check for the 
> COMMENT tokens having the starting column less then the body of the current 
> definition. In order to know the current indentaion we need to look for 
> token.INDENT tokens too. 
>
> So, the only tokens that we are interested in are:
>
>1. token.INDENT for keeping track of lastindent
>2. token.DEDENT and token.COMMENT whose starting col is less than 
>lastindent . Whenever we encounter this case, we need to close all open 
>definitions with the level of indentation grater than the current starting 
>column of found token (DEDENT  or COMMENT)
>3. token.NAME with the string value in ('def', 'class'). Whenever we 
>encounter this case, we add another open definition to both the list of 
>results and to the list of open definitions at this level of the 
>indentation.
>
> After passing through all tokens we now have closed all open definitions 
> and our resulting list contains each definition in file with its starting 
> (row,col) position and also with the line number where it ends.  Finally we 
> filter our resulting list to exclude all definitions that are at the deeper 
> level and keep only those which start at the zero indentation. As I write 
> this, I've just realized that I can substantially simplify script by 
> ignoring those deeper definitions in the previous phase.
>
> Yes, I have further simplified and speed up code by ignoring all tokens 
> that do not start at the column zero. Instead of keeping a list of open 
> definitions we now have just one open definition (the last one).
>
> Finally, we can start emitting the nodes. We start with the [1, 1, ''] 
> node. This is the block of lines that comes before first definition. While 
> walking through the list of definitions, we check every time to see if the 
> end of the last node in list is less than the start line of this node. In 
> that case we have to insert one '...some definitions' block which starts 
> where last node ends and ends where this node starts. At the end we add 
> [end_of_last_node, None, ''] to our final list of nodes. These are the 
> lines of root node which come after 'at-others'.
>
> There are two functions that use the result of find_node_borders: 
> split_root and split_class. The first one is dealing with the root node and 
> generates the children of the root node for each node in the list. It also 
> sets the body of the root node. The other one split_class is very similar 
> function. It takes the body of the class definition without actual class 
> line, cuts as much white space at the beginning of each line, and use this 
> dedented text to calculate borders of its methods. Then does with the 
> resulting list of nodes almost the same as the split_root does, except it 
> adds indentation to at-others and also restores its first line (class line).
>
> Attached to this message is simplified version of the script.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/cd7f6885-397f-4292-8204-dea8a5cb7732n%40googlegroups.com.
import tokenize
import token
import io
from collections import defaultdict
c.frame.log.clearTab('Log')
def split_root(root):
'''
Parses the text of the body and separates all
top level function definitions

Re: A new possible approach to importing python source files

2021-12-03 Thread vitalije


>
>
> Clearly, you think differently than I do, and that's a very good thing :-) 
> I'd like to get a feel for exactly how the code works, by "seeing it in 
> action", so to speak.  I'll then attempt a theory of operation for my own 
> benefit.
>
>
I think that is a good idea. I would like to explain a little more how my 
code works to make that easier.

The function find_node_borders uses python tokenizer to search for suitable 
node starting lines (code lines starting with token.NAME whose string value 
is either def or class. For each found line it appends a list [start_row, 
start_col, None, line_txt] to the list of possible results. The third 
element which at this time is None, should be later changed to the end line 
for this definition. The end of the definition might be noticed in the 
generated tokens in two different cases. The first case is when we have non 
comment python statement following the last line. In that case, tokenize 
will emit token.DEDENT token. However, the unindented comment lines will 
not trigger this DEDENT token, so we need to also check for the COMMENT 
tokens having the starting column less then the body of the current 
definition. In order to know the current indentaion we need to look for 
token.INDENT tokens too. 

So, the only tokens that we are interested in are:

   1. token.INDENT for keeping track of lastindent
   2. token.DEDENT and token.COMMENT whose starting col is less than 
   lastindent . Whenever we encounter this case, we need to close all open 
   definitions with the level of indentation grater than the current starting 
   column of found token (DEDENT  or COMMENT)
   3. token.NAME with the string value in ('def', 'class'). Whenever we 
   encounter this case, we add another open definition to both the list of 
   results and to the list of open definitions at this level of the 
   indentation.

After passing through all tokens we now have closed all open definitions 
and our resulting list contains each definition in file with its starting 
(row,col) position and also with the line number where it ends.  Finally we 
filter our resulting list to exclude all definitions that are at the deeper 
level and keep only those which start at the zero indentation. As I write 
this, I've just realized that I can substantially simplify script by 
ignoring those deeper definitions in the previous phase.

Yes, I have further simplified and speed up code by ignoring all tokens 
that do not start at the column zero. Instead of keeping a list of open 
definitions we now have just one open definition (the last one).

Finally, we can start emitting the nodes. We start with the [1, 1, ''] 
node. This is the block of lines that comes before first definition. While 
walking through the list of definitions, we check every time to see if the 
end of the last node in list is less than the start line of this node. In 
that case we have to insert one '...some definitions' block which starts 
where last node ends and ends where this node starts. At the end we add 
[end_of_last_node, None, ''] to our final list of nodes. These are the 
lines of root node which come after 'at-others'.

There are two functions that use the result of find_node_borders: 
split_root and split_class. The first one is dealing with the root node and 
generates the children of the root node for each node in the list. It also 
sets the body of the root node. The other one split_class is very similar 
function. It takes the body of the class definition without actual class 
line, cuts as much white space at the beginning of each line, and use this 
dedented text to calculate borders of its methods. Then does with the 
resulting list of nodes almost the same as the split_root does, except it 
adds indentation to at-others and also restores its first line (class line).

Attached to this message is simplified version of the script.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/7b78dee2-4370-44b1-8d3d-25a266e747b8n%40googlegroups.com.
import tokenize
import token
import io
from collections import defaultdict
c.frame.log.clearTab('Log')
def mk_py_importer():
def find_node_borders(txt):
'''
Returns a list of tuples (startrow, endrow, headline)
for direct children of the node.
'''
inp = io.StringIO(txt)
tokens = list(tokenize.generate_tokens(inp.readline))
res = []
lastindent = 0
open_definition = None
for i, tok in enumerate(tokens):
row, col = tok[2]
if col > 0: continue
if tok[0] == token.INDENT:
lastindent = len(tok[1])
continue
case_1 = tok[0] == token.COMMENT and lastindent > 0

Re: A new possible approach to importing python source files

2021-12-03 Thread vitalije



On Thursday, December 2, 2021 at 10:31:56 AM UTC+1 Edward K. Ream wrote:

>
> > Idea: use tokenize python module...
>
> This idea might work, but the attached script "wishes away" the 
> difficulties involved in generating code.
>
>
Attached to this message is an improved version of this script, which 
doesn't "wish away the difficulties", but does full import. Each module 
level defined function and class goes into a separate node and furthermore 
for each class node that has more than 20 lines, each method goes into a 
separate child node to. All other lines that are between the nodes are in 
their separate nodes with the heading '..some declarations'. If the body 
contains only one string definition its headline is changed to `__doc__` 
and if it contains only comments, then its headline is changed to '...some 
comments'.

This is pretty much how I would split any module by hand. I don't know what 
your tests for Python importer look like, but I suggest that you try this 
script against those tests. The whole script is just 150 lines long 
including the comments and doc strings. It is short and it is easy to 
modify it any way you like. It has successfully imported every python 
module from the standard library that I throw at it, each time the perfect 
import.

I would expect that your code in the end will have much more than 150 
lines. The more lines of code, more space for bugs to hide, more effort 
later to read and understand the code in order to change or fix something. 
The choice is yours.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/ce3150fb-ea17-4e18-87e9-9e140c7529f5n%40googlegroups.com.
import tokenize
import token
import io
from collections import defaultdict
c.frame.log.clearTab('Log')
def mk_py_importer():
def find_node_borders(txt):
'''
Returns a list of tuples (startrow, endrow, headline)
for direct children of the node.
'''
inp = io.StringIO(txt)
tokens = list(tokenize.generate_tokens(inp.readline))
res = []
open_definitions = defaultdict(list)
lastindent = 0
def close_defs(row, col):
for k in open_definitions:
if k >= col:
for r in open_definitions[k]:
if r[2] is None: r[2] = row
del open_definitions[k][:]

for i, tok in enumerate(tokens):
row, col = tok[2]
if tok[0] == token.INDENT:
lastindent = col+len(tok[1])
continue
if (tok[0] == token.COMMENT and lastindent > col) or tok[0] == token.DEDENT:
close_defs(row, col)
elif tok[0] == token.NAME and tok[1] in ('def', 'class'):
res.append([row, col, None, tok[-1].strip()])
open_definitions[col].append(res[-1])
i = 1
nodes = [[1,1, '']]
for a, col, b, x in res:
if col > 0: continue # ignore deeper definitions
if a > nodes[-1][1]:
nodes.append([nodes[-1][1], a, '...some declarations'])
nodes.append([a, b, make_headline(x)])
nodes.append([nodes[-1][1], None, ''])
return nodes
def make_headline(line):
line = line.strip()
if line.startswith('class '):
return line[5:].strip()[:-1]
else:
return line[4:].partition('(')[0].strip()
def rename(p):
toks = [x for x in tokenize.generate_tokens(io.StringIO(p.b).readline)
if x[0] not in (token.NEWLINE, token.NL, token.ENDMARKER)]
if all(x[0]==token.STRING for x in toks):
p.h = '__doc__'
elif all(x[0] == token.COMMENT for x in toks):
p.h = '...comments'

def split_root(root):
'''
Parses the text of the body and separates all
top level function definitions and class definitions
in separate nodes which are all direct children of
the root.

In the second phase, this function can be called on
each of the children with more than a certain threshold
number of lines.
'''
root.deleteAllChildren()
txt = root.b
lines = txt.splitlines(True)
def body(a, b):
return ''.join(lines[a-1:b and (b-1)])
nodes = find_node_borders(txt)
a, b, h = nodes[0]
root.b = f'{body(a, b)}@others\n{body(nodes[-1][0], None)}'
for a, b, h in nodes[1:-1]:
child = root.insertAsLastChild()
child.h = h
child.b = body(a, b)
if child.b.startswith('class ') and (b - a) > 20:
split_class(child)
if h == '...some

A new possible approach to importing python source files

2021-12-01 Thread vitalije

Idea:
use tokenize python module to find where all function/method and class 
definitions start,
and then use this data to find lines where top level children should start. 
After creating the top level children, the process can be repeated for all 
nodes which have more than certain threshold number of lines, generating 
the second level children.

A little bit of background story (feel free to skip if you just want to see 
the example code):
A long ago I've tried to solve this problem in more efficient way for 
importing JavaScript files. I remember looking in the Importer class and 
the way Leo did imports at the time and feeling that it was too 
complicated, much more than necessary. I can't say that I've solved this 
problem in general, but for a very specific case, it worked pretty well.

Recent posts about improving Leo in this area, especially regarding Python, 
made me think again about this problem.

I strongly feel that the main problem with the current implementation is 
insisting on the use of scan_line. This is maybe suitable for unification 
of all other source languages, but it is far from the optimal when we talk 
about the python source files.

The way I see this problem is to search and find the lines where a new node 
should start. Whether this node should be indented or not, I would rather 
leave for the next phase. First of all, the outline structure of my python 
files which I start from the scratch in Leo usually have in the top level 
node a few lines, then comes at-others and usually after at-others comes 
the block with `if __name__ == '__main__':. If I have a lot of imports, 
then I usually put all imports in one section node `<>`.

The first line where I would introduce the first child node is actually 
first function definition or first class definition. Everything before 
should go directly in the root node. Imports can be extracted later.

Attached to this message is a python script which can be executed inside 
Leo.
It imports any python module from the standard python library and checks to 
see if the import is perfect or not. At the moment it just extracts top 
level definitions in separate nodes, the direct children of root.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/a33ace8a-ee35-4166-8ca8-d25a69f6cdf0n%40googlegroups.com.
def find_node_borders(txt):
'''
Returns a list of tuples (startrow, endrow, headline)
for direct children of the node.
'''
inp = io.StringIO(txt)
tokens = list(tokenize.generate_tokens(inp.readline))
res = []
open_definitions = defaultdict(list)
for i, tok in enumerate(tokens):
row, col = tok[2]
if tok[0] == token.DEDENT:
for k in open_definitions:
if k >= col:
for r in open_definitions[k]:
if r[2] is None: r[2] = row
del open_definitions[k][:]
elif tok[0] == token.NAME and tok[1] in ('def', 'class'):
res.append([row, col, None, tok[-1].strip()])
open_definitions[col].append(res[-1])
i = 1
nodes = [[1,1, '']]
for a, col, b, x in res:
if col > 0: continue # ignore deeper definitions
if a > nodes[-1][1]:
nodes[-1][1] = a
nodes.append([a, b, make_headline(x)])
nodes.append([nodes[-1][1], None, ''])
return nodes

Re: "Random" de-hoist

2021-09-08 Thread vitalije

I doubt that it has anything to do with your typing. Most likely hoisted 
position becomes invalid and Leo automatically de-hoist. As I see from your 
keys the last thing you have done is Ctrl-s. Have you altered this shortcut 
to execute some script or it is still attached to the Leo's save command? 
Perhaps you have some plugin that has registered an event handler to 
execute before or after the save. If any of these handlers change outline 
or if they select positions out of the hoisted subtree then Leo will 
automatically de-hoist.
Vitalije
On Tuesday, September 7, 2021 at 4:13:34 AM UTC+2 Phil wrote:

> Thanks for that Edward. I had a "random dehoist" again. Here's the 
> sequence of events I captured. I don't see any obvious reason why the tree 
> should dehoist. Hopefully this makes sense, and is helpful for you.
>
> 
>
> This is the text I was editing, in one node, typing line N, then copying 
> text from another node and pasting it as line N+1.
>
> line N:  local level = t[2]
> line N+1:print(indent_text(tostring(s), 2*level))
>
>
> Here's the view-lossage output; I've added annotations to the right:
>
> l  {start typing line N}
> o
> c
> a
> l
> Space
> l
> e
> v
> e
> l
> Space
> =
> Space
> [
> 2
> ]
> BackSpace  {noticed the error, backspaced to correct it}
> BackSpace
> BackSpace
> t
> [
> 2
> ]  {finished typing line N}
> Return {after the Return, I clicked on another node to copy and 
> paste some code}
> Ctrl+c {selected text with the mouse, copied it, then clicked back 
> to the target node}
> Ctrl+v {pasted the text}
> Ctrl+s {saved, as I do frequently - it is here that my hoisted 
> subtree dehoisted}
> v
> i
> e
> w
> -
> l
> o
> s
> s
> a
> g
> e
> Return
>
>
> On Saturday, September 4, 2021 at 11:34:52 AM UTC-5 Edward K. Ream wrote:
>
>> On Sat, Sep 4, 2021 at 10:15 AM Phil  wrote:
>>
>>> When working in a hoisted tree, I occasionally experience "random" 
>>> de-hoisting.
>>>
>> ... 
>>
>>> Any ideas?
>>>
>>
>> The view-lossage command shows the last keystrokes you typed.
>>
>> Edward
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/3cac79e4-9114-4906-a1f6-77440c17d560n%40googlegroups.com.

Re: ENB: A valid mypy complaint

2021-03-03 Thread vitalije

Setting p.v to None is just a convention. I don't think that any other part 
of Leo depends on this convention except p.__bool__


It would be trivial to change p.__bool__ implementation for example: 
instead of: return self.v is not None, we could have return self._valid. 
And then instead of setting self.v = None, just: self._valid = False. This 
would remove mypy complains, although I am not sure if it is really 
necessary.  The only thing that needs to be dealt with is searching for 
tests p.v is None or p.v is not None, but I doubt that there are many such 
places.



> All comments welcome. I would especially like to know how leojs handles 
> this issue.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/38b88f9d-9a1f-4c68-8fc2-fdcb9f31009en%40googlegroups.com.

Re: ENB: The kernel of leojs: Leo in JS

2020-12-16 Thread vitalije

Here are some ideas which you are free to ignore if you find them 
uninteresting or useless.

There is also a possibility to have Leo's core implemented in rust (or some 
other low level language) and exported as both python extension module and 
node module. I've started to work on this idea, but right now I am too busy 
to work on that project (mini_leo <https://github.com/vitalije/mini_leo/>). 

I did offer Felix to use this code (instead of leoBridge) when he started 
his project, but he wasn't interested.

Exporting rust functions to python and node is easy. There is a possibility 
to export same functions as a wasm module for use in the browsers too.

I don't fully remember current state of the project but IIRC there are 
already functions for reading/writing .leo files, derived at-file files, 
iterating through outlines, basic outline manipulations.
Here is a benchmark test:

Python 3.7.0 (default, Jun 28 2018, 13:15:42)
[GCC 7.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
  with open('leo/test/activeUnitTests.txt', 'r') as inp:
  s = inp.read()
  return _minileo.outline_from_str(s)
>>> import _minileo
>>> import timeit
>>> print(timeit.timeit(f, number=100)/100*1000, 'ms')
6.344196610007202 ms
As you can guess function outline_from_str parses derived at-file content 
and builds outline. leo/test/activeUnitTests.txt used to be the largest 
derived file in Leo's code. There is also a function load_leo to load 
outline from .leo file along with all external (derived) files.

I've also some ideas of using sqlite3 in combination with a few functions 
written in rust for dealing with the outlines. I had some very promising 
experiments with this idea. The sqlite3 proves to be fast enough (faster 
than python) for iterating outlines, handling undo/redo, searching for 
nodes,...
And exporting this functionality to all scripting languages (python, 
nodejs, and javascript in browsers) is straightforward.

I don't know when will I have the time to continue work on this project, 
but the code is there, for what it's worth.

Reimplementing Leo's core classes in Python to be compiled to JavaScript, 
today, is IMHO a bad idea (YMMV). Python is great for other parts of Leo, 
but not so great for the core. If the core should be reimplemented at all, 
I believe it would be complete waste of time unless it is reimplemented in 
a language that is substantially faster than the current python 
implementation. Rust seems to be fine choice these days for this task. The 
library can be cross compiled to work on any machine, including browser and 
it also allows exporting native rust functions to both python and nodejs. 
But Rust isn't the only one in town. There are also zig and v languages. 
Both of them claim that they've fixed almost all problems with C language. 
Both of them offer good cooperation with python, nodejs and javascript in 
browser.

I dream of a Leo core module that could be easily used in server code, 
browser front-end and native gui front-end. With such a module and little 
bit of scripting it would be trivial to have outlines displayed on the web 
and editing outlines remotely. Imagine the combination of leo-ver-serv 
(which keeps detailed history of my Leo outlines taking snapshots every few 
idle seconds), editing outlines remotely, and displaying outlines in 
browser. That combination would be better for hosting code than github. 
Currently presentation of Leo outlines on github is terrible. Imagine that 
people can see the code organized in Leo outlines, and even to suggest some 
change by editing nodes directly in browser.

My two cents.
Vitalije
On Wednesday, December 16, 2020 at 5:27:06 PM UTC+1 Edward K. Ream wrote:

> This Engineering Notebook post considers strategies for implementing the 
> crucial parts of Leo in javascript. leojs issue #9 
> <https://github.com/boltex/leojs/issues/9> recaps what must be done. The 
> following must be present in leojs 0.1:
>
> - Leo's Position and VNode classes,
> - Outline-oriented undo,
> - Outline-oriented search.
>
> That's a lot of complex code!
>
> True, leojs is worth any amount of work. However, it would be stupid to 
> dive into coding! This morning I considered some cute possibilities for 
> doing significant parts of the work in python. Then I thought to google: 
> "python to javascript" :-)
>
> One of the first hits was this overview 
> <https://www.infoworld.com/article/3209651/how-to-convert-python-to-javascript-and-back-again.html>,
>  
> which leads to:
>
> - JavaScripthon <https://github.com/metapensiero/metapensiero.pj>: a 
> Python 3 to ES6 JavaScript translator.
> - RapyScript <https://github.com/atsepkov/RapydScript>: pre-compiler for 
>

Re: SB: Leovue progress report

2020-11-23 Thread vitalije


>
> Uncaught ReferenceError: exports is not defined app.js line 21793. I 
> would appreciate any help anyone might give me about this. 
>
>
This kind of error suggests that the script your browser loads is intended 
for nodejs scripting environment not for the browser.  In order to be used 
in the webpage this scripts must be transformed or repackaged. IIRC leo-vue 
uses webpack as build tool. The build tool (whatever it might be) bundles 
all separate javascript files in one big javascript file. That file should 
be loaded in the webpage. It means that the script tag in your index.html 
needs a src attribute to be correctly set to load this big javascript which 
contains all separate files bundled together.

Look in the webpack configuration file to find out where the result of 
build process should be and then adjust `` inside your 
index.html to the correct path.

HTH Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/e6b0228f-b91d-409d-afbb-3c00097c6349n%40googlegroups.com.

Re: Could someone help me install leovue on a server?

2020-11-07 Thread vitalije



On Saturday, November 7, 2020 at 1:25:02 PM UTC+1 Edward K. Ream wrote:

> After building leovue, per the instructions of #735 
> <https://github.com/leo-editor/leo-editor/issues/735>, I uploaded 
> leovue/dist/static to my server, but the site is not working.
>
>
Have you uploaded leo/dist/index.html too? And have you changed 
configuration options in index.html to show your Leo outline? 

What do you see in web console (click F12 and look in the console tab)? Are 
there any error messages?

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/eb22b96d-10ad-4826-90b6-189fd61d91c6n%40googlegroups.com.

Re: Unifying leoPy.leo and leoPlugins.leo is a big win

2020-10-18 Thread vitalije



On Saturday, October 17, 2020 at 10:22:44 PM UTC+2, Edward K. Ream wrote:
>
>
>
> On Sat, Oct 17, 2020 at 7:30 AM vitalije > 
> wrote:
>
> I think you should really try using public/private outline scheme. 
>>
>
> Thanks for the reminder. I'm not sure I always want ctrl-s to save to both 
> outlines. Instead, I'll create a new command that only calls c.
> fileCommands.save_ref(). 
>
> I use it in my LeoPy.leo and it was never an issue. If all my work is done 
in the private zone, git never showed LeoPyRef.leo changed. 
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/bc68dccb-aa3b-40d4-8d5b-006e13f83817o%40googlegroups.com.

Re: Unifying leoPy.leo and leoPlugins.leo is a big win

2020-10-17 Thread vitalije


>
> Similarly, the open-leo-py-ref-leo command makes it significantly easier 
> to keep LeoPyRef.leo in sync with my personal leoPy.leo files. It may sound 
> like a small matter, but it isn't. 


I think you should really try using public/private outline scheme. Just to 
remind you of this rather hidden Leo feature.

If an outline has a top level node with the headline 
`---begin-private-area---`, then this node is separating the public part of 
the outline (before this node) and the private part of the outline (after 
this node). The first line of this node should have a path to the public 
(shared) Leo document.

Now when you execute c.fileCommands.save_ref() command, the public part is 
saved in the shared Leo document. In my private Leo documents I always have 
a node with headline `@button n-save @key=Ctrl-s` and the body line always 
contains at least these two lines:

c.save()
c.fileCommands.save_ref()

That way my public or reference Leo files never get out of sync.

When I pull changes from the others, I have to remember to execute 
`read-ref-file` which updates public part of the outline from the reference 
file.

Vitalije

>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/db3be5ac-30a6-47bf-8d61-fda94cc272b6o%40googlegroups.com.

Re: ENB: Expanded/collapsed status of outline nodes

2020-09-24 Thread vitalije


>
>
> How do you want to go forward?
>
>
I wrote about this idea to check whether it has any chance of being 
accepted or not. As it seems you are willing to give it a chance, I might 
do it in the separate branch. However, I have some more ideas (not fully 
formulated yet) that I wish to explore before going into the 
implementation. Those ideas might have some influence on the final 
implementation.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/2c365858-f95d-41d9-8f2c-59008d029a65o%40googlegroups.com.

Re: ENB: Expanded/collapsed status of outline nodes

2020-09-24 Thread vitalije


>
>
> Could Leo's Position class call these functions automagically? That would 
> seem to be a complete solution.
>
>
This two functions can be methods of c, and c could have an attribute 
expandedLabels. I guess the outline manipulation methods located in 
Position class, can call these c methods when appropriate to update 
c.expanded_labels set.
 

> This seems like a promising strategy.
>
> Edward
>

I am glad we agree on this. It looks like a promising strategy to me too.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/18e2eea5-48ce-4c78-904d-18f6e3f4d3b6o%40googlegroups.com.

Re: About changes to VR3

2020-09-23 Thread vitalije

If my memory is correct, I've added some years ago code to the free layout 
to allow Leo to remember layout of panes. Before that if one opens vr pane 
it would change position of splitters and after closing pane body would 
become smaller than it used to be before opening vr pane. That was annoying 
and there was a feature request for preserving layout.

When you open some pane in a separate window, it is usually removed from 
the nested splitter and I guess this new layout is saved, so when you 
reopen Leo the pane is missing from the layout. User needs to use eastern 
egg menu to split some area and then to add vr3 pane to the newly formed 
part of layout.

This two step action can be turned into a new command. If you want to 
programmatically open vr3 pane inside main Leo window, you need first to 
add new area by splitting an existent pane and then to set vr3 as a 
provider for the newly created area.

HTH Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/b6fdcf4b-0973-4230-9544-6d6708f7fd57o%40googlegroups.com.

ENB: Expanded/collapsed status of outline nodes

2020-09-18 Thread vitalije

alling 
these functions, nothing horrible would happen. The worse thing that could 
happen is that some of the clones would loose their expanded/collapsed 
state. But most of the ordinary nodes will have their expanded state 
preserved.

Your comments.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/5cf6de23-4e0b-4608-b60c-96d874cccf0ao%40googlegroups.com.

Re: Discuss: drop support for Qt docks?

I agree that removing support for docks will reduce complexity, and that is 
desirable. Personally I haven't used docks, so I won't be missing them. 
Perhaps users who really use this feature should raise their voices now.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/4735722c-9baf-4a28-b7b0-264635c445dco%40googlegroups.com.

Re: The docks3 branch is ready for testing


>
>
> I'm lost. What feature are you talking about?
>
>
Sorry, I don't express myself clearly. The feature of nested_splitter which 
allows opening separate top level windows for outline, body and other Leo 
widgets. (Right click on the splitter and choosing a command in the "Open 
window" sub-menu)

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/02ae7c4f-3909-418e-b519-a359629d3844o%40googlegroups.com.

Re: The docks3 branch is ready for testing

I didn't use this feature to have widgets in the separate top level window. 
Perhaps it should be improved a bit to be really useful. For example, when 
switching between Leo documents, these top level windows should also switch 
their content. Right now, they don't switch and it is possible to have 
displayed outline from one document while editing another document. When 
you extract a widget in separate window in one document, all other 
documents still have that widget inside the main window.

It seems that nested_splitter just needs a bit of polishing.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/54e23d29-2bf2-47fa-816f-9617587e6cfdo%40googlegroups.com.

Re: The docks3 branch is ready for testing


>
>
> Docks can be floated, which might be useful when using multiple monitors. 
> Perhaps this is dock's major advantage. Docks are also necessary for the 
> pyzo_in_leo plugin, but I *would *be willing to abandon that plugin.
>
>
It is possible to have outline, body and tabs in separate top level windows 
in no-docks. Just right click on the split line and choose "openWindow" 
then any widget you want and it will be displayed in the separate top level 
window that can be moved to the different monitor.
 

> Good point. Another option would be to use the outline dock as the only 
> possible central widget.
>
>
Also I've noticed that window state is kept in the global cache using a key 
which contains the file name. That seems to me unnecessary. Both global and 
local cache use the same code under the hood, but the local one mangles the 
keys using file name. So, it would be simpler if the layout was stored in 
the c.db['windowState'] instead of g.app.db[f'windowState:{c.fileName()}']

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/da146e09-4786-4608-87a1-a56597e201c2o%40googlegroups.com.

Re: The docks3 branch is ready for testing


>
>
> I'm pretty clueless about the Qt docks code myself. Leo is creating a new 
> QMainWindow for every outline. (Each DynamicWindow is a subclass of 
> QMainWindow). That may be causing problems, maybe even exposing a latent Qt 
> bug.
>
> There is a *lot* of cruft in the docks-related code. Recent code has 
> (supposedly!) changed over to using a single set of global Qt docks.
>
>
I have tried docks once and gave up them shortly. This morning I decided to 
look at them again and try to help with this issue (if I can).

Initial look was awful. Then it was too easy to mess up docks so much that 
the layout become totally unusable. I very much dislike clearing my cache 
because I keep there some data important for my work process, so I tried to 
delete only entries related to dock layout using script. But, those entries 
are saved again when closing Leo and they are saved in unmanageable state 
again. I had to delete entries using script and then to kill Leo, to start 
again with the default layout. Then I realized that in order to achieve my 
usual layout (like the one I use without docks: body on the right side from 
top to bottom, and on the left outline above tabs), I have to change 
setting @string central-dock-widget=body. Once I added this setting and 
restarted Leo, layout got broken again because saved layout had different 
central widget. When I finally managed to adjust layout to my liking, I 
have saved and closed Leo. On the next start layout was well preserved, but 
hitting Ctrl-n (new outline), I got again default layout, not the copy of 
first open Leo layout. This is most likely because setting 
central-dock-widget was local to the first file, and new outline got the 
default value for this setting and therefore the layout of the first loaded 
outline can't be applied.

I would be really surprised if a newbie had enough patience and commitment 
to adjust the initial layout to something usable.

Then I compared docks and no-docks Leo window. The only difference is that 
in docks all three panes (outline, body and tabs) have small label above. 
In effect it reduces useful screen space. So, I wander what are the reasons 
that one would prefer docks over no-docks layout.

Can anyone who uses docks and prefer docks explain me what are the benefits 
of using docks? Perhaps I miss something and I am not using them correctly.

Anyway, I think that it isn't enough to have just single hard-coded value 
in app.defaultWindowState. If this state can't be applied when 
central-dock-widget isn't the same as was used in this hard-coded value. 
There should be hard-coded value for each possible value of 
central-dock-widget. Perhaps these values might be stored in the theme 
files. There should be a command to launch a dialog for selecting one of 
the predefined layouts. If one messes up docks layout, this command should 
allow easy restoring layout to something usable.

It might be useful to have a command which would add current layout to the 
list of predefined layouts (that can be selected from the dialog) and a 
setting which one should be used for new outlines.

Unless I learn about some feature of the docks interface that makes it 
superior over the no-docks interface, I will stick to the no-docks 
interface.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/c026d1d0-6bb2-478e-a35b-f1c708eecc95o%40googlegroups.com.

Re: Leo Node Move Error

2020-09-04 Thread vitalije


>
>
> My *tentative* conclusion is that there may be a bug in 
> fc.retrieveVnodesFromDb. However, there are no special cases in this code, 
> so what the bug could possibly be is a big mystery. Iirc, Vitalije wrote 
> this code. Any insights would be appreciated.
>
>
I am almost 100% sure that the fc.retrieveVnodesFromDb works correctly. It 
just restores outline from the rows retrieved from db. Saving and loading 
from db work in tandem. If the outline being saved has broken links, then 
the outline loaded afterwards will have broken links too.

Most likely outline got broken links during some modification and later was 
saved as such.

Here is a script that can fix broken links:
from collections import defaultdict
def relink_outline(c):
'''Normalizes outline fixing broken links'''
parDict = defaultdict(list)
for p in c.all_positions():
parDict[p.gnx].append(p._parentVnode())
gnx2v = c.fileCommands.gnxDict
for gnx, parents in parDict.items():
v = gnx2v[gnx]
v.parents = parents
c.checkOutline()
relink_outline(c)

When saving to xml information about parent links is omitted. When loading 
from xml file, correct parent links are infered again. Perhaps I can change 
writing to db to omitt parent links and just infer them when loading from 
db.

It is still a mistery how k-hen got his outline to this invalid state. My 
guess is that it was caused during the updating clones in both @file files 
and @nosent files. That would explain why we haven't noticed this bug 
before. It is not very common to have shared clones in both @nosent and 
@file files.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/56aa7952-a09d-4da5-b176-206916448e26o%40googlegroups.com.

Re: Wow: wasmtime

2020-09-03 Thread vitalije

>
> I forgot to include the potential for malware. Some links:
>

According to the research mentioned in your third link that claims half of
the web sites using wasm use it maliciously, the malicious usage consist in
obfuscating and mining.

According to the first link, from the figure 4 it is clear that malicious
site will use users CPU to mine cryptocurrency even if the web browser
doesn't support wasm. So it is not only wasm's fault.

<https://www.virusbulletin.com/files/7115/3933/8355/dark-side-webassembly-fig4.jpg>

By definition wasm code can't do anything that ordinary javascript in the
browser can't do. If obfuscation is malicious then I don't know any serious
web site today that isn't malicious, because every web page uses minimized
and very often obfuscated javascript. So, as far as I am concerned,
obfuscation is not malicious per se.

Concerning user tracking, all major social networks use it all the time,
regardless of wasm. We are writing on the google forum and google use our
posts to dig some data from them. Who knows how many of us on this forum
have gmail accounts, github accounts,... all these platforms collect data
about us and about the way we use internet. They have done this long before
wasm technology. Therefore I don't see any special threat in using and
spreading wasm technology.

Comparison with the Flash technology is a bit stretched (IMHO). Flash was
used to overcome some deficiencies in browsers, their incompatibilities and
yes flash had some potential for writing malicious software. But wasm is
much safer and it relies on the browser safety mechanisms, it doesn't have
to be enabled or installed by the user. It is an integral part of newer web
browsers, no less than the javascript is.

Vitalije

--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/8b3961ac-760f-420f-aa48-1a1fa0eb3efbo%40googlegroups.com.

Re: Status report and Aha re positions and vnodes

2020-09-01 Thread vitalije

Profiling gives almost the same result for both methods.
import timeit
def f1():
for p1 in expandables:
p1.isExpanded()
def f2():
for p1 in expandables:
ec.should_be_expanded(p1)

def tt(s, fun):
t1 = timeit.timeit(fun, number=1000)
g.es(f'{s} average: {t1:.3f}')
tt('using p.isExpanded', f1)
tt('using ExpandController', f2)

On my machine this gives:
using p.isExpanded average: 0.816
using ExpandController average: 0.861


Vitalije 

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/1767bbf3-f7fb-40c3-867f-bc73f081fa4bo%40googlegroups.com.

Re: Status report and Aha re positions and vnodes

2020-09-01 Thread vitalije

I don't fully understand your previous post. Here is an example script that 
shows how expanded/collapsed state can be deduced without 
v.expandedPositions.
def p_to_count(p):
v = p.v
if len(v.parents) == 1:
return 0
for i, p1 in enumerate(sorted(c.all_positions_for_v(v))):
if p1 == p:
return i
return -1 # position isn't valid

class ExpandController:
def __init__(self, c):
counts = {}
self._expanded = expanded = set()
for _p in c.all_positions():
i = counts.get(_p.v, 0)
if _p.isExpanded():
expanded.add((_p.v, i))
counts[_p.v] = i + 1

def should_be_expanded(self, p):
v = p.v
i = p_to_count(p)
return (v, i) in self._expanded

def expand_position(self, p):
i = p_to_count(p)
if i > -1:
v = p.v
self._expanded.add((v, i))

def collapse_position(self, p):
i = p_to_count(p)
if i > -1:
v = p.v
self._expanded.remove((v, i))

def toggle_position(self, p):
i = p_to_count(p)
if i > -1:
v = p.v
if (v, i) in self._expanded:
self._expanded.remove((v, i))
else:
self._expanded.add((v, i))

# now let's test if both methods give the same result
ec = ExpandController(c)
expandables = [x for x in c.all_positions() if x.v.children]
# only nodes that have children can be expanded
for p1 in expandables:
a = p1.isExpanded()
assert a == ec.should_be_expanded(p1), repr((p_to_count(p1), a, p1))
if a: # let's toggle position using p methods
p1.contract()
else:
p1.expand()
ec.toggle_position(p1) # let's toggle in ExpandController too
assert p1.isExpanded() == p1.isExpanded()
if a: # return to the previous state
p1.expand()
else:
p1.contract()
ec.toggle_position(p1) # in ExpandController too

g.es('all tests pass')


The script creates an ExpandController which copies the expanded/collapsed 
state from the Leo during the init.
Then it creates a list of all positions that have children, because only 
those positions can be expanded/collapsed.
Then for each position in this list check to see if both methods give the 
same expanded state. Additionally it
toggles the state checks the results are still the same, and toggles once 
more to restore the previous state.

The script doesn't try to change the outline to see if the expanded state 
would survive, but I am pretty sure that ExpandController is more immune to 
the outline changes than the v.expandedPositions method.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/19cbf32d-26bf-4899-a00a-a361288f1412o%40googlegroups.com.

Re: Status report and Aha re positions and vnodes

2020-08-31 Thread vitalije

There are possibly many ways to specify which node should be expanded. Some 
of them are more sensitive to outline changes than others. Here is an idea 
of encoding expanded positions in more robust way.

When traversing outline we can encounter same v node several times if it is 
cloned. A pair (v, count) where count is integer showing how many times 
this same v node was seen during the traversal, can be used to make 
distinction between clones. The first occurrence of  v is encoded as (v, 
0). The second occurrence is encoded as (v, 1),...

Now if we want to know whether the position p should be expanded, the 
following code (untested) might give the answer:
def should_p_be_expanded(p, expanded): # expanded is a set of pairs (v, i)
v = p.v
for i, p1 in enumerate(sorted(c.all_positions_for_v(v))):
if p1 != p: continue
return (v, i) in expanded
return False


Of course if p.v has no clones then, correct answer is just p.v.isExpanded()

This scheme can also be broken by outline change, when this change affects 
ordering of clones, but it is resilient against inserting and deleting new 
nodes, and moving nodes that are not ancestors of a node. That covers a lot 
of more frequent kinds of outline changes.

My 2 cents.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/e0cefb2b-5baf-47db-825a-6d8245840d59o%40googlegroups.com.

Re: ENB: Ahas re positions

2020-08-26 Thread vitalije

 

>  It is visualised as expanded/collapsed in the gui, indeed right, but its 
> important in the model/controller as much as the 'headline string' or 'body 
> content' because of those commands which set the selected node based on 
> whats collapsed/expanded. So it is real model data/state as much as 
> headline content and body content.
>
> Nevertheless this piece of data *IS* stored in the Qt GUI, and I am sure 
that any GUI you can think of that implements Tree widget will have this 
information too. Whenever you have to keep a copy of some data in two or 
more places, you'll have to synchronize this data and there is always a 
possibility of getting out of sync. OTOH, when the model is designed in 
such a way that a single piece of information is always kept in one and 
only one place, there is no need for synchronization and it is impossible 
by design to get out of sync.

You might decide for your GUI not to keep track about expanded/collapsed 
state and to ask through the Leo bridge every time you need this 
information, but still under the hood, bridge won't be able to give you a 
stable response. If you decide to keep track of the expanded/collapsed 
state in your GUI, you will be able to set this information in Leo 
correctly before executing any of the commands you mentioned, or better 
yet, those commands can be separated in several different commands and you 
just choose one you want depending on the expanded/collapsed state.

This approach can be taken in any GUI. For example:
# instead of
def my_command(...):
if p.isExpanded():
do_one_thing()
else:
do_other_thing()

# it should be
def my_command_when_collapsed(...):
do_one_thing()

def my_command_when_expanded(...):
do_other_thing()

And then, let the GUI choose which command to call, depending on the 
current expanded/collapsed state.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/bc5ccb55-a168-4d49-8219-17dd98d6031fo%40googlegroups.com.

Re: ENB: Ahas re positions

2020-08-26 Thread vitalije

I doubt this would be a very good idea. Undo/redo already has a lot of
things to do. Adding some more chores to them will likely make things more
complicated, slower and probably introduce more bugs.

Instead I suggest another approach. Please consider the rest of this
message not as a critic of Leo's internals. I can't write any formal proof
of the following claims, so consider them just as my personal feelings
about Leo's position class.

The way I see it, right now, positions are mixture of Model, View and
Controller. Position methods are used to:

1. [Model] traverse the tree, retrieve the outline data
2. [View] represent outline nodes that can be viewed, clicked, double
clicked, expanded or collapsed
3. [Controller] modify tree

The fact that they are so involved in all three parts of MVC makes them
vulnerable and unreliable not only because they can become invalid - that
is just a part of the problem. There are a lot of other parts of Leo that
can be simplified by avoiding usage of positions. In every piece of code I
wrote in Leo, so far, I found that avoiding positions makes code shorter,
cleaner and more robust. More than once I've found some method in Leo that
was too complex or it contained a bug somewhere, and when I rewrote it
without positions it became shorter, faster and more reliable. At least
that is my experience with the Leo positions. There are even some places
where one piece of code doesn't have a position but just a vnode, and in
order to call some method that expects position one must first construct a
dummy position containing the vnode, then call the other method which
expects position, and inside this method uses p.v to do its job.

Now, regarding the initial problem (I believe it was about
expanded/collapsed state of nodes). Whichever GUI we can imagine that Leo
will have in the future, and also in the Qt GUI at the present time, the
GUI code will have the information about which visible nodes are expanded
and which are collapsed. It is an essential bit of information required to
show the outline. So, most natural place to hold this information is GUI
code. If we put it somewhere else, we will have to deal with the
synchronization between GUI and that other place.

Some Leo commands for moving around the outline, selecting relative
positions depend on this information (expanded/collapsed) but apart from
that, expanded/collapsed state has no effect on the true meaning of the
outline. External files will contain the same content regardless of whether
their nodes were expanded or collapsed. So, we can consider this
information as something inherently tied to the GUI or View part of MVC.

The simplest possible solution will be just to delegate p.isExpanded method
to the GUI and find corresponding tree item and ask it if it is expanded or
not. For example:
g.app.gui.isExpanded(p) # or c.frame.tree.isExpanded(p)
# instead of
p.isExpanded()

IMHO the most natural place to store the expanded/collapsed bit about
outline node is in the GUI/View. Also rethinking positions and their
methods might be beneficial.

My 2 cents.
Vitalije

--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/43ba1540-646c-414e-a9e8-cf2f361a693eo%40googlegroups.com.

Re: Leo Write-Only Mode

2020-08-24 Thread vitalije



On Monday, August 24, 2020 at 3:15:50 PM UTC+2, k-hen wrote:
>
>
> Pardon the beginner question, but are you defining @coffee, @pug, etc 
> yourself? 
> Why not @md then instead of md: on the headline?
> Either's fine of course, just trying to understand the differences and the 
> best practices.
>
>  
Yes I am. Why not @md? There is no particular reason at all. A monkey sign 
is usually signaling some special meaning in Leo outlines, but it is just a 
convention. If you write your script you can choose whatever scheme you 
like. I guess it would be better if instead plain `md:` I used `@md`, but 
either one has just a simple purpose to mark somehow nodes that I want to 
be processed.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/ee357eb1-0623-441c-a121-d10b9147a748o%40googlegroups.com.

Re: Leo Interrupt

2020-08-21 Thread vitalije

leo-ver-serv will store all the history of Leo files in the sqlite3 
database files named after the original Leo files, just with added 
'.history'. If for example you edit workbook.leo file, leo-ver-serv will 
store changes to workbook.leo.history in the same folder.

Vitalije

On Friday, August 21, 2020 at 8:53:29 PM UTC+2, k-hen wrote:
>
> Excellent, I'll check it out - thanks! - do you happen to know if that 
> history_tracer works with the sqlite/db back-end?
> Re: interrupt, that's great news too - in addition to just Leo functions, 
> if/when I get database queries working, being able to cancel the command is 
> a relatively critical use case there too :-)
>
> Kevin
>
>
> On Friday, August 21, 2020 at 1:24:43 PM UTC-4 vitalije wrote:
>
>>
>>
>> On Friday, August 21, 2020 at 4:14:43 PM UTC+2, k-hen wrote:
>>>
>>> Yes, absolutely, I'm a huge proponent of Git and it's toolchain :-)
>>> That said I want my personal comments and notes and settings *outside* 
>>> of git and not checked in as well. 
>>> So I need both regular backups/protection for Leo in _addition_ to the 
>>> Git repo which is synced with other devs.
>>>
>>>
>> I am not sure this is relevant, but there is a history_tracer plug-in 
>> which takes snapshots of your Leo files after each change. That is 
>> something like making git commit after each edit. You need to have 
>> leo-ver-serv program running and your Leo documents must be on the list of 
>> files that leo-ver-serv should keep an eye on.
>>
>> By the way, I would also like that we could interrupt Leo without killing 
>> it. On several occasions I missed that functionality. I've added support 
>> for interrupting the execute-script command in Leo, and it wouldn't be so 
>> hard to support many other commands. I don't have the time to do it myself 
>> right now, but I guess it would be sufficient to add try/except 
>> KeyboardInterrupt pair around some global method that executes commands. 
>> This might not cover all commands, but it certainly would cover a lot of 
>> them. Edward was recently refactoring key handling code and I believe there 
>> is a method something like a main command handler there.
>>
>> Vitalije
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/326dddfc-790f-4fa6-9a10-4f917ea88c1eo%40googlegroups.com.

Re: Leo Write-Only Mode

2020-08-21 Thread vitalije

On Friday, August 21, 2020 at 4:08:35 PM UTC+2, k-hen wrote:
>
> Thank you for the import script and the responses.
>
> I feel like maybe I'm missing something with @nosent though, why is it 
> much harder to use? 
>
> Let's say I have a directory of files that are each templated, i.e. 
> possibly having a header and footer. 
> These files are checked into git and synchronized using @auto nodes and 
> can both be edited and read _into_ Leo when changes occur, e.g. after git 
> pulling.
> Then I can clone the body (inner section) of these nodes and write them 
> out somewhere else using an @nosent file.
> Never would I want a change to this output file to be read back _into_ Leo.
> This file is local only and would not be checked into git, it could be 
> deleted and would always be overwritten.
> As you know, Git prefers small files rather than large ones and this one 
> could be quite massive and treated as a 'large file' and therefore lose 
> certain functionality. 
>
> Sure a script could do this too, but it's a poor man's version of using 
> includes. 
>
>
>

You can try md_docer plug-in or write your own scripts using the code from 
md_docer as an inspiration.

In almost all my Leo documents now, the first thing I do is to add a node 
with the headline '@button nsave @key=Ctrl-s'. So, whenever I press Ctrl-s 
to save my Leo document, script in this node is executed. There is alwayas 
a 'c.save()' line in this script, so that the document is saved, but then 
my script writes all nodes that need to be written in some special way. For 
example, nodes whose headline starts with '@coffee ...', are compiled with 
the coffee compiler, '@pug' nodes are compiled with the pug compiler, @css 
nodes are compiled with sass compiler, ... The limit is just your 
imagination. You can easily write your nodes without sentinels and Leo 
won't try to read file and synchronize Leo. Actually I don't have coffee, 
scss, pug and other types of files that are considered to be source files. 
I have just compiled versions written as a real files.

HTH Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/5131e5c1-413a-4acc-8802-08ee8e491f62o%40googlegroups.com.

Re: Leo Interrupt

2020-08-21 Thread vitalije

On Friday, August 21, 2020 at 4:14:43 PM UTC+2, k-hen wrote:
>
> Yes, absolutely, I'm a huge proponent of Git and it's toolchain :-)
> That said I want my personal comments and notes and settings *outside* of 
> git and not checked in as well. 
> So I need both regular backups/protection for Leo in _addition_ to the Git 
> repo which is synced with other devs.
>
>
I am not sure this is relevant, but there is a history_tracer plug-in which 
takes snapshots of your Leo files after each change. That is something like 
making git commit after each edit. You need to have leo-ver-serv program 
running and your Leo documents must be on the list of files that 
leo-ver-serv should keep an eye on.

By the way, I would also like that we could interrupt Leo without killing 
it. On several occasions I missed that functionality. I've added support 
for interrupting the execute-script command in Leo, and it wouldn't be so 
hard to support many other commands. I don't have the time to do it myself 
right now, but I guess it would be sufficient to add try/except 
KeyboardInterrupt pair around some global method that executes commands. 
This might not cover all commands, but it certainly would cover a lot of 
them. Edward was recently refactoring key handling code and I believe there 
is a method something like a main command handler there.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/0cfc9392-f434-4da8-ab98-ed2f134c7900o%40googlegroups.com.

Re: A regex puzzle

2020-08-17 Thread vitalije



On Monday, August 17, 2020 at 4:59:39 PM UTC+2, Thomas Passin wrote:
>
> Nested comments aren't allowed in js and ts, are they?
>
>
Probably not, but that was not my point.
Looking again in the original regex it seems the first group would match 
any number of sequences of triplets containing "/*" followed by any 
character. For example:
/*a/*b/*c/*d The second group will match everything that follows. In 
the given example first group doesn't match anything, and the second group 
matches whole string.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/bedf3f6a-01b4-4dfa-90bb-1b5ae417560do%40googlegroups.com.

Re: A regex puzzle

2020-08-17 Thread vitalije

re.compile(r'(/\*(:?.*?)\*/)')

The inner comment text must be grouped separately to be able to apply *? 
operator on just the inner characters. Without this grouping, *? operator 
applies to all matched characters to the left. Your regex would match 
smaller part if you have had nested comments. Like
/* first comment /* blah blah */ */



Vitalije


On Monday, August 17, 2020 at 3:49:13 PM UTC+2, Edward K. Ream wrote:
>
> I would like a regex that finds complete and *disjoint *typescript 
> multiline block comments
>
> The following does not work, because the flags do not play well together.
>
> re.compile(r'(/\*.*?\*/)(.*)', re.DOTALL | re.MULTILINE)
>
> For example, even with .*?, the pattern will match the *entire* string:
>
> /* first comment */
> body
> /* second comment */
>
> The relevant code is in the bug-1617 branch. It's not pretty.
>
> I don't know of any elegant solution. Do you?
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/0aa54cda-715a-4c60-8cf6-a36b297bba42o%40googlegroups.com.

Re: Unable to Apply Certain Commands Globally

2020-08-16 Thread vitalije



On Monday, August 17, 2020 at 5:06:51 AM UTC+2, Thomas Passin wrote:
>
>
> Similarly, if I create a path node for a directory, then double click on 
> it, Leo (I suppose it's the active path plugin, right?) will populate the 
> subtree with @auto nodes without content for each file in that directory.  
> But I can't work out how to get each node to populate with the contents of 
> the file that's already there.  I've tried using just about all the menu 
> commands in the right click menu as well as the menu bar menu, but again I 
> don't see anything happening.  What should I do to populate all the files 
> at once instead of one-by-one by double-clicking?
>

Have you tried File -> Read/Write files -> read @auto nodes? On my machine 
it is bound to "Alt-/" key. This command should read all @auto files in the 
selected subtree.
 
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/514c03de-18e2-4c93-b2bc-51e12a65eac7o%40googlegroups.com.

Re: Nodes arbitrairily contract when inserting near the root node

2020-08-16 Thread vitalije

On Thursday, August 13, 2020 at 7:21:38 PM UTC+2, Félix wrote:
>
> To Edward, or Vitalije, ... or anyone really :)
>
> (trying to debug yet another behavior in leoInteg, only to find out that 
> Leo does it in the first place, so my code was correct all along!!)
>
> I am sorry for being late, but the behavior  that you see is just a 
consequence of the position instability. When you insert a node in to the 
tree (or delete one), positions of all following siblings and their 
subtrees become invalid. Leo decides which nodes should be expanded by 
keeping track of the expanded positions. When those positions become 
invalid the information is lost. There is nothing you can do about it at 
the moment.

I've made some experiments with outline drawing code that use native qt 
tree item indexes instead of positions but that has not been merged into 
the devel branch. To be able to use qt tree item indexes instead of 
positions, Leo have to keep track of all drawn tree items and not to 
discard them on each redraw, but that is not how Leo draws tree at the 
moment.

Lately I am too busy working on some other stuff, so I can't work on Leo. 
Once I get more time, I plan to continue my work on the issue 1598, which 
might lead to some improvements in the way Leo handles positions/vnodes and 
drawing.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/fa09c653-0b92-490f-9038-2ff61d4f8c0ao%40googlegroups.com.

Re: FYI: Recent improvements to the TypeScript and Rust importers

2020-06-23 Thread vitalije


>
>
> I am studying mini_leo (a rust program) 
>
 
I am sorry for the mess that mini_leo currently is in. While studying just 
keep in mind it is far from its original idea. There is almost no 
documentation, no comments in the code and it contains traces from many 
different experiments. You might be puzzled by some code pieces that 
actually are just residuum of past experiments.

Also I plan to rewrite rust parsing code using the python code from my 
#1598 efforts in Leo (once I finish it). I believe this code is better than 
the present rust version and it handles unknownAttributes too. Anyway 
present rust code gives just a taste of what might be possible to achieve.

I haven't decided yet, which GUI framework to use for mini_leo. There are 
no standard GUI for rust applications yet, although there are many 
experiments in this area. Most likely candidate for now is using rust as 
back-end web server and creating GUI front-end in web browser. I've 
recently written several single page web applications using elm programming 
language and it was a very pleasant experience to work in elm. I plan to 
write about these experiences once I got more time. I've finally understood 
the real benefit of typed languages. I still like python and its dynamic 
typing, but now I see that creating good types can really simplify code.

With well designed types it is sometimes possible to banish all invalid 
states and make them impossible to happen. If the invalid states can't 
exist, there is no need later to check for them and code becomes much 
simpler. When there is a possibility to create an invalid state then in 
every piece of code that deals with the state there must be a check for 
these invalid states and those checks and guards are complicating every 
function. I haven't tried to use this technique in rust yet, but I guess it 
should be possible. Perhaps it is a natural thing to do in rust and I just 
missed the opportunity to use it before. I'll be writing more about this.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/e20d8039-3403-46a7-b0e7-56e47a729702o%40googlegroups.com.

Re: Loading Leo using rust

2020-06-22 Thread vitalije

I am sure I have tried it on Windows too, but I can't remember if the name 
of the dll file has some significance in Windows or not. I know that on 
Ubuntu I had to rename *libmini_leo.so* to *_minileo.so*.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/9654933f-d70b-4723-ac00-9345c017e1a1o%40googlegroups.com.

Re: Loading Leo using rust

2020-06-22 Thread vitalije

Have you tried renaming mini_leo.dll to _minileo.dll or _minileo.pyd ?
Vitalije

On Monday, June 22, 2020 at 3:43:07 PM UTC+2, Edward K. Ream wrote:
>
>
>
> On Mon, Jun 22, 2020 at 1:40 AM vitalije > 
> wrote:
>
> Actually you didn't need to reinstall rust. The rustup utility is capable 
>>> of keeping several different tool chains at once. You should have run:
>>>
>>
>> rustup default nightly
>>
>> This would set default tool chain to nightly version, and following cargo 
>> commands would use nightly version of rust.
>>
>
> Good to know. Thanks.
>
> Now I'm stuck. How do I run mini_leo?
>>>
>>>
>> mini_leo doesn't have any GUI yet, so it can't be run on its own. I 
>> haven't decided which GUI to use yet.
>>
>> At its current state mini_leo is just a python extension module. Copy 
>> mini_leo.dll somewhere on PYTHONPATH and try importing "_minileo".
>>
>
> I've tried various things and can import neither _minileo nor minileo.
>
> This can wait until you have more time.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/8d306a2e-b4ea-4dbb-a4e2-ab2e85e547d7o%40googlegroups.com.

Re: Loading Leo using rust

2020-06-22 Thread vitalije


>
>
> Installation was non-trivial. You *must* install the "nightly" version of 
> rust!
>
> It's not enough to do `rustup toolchain install nightly`. I had to do a 
> complete reinstall of rust.
>
>
Actually you didn't need to reinstall rust. The rustup utility is capable 
of keeping several different tool chains at once. You should have run:

rustup default nightly

This would set default tool chain to nightly version, and following cargo 
commands would use nightly version of rust.
 

> After that, `cargo build --lib --release` appears to have worked.
>
> > This command will build in the `target/release/libmini_leo.so` on linux 
> or `libmini_leo.dll` on Windows.
>
> Now I'm stuck. How do I run mini_leo?
>
>
mini_leo doesn't have any GUI yet, so it can't be run on its own. I haven't 
decided which GUI to use yet.

At its current state mini_leo is just a python extension module. Copy 
mini_leo.dll somewhere on PYTHONPATH and try importing "_minileo".
import _minileo
t1 = _minileo.load_leo('leo/core/LeoPyRef.leo')
print(f'total number of nodes {_minileo.tree_len(t1)}')
for level, v in _minileo.iternodes(t1):
print('-'*level, v.h)


t1 in the above example is just an int, a handle to loaded outline. Many 
_minileo functions expect an outline handle as their first argument.

Function iternodes(outlinehandle) returns a generator which yields tuples 
(level, vdata). VData instances have h, b, and gnx fields. To actually 
change tree one must use _minileo functions. Yielded vdata instances are 
just copies taken from the tree, so changing them won't change the 
original. 

Function outline_from_str(content) parses the content which should be in 
leo-thin-5 format, and returns a handle to the loaded outline. The 
outline_from_file is the same just reads content from the given file. The 
outline_from_leo_str and outline_from_leo_file functions expect xml content 
and return outline handle.

I don't have much time now to write or to work on Leo. I hope next week I 
will continue to work on the #1598 and write more about it.
I've attached small python script that demonstrate using mini_leo extension 
for loading Leo outline into Leo commander c.

HTH Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/2d665600-284b-4c66-a34f-e8f751dfaf14o%40googlegroups.com.
import sys
import os
if __name__ == '__main__':
if len(sys.argv) < 2:
print('usage: python mini_leo_demo.py ')
sys.exit(1)
LEO_INSTALL = sys.argv[1]
sys.path.append(LEO_INSTALL)

import leo.core.leoBridge as leoBridge
import leo.core.leoNodes as leoNodes
import leo.core.leoGlobals as g
from collections import defaultdict
import _minileo
import timeit
def build_tree(c, it):
'''
This function builds a tree of vnodes from the
iterable generating tuples of the following form:

(parent_gnx, gnx, childIndex,  h, b, ua)

The tuples must be in the outline order.

Returns vnode instance that is a root of this tree.
'''
gnxDict = c.fileCommands.gnxDict
# returns v node for the given gnx
# if necessary it will create a new node
def getv(gnx, h, b, ua):
v = gnxDict.get(gnx)
if v is None:
v = leoNodes.VNode(c, gnx)
v._headString = h
v._bodyString = b
if ua:
v.unknownAttributes = ua
gnxDict[gnx] = v
return v

# root is handled first, before the loop
parent_gnx, gnx, childIndex, h, b, ua = next(it)
vparent = gnxDict.get(parent_gnx)
root = getv(gnx, h, b, ua)
vparent.children.insert(childIndex, root)
root.parents.append(vparent)

# now rest of the tuples
for parent_gnx, gnx, childIndex, h, b, ua in it:
vparent = gnxDict.get(parent_gnx)
v = getv(gnx, h, b, ua)
vparent.children.insert(childIndex, v)
v.parents.append(vparent)

return root
def to_build_iterator(it):
root = next(it)[1]
stack = [root.gnx]
ccounter = defaultdict(int)
seen = set()
skip_level = 1
for lev, v in it:
gnx = v.gnx
if skip_level < lev: continue
skip_level = 1
if gnx in seen:
skip_level = lev
seen.add(gnx)
stack[lev:] = [gnx]
pgnx = stack[-2]
i = ccounter[pgnx]
ccounter[pgnx] = i + 1
yield pgnx, gnx, i, v.h, v.b, {}
def new_c():
c = g.app.newCommander('dummy.leo')
hroot = c.hiddenRootNode
c.fileCommands.gnxDict[hroot.gnx] = hroot
return c

def load_leo_file(fname):
c = new_c()
t1 = _minileo.load_leo(fname)
build_tree(c, to_build_iterator(_minileo.ite

Re: ENB: about external file format 5-thin

2020-06-06 Thread vitalije

On Saturday, June 6, 2020 at 4:18:00 PM UTC+2, Thomas Passin wrote:
>
> Edward also mentioned redundancy.  IMO, redundancy that helps in error 
> recovery is good.  Remember, there are going to be tens of thousands of 
> files in the new format eventually.  Some of them will have mis-used 
> directives, some of them will have some kind of corruption.  We need to 
> have a good chance of recovering those files anyway. 
>

While I would agree that redundancy usually means better error recovery, I 
really doubt that this can be applied here. The redundant parts that I've 
mentioned doesn't add any valuable information that could possibly be used 
for error recovery. And by the way for the redundancy to be used for error 
recovery you must have error recovery tools that can use it (which AFAIK 
Leo doesn't have). So the redundancy here means just more complexity, more 
garbage and nothing valuable in return. 

As I said before I won't insist on this change, but for the sake of being 
precise I won't let go false arguments either.

You wonder why the speed of reading and writing matters. Perhaps when you 
use Leo it doesn't matter to you if it will load 200ms faster or not. But 
If a developer wants to run thousand of tests than 20ms less actually means 
20 seconds less. Waiting 20 seconds more for tests to finish, might break 
developer's thought flow. Keeping developer's thought flow leads to better 
code. So in the end users will benefit even if they don't care about this 
micro optimizations. 

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/4dd661c1-77e3-4c25-808d-7b96afb175dao%40googlegroups.com.

Re: ENB: about external file format 5-thin

2020-06-06 Thread vitalije

>
> 6. Changing Leo's file format to make your new code easier to test would
> be letting the tail wag the dog. I am confident that you can find a robust
> testing strategy that does not depend on a new file format.
>

I wrote this post not because I couldn't make tests. The attached Leo
document contains scripts that do tests read and write functions performing
a round trip on all external files found in the Leo installation folder.
Each external file is read/parsed using a function *nodes_from_thin_file*
which is a generator yielding tuples suitable to be piped into the
*build_tree* which I wrote and tested earlier. The first testing part
compares tuple values with the values found and prepared using normal Leo's
read logic. Then test script actually builds a VNode instance representing
the whole external file and uses function *v_to_string* to generate the
content of the external file and compares the resulting content with the
source file.

I understand your unease for making this kind of change. There is nothing
urgent in my proposition. If we change write code so that it outputs
starting sentinel *@+leo-ver=6*, we can use two different functions for
parsing the rest of the file content. Old files having *@+leo-ver=5* will
be loaded using the old reading code. So there won't be any inconveniences
for users, developers and future maintainers.

Explicit is better than implicit, I agree. Then why is the node level
encoded using '*', '**', '*3*', '*4*', ...? Why is it better than just
simple '1', '2', '3', ..? Isn't the second variant more explicit?

The need for *@last* directive is a result of having *@-leo *sentinel. Try
it yourself, delete the closing Leo sentinel and all `*@@last` *lines
before it, and Leo will read this file correctly placing the last lines at
the end of the node. The closing leo sentinel doesn't add any useful
information to the reading process. But because it exists it generates a
need for the at-last directives. Which means more code to execute, more
regex searches to perform and no gains in return.

If you edit external file and separate the opening *@+others* or *@+<<
*sentinel
from the following start node sentinel (for example insert a few lines
between them), Leo will read this file correctly, but in the following
write it will report file as being changed even if user didn't change
anything. If those two sentinels are expressed *explicitly* not on their
own separate line but in the following node start sentinel as a single
character (for example "+/-" can represent the presence/absence of this
directive), there won't be possible to separate those two sentinels and we
would have two pattern less to match while reading.

Even if you prefer user being able to better understand sentinels, having
two consecutive lines containing the same *<> *text is not
helping a lot. But it does cause user to see (and read) more garbage
content.

Perhaps we could have a new setting *@int default-external-file-format=5*
by default and user can override it to 6 in myLeoSettings.leo. I am sure
format-6 would be faster to read and write and some users would prefer to
use it instead.

Anyway, I won't insist on changing the format, but if we are changing
something it would be better to make all changes at once.

Regarding the first node start sentinel, perhaps new read code can just
skip this sentinel and use the values from the xml for gnx and headline.
When writing a file, Leo can check to see if this sentinel is present in
the external file and if it is, it will keep this sentinel line unchanged.
Leo always reads existing file to check whether there is a change or not,
so this check won't be too expensive. This way single external file can be
opened using different paths in different outlines without generating
unnecessary file changes.

Or we can just skip this sentinel when writing file. This will cause a
single change to each external file, but after this no changes will ever be
caused by accessing this file from different outlines.

Vitalije

--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/9ee3ffce-95ba-4fb4-9472-7589d61a4275o%40googlegroups.com.

Re: ENB: about external file format 5-thin



> This has bothered me five or ten times when for unusual reasons I wanted 
> to @file one external file from two Leo-Editor files.  In most cases 
> this problem caused me to do something else.  In one or two cases I 
> lived with this problem. 
>
> -- 
> Segundo Bob 
> segun...@gmail.com  
>

One way to solve this issue is to add a node with the correct @path 
directive one level above the @file node. This will allow that @file node 
in both outlines have the same headline. Then it is necessary to make sure 
that these @file nodes  have the same gnx in both outlines. To achieve this 
you should copy the @file node from the one outline and then paste it 
retaining clones in the other outline. After this both outlines will 
produce the same external file.

It is not impossible to solve this problem using this trick, but it is 
cumbersome. It would be much easier if the top level gnx and headline were 
not written in the external file. Every outline could have its own gnx and 
file path, but they would produce the same output.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/cd5206e9-c6b3-4d1f-885b-452489228b80o%40googlegroups.com.

Re: ENB: about external file format 5-thin


>
>
>
> I just used @delims the other day for a Windows command file.  In cmd 
> files I use "::" as a comment marker.  I didn't find a Leo file type for 
> cmd files, so I just went ahead and used the directive.  
>

Ok, this is a valid use case, though I didn't object this kind of 
usage.This kind of directives may be skipped when writing external file. 
Which delimiters were used to write external file can (and should) be 
deduced from *@+leo* sentinel line. If those delimiters don't match 
delimiters defined for this file extension (or if there are no defaults 
like in your case), the *@delims* directive can be automatically added to 
the top level body. That way we could prevent a possibility of having 
different pairs of delimiters in a single external file. A possibility to 
create such ambiguous file is the main reason why these directives are 
considered dangerous. Handling them during the process of parsing the 
external file content makes this code complex. And I can't think of a valid 
use case for this kind of situation.

Delimiters are used in order to allow Leo sentinels to be written in the 
external file as a comment lines using the proper syntax for the given 
file. If we have two *@delims* directives with the different values inside 
one external file, this file can't be syntactically correct.

I am not against letting user to choose which delimiters to use for any 
given file. I am just suggesting that this choice should be limited to one 
set of delimiters per file. If we agree on this limitation, then the *@delims 
*directive can be used but it doesn't have to be written in the external 
file. If it is necessary (i.e. if it clashes with the default delimiters), 
then reading code would add it automatically in the top level body. Or 
perhaps it can be written just as a  flag in the *@+leo* sentinel signaling 
only that this directive was (or was not) present in the top level body. 
The delimiters deduced from the *@+leo *should be used for the entire file.

I hope I made my point a bit more clear.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/aed1ad6e-d268-4cde-b467-2dc22fce0069o%40googlegroups.com.

Re: ENB: about external file format 5-thin

I forgot to mention that round trip using new functions is 1.9 times faster 
than using c.atFileCommands. Test script compares round trip of 
leo/core/leoGlobals.py

$ python p.py
setting leoID from os.getenv('USER'): 'vitalije'
f_new average: 30.429ms
f_old average: 58.055ms

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/4e42d534-ac4c-4ec2-b30f-14487f2051dbo%40googlegroups.com.

ENB: about external file format 5-thin

For the past few days I've been working on the reusable functions for both
parsing content of external files and writing external files. In the
attached Leo document there are two new scripts. One is for generating the
test data, and the other is for testing these two new functions. All tests
are passing and round trip (*text-> outline -> text*) confirms that these
functions have almost the same effect as Leo's FastAtFile reading and
atFile writing methods.

Thinking about the format of external files and looking at them, I've come
to the conclusion that this format contains some redundant information.
This is not a big problem, but since I am currently working on this part of
the Leo's code base, I wish to propose some improvements to this format.
Having redundant information means that different files may produce the
same outline. This can cause problems when testing round trip
transformations.

First of all I have to say, that I wrote two simple scripts that can
automatically convert current external file content to the new format and
back to the original format.

- top level node gnx and its headline are not necessary. Both headline
and gnx are present in the xml. They don't provide any useful information.
This also can cause problems when two different outlines contain the same
external file. If the top level node have different path or different gnx
in those outlines than they would produce different file even if they have
the same content.
- *@+<<* sentinels are redundant too. When we encounter the node whose
headline is a section reference, we know that the section reference was
just before the opening node line.
- *@-<<* sentinel and *@afterref* can be joined in one. The section name
is not necessary because opening and closing sections must be properly
nested. We know for sure that the closingsection has the same headline
as the last open one. The closing *@-<<* sentinel can give a clue
whether the following line is *@afterref* or an ordinary line. For
example *@-<<[* means same as closing section sentinel followed by an
*@after* line, while* @-<<]* means there is no *@after *line after this
closing sentinel.
- *@+others* is not necessary because when we hit the first open node
without the section reference in its headline we know for sure that just
before this node was @others directive. Also when we encounter new open
node with the different identation we can be sure that just before this
node was *@others* directive. In the reading external file this line is
used just to push current node data on the stack. But this signal can be
added to the opening node sentinel as a single character.
- format of *@+node* sentinel can be changed so that headline comes
first and gnx and level at the end of the line for example:
#@ at.findFilesToRead:ekr.20190108054317.1:6
instead of
#@+node:ekr.20190108054317.1: *6* at.findFilesToRead
It would be nicer to read source code using other editors
- closing *@-leo* line is not necessary and there is no need for *@last*
directives either. Last lines are just last lines of the top level node.
- *@first* directive can be present in the body, but it doesn't need to
be written in the external file, because we know that all lines coming
before `*@+leo*` sentinel are first lines.

Also so called "dangerous directives" (*@comment* and *@delims*), are never
used in the Leo's code base. Personaly I can't think of the use case for
those directives. If anyone knows for a specific use case where these
directives can solve a real life problem which can't be solved without
these directives, please share it here. I wish to understand why would
anyone wish to use these directives. If no such use case can be found, I
would strongly suggest dropping support for those dangerous directives. It
would allow us to further simplify both reading and writing code.

Less sentinel lines means less parsing less ambiguity and less work which
leads to both simpler code and faster execution.

Your thoughts, please.

Vitalije

--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/0dda5ea8-b156-4ff6-a76e-5322894956f0o%40googlegroups.com.

issue-1598-experiments.leo
Description: Binary data

Re: ENB: reusable Leo functions and some experimental results (work on issue 1598)

2020-06-02 Thread vitalije



>
> I have a question - you say that the nodes_from_leo_xml()function will be 
> used for various paste commands. This way, the xml bits have to be parsed 
> each time the nodes are processed.  I would have expected that xml not be 
> used for internal communications, but only for external interactions.  I 
> realize that mostly the xml parts are short and quick to parse, but still I 
> would lean to using plain text or even python objects.  Are you doing it 
> this way mostly because Leo copies nodes as xml in the first place?
>
>
Yes, that is the main reason for using xml. But even if in the future we 
find a new format, or even several new formats, it would be enough just to 
write a function for each input format that generates tuples and then send 
those tuples to build_tree. We can treat files with Leo sentinels (@file 
external files) as another format for describing the outline shape and 
content. Parsing sentinels and decoding the outline shape and content to 
the bunch of tuples is enough for build_tree function to create the 
outline. All those reading functions for any given format can be easily 
tested. We just need to compare generated tuples with the expected ones. If 
the generated sequence of tuples is correct, then the outline will be 
correct too. Many of the tree operations are very difficult to test because 
they change a lot of things inside Leo and creating a new commander for 
each test is slow. Testing sequences of tuples is fast and easy.

In the attached Leo file, I tried at first to test 1000 round trips 
xml->outline->xml, but it was too slow when using Leo's current xml writing 
code. The technique of reusing a single Leo commander instance whose 
internals are not disturbed by the tests, allows testing to be 222 times 
faster. 

Vitalije


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/ae36be53-f973-4c52-a9b1-12afa708c46f%40googlegroups.com.

Re: ENB: reusable Leo functions and some experimental results (work on issue 1598)

2020-06-02 Thread vitalije

Here is a function that generates tuples from the xml content:
def nodes_from_leo_xml(contents):
'''
Parses contents as xml Leo document and returns
a generator of the tuples

(parent_gnx, gnx, childIndex, h, b, ua, descendentUas)

suitable to be piped to the build_tree function.
'''
xroot = read_xml(contents)
v_elements = xroot.find('vnodes')
t_elements = xroot.find('tnodes')
bodies, uas = get_bodies_and_uas(t_elements)
heads = {}
def viter(parent_gnx, i, xv):
gnx = xv.attrib.get('t')
d_uas = xv.attrib.get('descendentVnodeUnknownAttributes')
d_uas = d_uas and resolve_ua('xxx', d_uas) # key is not important 
here
h = heads.get(gnx)
if not h:
h = xv[0].text or ''
heads[gnx] = h
yield parent_gnx, gnx, i, h, bodies[gnx], uas[gnx], d_uas
for j, ch in enumerate(xv[1:]):
yield from viter(gnx, j, ch)
else:
yield parent_gnx, gnx, i, heads[gnx], bodies[gnx], uas[gnx], 
d_uas

for i, xv in enumerate(v_elements):
yield from viter('hidden-root-vnode-gnx', i, xv)



Now all three paste commands: (paste-node, paste-retaining-clones and 
paste-as-template) can reuse these functions. Each of them will use 
nodes_from_leo_xml to generate tuples and then paste-node will reassign 
on-the-fly all indices, paste-retaining-clones will pass the generated 
tuples without change, and paste-as-template will change some of the 
indices on-the-fly.

The next thing I am going to do is to write a function for generating 
tuples from the external @file files. I hope this function can be used 
inside nodes_from_leo_xml to include tuples from the @file files 
on-the-fly. That way we would have a function to fully load Leo document 
along with all the external files. The same need to be done with the 
importers: create a function which generates tuples from the @auto files. 
Handling the descendentVnodeUnknownAttributes will be the next goal after 
all kinds of external files can generate tuples.

This is far from being production ready, but it shows great potential so 
far.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/bd1f535f-9402-4ace-8c40-6ac13635369d%40googlegroups.com.

ENB: reusable Leo functions and some experimental results (work on issue 1598)

2020-06-02 Thread vitalije

I hope google will accept the attachment containing scripts.

I've started implementing some of the propositions discussed earlier in 
another thread.

First of all I think Leo should have a single function that will build an 
outline from some iterable.
def build_tree(c, it):
'''
This function builds a tree of vnodes from the
iterable generating tuples of the following form:

(parent_gnx, gnx, childIndex,  h, b, ua)

The tuples must be in the outline order.

Returns vnode instance that is a root of this tree.
'''
gnxDict = c.fileCommands.gnxDict

def getv(gnx, h, b, ua):
v = gnxDict.get(gnx)
if v is None:
v = leoNodes.VNode(c, gnx)
v._headString = h
v._bodyString = b
if ua:
v.unknownAttributes = ua
gnxDict[gnx] = v
return v

# root is handled first, before the loop
parent_gnx, gnx, childIndex, h, b, ua = next(it)
vparent = gnxDict.get(parent_gnx)
root = getv(gnx, h, b, ua)
vparent.children.insert(childIndex, root)
root.parents.append(vparent)

# now rest of the tuples
for parent_gnx, gnx, childIndex, h, b, ua in it:
vparent = gnxDict.get(parent_gnx)
v = getv(gnx, h, b, ua)
vparent.children.insert(childIndex, v)
v.parents.append(vparent)

return root


For any operation that need to build some sub-tree, it would be easier to 
implement just an iterable which yields tuples and then call build_tree. 
The build_tree should be tested and proved to work correctly. Then all 
other commands just need to test generated tuples which won't have any bad 
effect on the Leo's outline.

I have to go now. So, I'll write more later.
In the attached Leo document there are two python scripts:

   1. makedemos.py  - creates a zip file containing 1000 random Leo 
   documents
   2. testdemos.py - tests roundtrip of each of these 1000 random 
   documents. Loads file, builds tree, then generates xml from the outline and 
   compares it with the input.

The testdemos.py contains three similar scripts. The test_1 uses Leo 
c.fileCommands.putLeoFile to generate xml output. The test_2 uses new 
functions to generate xml output but for each random file creates a 
separate commander. Finally, test_3 reuses a single commander instance. For 
each random file, this commander is reset.

Running testdemos.py on my computer gives the following output:

test_1 (  10 files) -  4.891s  average:489.07ms
test_2 (1000 files) - 12.833s  average:12.83ms
test_3 (1000 files) -  2.216s  average: 2.22ms
1/2 ---> 38.11 times faster
1/3 ---> 220.71 times faster

Roundtrip xml->outline->xml using new reusable functions takes at least 38 
times less time.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/f96c2078-0b60-49a6-9146-36897135e7d9%40googlegroups.com.


issue-1598-experiments.leo
Description: Binary data

Re: Question about tnodeList attribute

2020-05-31 Thread vitalije

Thanks for the advice. 

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/db298994-e4f1-4c90-8023-df0a12e94fe5%40googlegroups.com.

Question about tnodeList attribute

2020-05-31 Thread vitalije

I've started to work on the #1598. For now I am still reading code and
trying to figure out how it works.

I have found a list of native vnode attributes:

nativeVnodeAttributes = (
'a',
'descendentTnodeUnknownAttributes',
'descendentVnodeUnknownAttributes',
'expanded', 'marks', 't', 'tnodeList',
)

However, analyzing the write code, I see that the only attributes that
are ever written are "*t*" and sometimes "*decendentVnodeUnknownAttributes*
".

The *v.tnodeList *is assigned and deleted on several places in Leo code,
but I can't find any place that it is actually used. I can't recall seeing
this attribute ever, and I don't remember what its purpose used to be.

I assume that all those ancient attributes can be safely ignored. Am I
missing something?

If those attributes can be ignored, I think Leo's reading and writing logic
can be greatly simplified.

Also there is a method FastRead.bytesToUnicode with the following __doc__:
"""
Recursively convert bytes objects in strings / lists / dicts to str
objects, thanks to TNT
http://stackoverflow.com/questions/22840092
Needed for reading Python 2.7 pickles in Python 3.4.
"""

Leo is now running on Python 3.5 or newer. Do we still need to warry about
this Python2.7 pickle compatibility? If not, code can be further simplified.

Vitalije

--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/e4da7fda-70a4-4b04-acf6-076c7d05c51b%40googlegroups.com.

Re: An implementation of paste-retaining-outside-clones command

Done at c9e92f5574cd.

This version handles unknown attributes using FastRead.scanTnodes method. 
It would be nicer if this method was a function because it really doesn't 
need neither c nor gnxDict. If it were just a plain function inside 
leoGlobals, I could use it as:

bodies, uas = g.scanTnodes(xtnodes)

instead of much uglier:

import leo.core.leoFileCommands as leoFileCommands
bodies, uas = leoFileCommands.FastRead(c, {}).scanTnodes(xtnodes)

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/b3f021f6-6f8e-480f-9d4d-a5649062cd39%40googlegroups.com.

Re: An implementation of paste-retaining-outside-clones command

I am glad to report that I've just found FastRead.scanTnodes method that I 
could use in paste-as-template. This method is a perfect example of what I 
was writing about: it provides functionality without disturbing anything 
else by returning an ordinary pair of dicts. Easy to use, easy to test.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/3dc71268-8f4e-4884-9aa1-0d49d2f1d147%40googlegroups.com.

Re: An implementation of paste-retaining-outside-clones command

Something is wrong with google groups code formatting. My code examples in 
the previous message have some line breaks that I didn't write. When I 
tried to edit that message, those line breaks disappeared and now they are 
again present.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/60585861-3645-4cad-9ce8-0c39d484e9f0%40googlegroups.com.

Re: An implementation of paste-retaining-outside-clones command

, harder to understand, and also
slower than it used to be.

I see here the fundamentaly wrong decesion to change inner code of methods
in order to make some tests against it. This kind of tests are useless
because they test not the production code but the different one. There are
many places in Leo that have `if g.unitTesting` guard. At some places they
just add some logging but there are places where this guard is not so
benign, where it changes the actual meaning of the method. I remember how
the experimental code I wrote before, whch was the inspiration for
FastAtRead, used to be testable and decoupled from everything else inside
Leo. It doesn't have to be this way.

For example *viter* generator function yields tuples of strings. It is very
easy to test it, to debug, to print the results and it won't spoil anything
inside Leo ivars. Its usage is harmless. I think both FastRead and
FastAtRead should do the same. Instead of actually creating vnodes and
manipulating them, they should generate necessary data for node creation
and linkage. Top level methods like readWithElementTree should use this
generator methods to actually create vnodes. FastAtRead.read_into_root
should also use some generator of tuples consisting of simple data types
(strings, ints, booleans) and just use those tuples to actually create and
link vnodes. That way it won't be necessary to add any kwargs for testing.
It would be trivial to test these generators and compare the values from
those tuples with expected ones. And if the generators work correctly than
we can be pretty sure that top level functions will work correctly too.

Again while writing this I've just realized that I forgot about unknown
attributes in paste-as-template.

Vitalije

--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/fb629913-6458-4e81-9752-341cdf7a5b00%40googlegroups.com.

Re: Proposal: @template nodes and related commands

2020-05-28 Thread vitalije

Look at this thread 
<https://groups.google.com/forum/#!topic/leo-editor/caTty1_Erq0>. I have 
implemented paste command that does what was asked for: keeps the identity 
of all nodes that have clones outside copied tree. Internal clones treats 
the same way as past-node.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/6ae8814a-2e5d-4845-95d5-db574f86a5ab%40googlegroups.com.

Re: An implementation of paste-retaining-outside-clones command

2020-05-28 Thread vitalije

It doesn't paste unknownAttributes but it would be trivial to add this 
functionality.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/588817f6-3562-409a-be98-9130ac872d52%40googlegroups.com.

An implementation of paste-retaining-outside-clones command

2020-05-28 Thread vitalije

This might not be the best possible name but it reflects an idea that came 
from Seth Jonson in another thread.

The idea is to clone all those nodes that are present in the outline 
outside copied tree. All nodes that are located only under the copied tree 
should not be cloned unless they have clones under the copied tree. In 
essence this is just what paste-node command does plus it keeps identity of 
all nodes that have clones elsewhere in the outline outside the copied tree.

Here is the code:

from xml.etree import ElementTree as ET
import leo.core.leoNodes as leoNodes

c.frame.log.clearTab('Log')
def paste_as_template(p):
def newgnx():
ni = g.app.nodeIndices
t_s = ni.update()
return f"{ni.userId}.{t_s}.{ni.lastIndex:d}"
def skip_root(v):
if v.gnx != root_gnx:
yield v
for ch in v.children:
yield from skip_root(ch)
def translate_gnx(gnx):
if gnx in outside:
return gnx
return newgnx()
gnx2v = c.fileCommands.gnxDict
def getv(gnx):
v = gnx2v.get(gnx)
if v is None:
return leoNodes.VNode(c, gnx), True
return v, False
def viter(pargnx, xv):
chgnx = xv.attrib.get('t')
b = bodies[chgnx]
gnx = translation.get(chgnx)
if gnx in seen:
yield pargnx, gnx, heads[gnx], b
else:
seen.add(gnx)
h = xv[0].text
heads[gnx] = h
yield pargnx, gnx, h, b
for xch in xv[1:]:
yield from viter(gnx, xch)
def dumptree(): # just for debuging
levs = {'':-1}
for pgnx, gnx, h, b in viter('', xvnodes[0]):
lev = levs[pgnx] + 1
levs[gnx] = lev
g.es(f'{"" * lev}[{gnx}]{h}')
def do_paste(vpar, index):
vpargnx = vpar.gnx
rows = viter(vpargnx, xvnodes[0])
pgnx, gnx, h, b = next(rows)
v, _ = getv(gnx)
v.h = h
v.b = b
vpar.children.insert(index, v)
v.parents.append(vpar)
pasted = v
for pgnx, gnx, h, b in rows:
v, isNew = getv(gnx)
if isNew:
v.h = h
v.b = b
vpar = getv(pgnx)[0]
vpar.children.append(v)
v.parents.append(vpar)
return pasted
def undoHelper():
v = vpar.children.pop(index)
v.parents.remove(vpar)
c.redraw(bunch.p)
def redoHelper():
vpar.children.insert(index, pasted)
pasted.parents.append(vpar)
c.redraw(newp)
xroot = ET.fromstring(g.app.gui.getTextFromClipboard())
#xroot = ET.fromstring(clipboard_text())
xvnodes = xroot.find('vnodes')
xtnodes = xroot.find('tnodes')
bodies = {x.attrib['tx']:x.text for x in xtnodes}
root_gnx = xvnodes[0].attrib.get('t')
outside = {x.gnx:x for x in skip_root(c.hiddenRootNode)}
translation = {x:translate_gnx(x) for x in bodies.keys()}
seen = set()
heads = {}
dumptree()
bunch = c.undoer.createCommonBunch(p)
if p.isExpanded():
vpar = p.v
index = 0
parStack = p.stack + [(p.v, p._childIndex)]
else:
parStack = p.stack
vpar = p.stack[-1][0] if p.stack else c.hiddenRootNode
index = p._childIndex + 1
seen.clear()
pasted = do_paste(vpar, index)
newp = leoNodes.Position(pasted, index, parStack)
bunch.undoHelper = undoHelper
bunch.redoHelper = redoHelper
bunch.undoType = 'paste-retaining-outside-clones'
c.undoer.pushBead(bunch)
c.redraw(newp)
paste_as_template(p)

The command is undoable.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/14528440-f602-4a39-8915-0ab140fd73fa%40googlegroups.com.

Re: Proposal: @template nodes and related commands

I would only suggest to leave out settings file from the proposal.

It doesn't add anything substantial and it might lead to worse user 
experience (having to open settings file to change something, then save it 
and then restart Leo or reload settings).

Instead command can look in the current outline for the @templates node and 
work the same way as you proposed. This won't require any change to the 
configParser and config classes and user would be free to change template 
and immediately see the effects. 

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/655b8fe8-c062-4900-94ef-f19c74338769%40googlegroups.com.

Re: On the Semantics of Copying Clones



> > Indeed we can look at vnodes and we'll see that each node found on the 
> > clipboard has a corresponding node with the exactly the same gnx among 
> the 
> > known vnodes (providing that we copied from the same outline and that 
> > outline has not been modified since). If we use this information to 
> decide 
> > whether node should be cloned or not then each node will be cloned and 
> we 
> > would have just what paste-retaining-clones does. 
>
>
> Right!  And you would be able to identify "internal sets" that match 
> and easily template-as-encapsulated with new vnodes, if the user 
> chooses to do the encapsulated template mode. 
>
>

I am sorry but it seems that you misunderstand my previous message. I was 
explaining why it would be useless to look in vnodes even though we can do 
that no usable information will be retrieved from such lookup. However your 
other idea (checking if the node is single or not in the whole outline) 
might work.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/fb5f0a3c-ecf7-464c-9b68-fc311e8a0c31%40googlegroups.com.

Re: On the Semantics of Copying Clones



On Wednesday, May 27, 2020 at 2:37:57 PM UTC+2, Differance wrote:
>
> On 5/27/20, Seth Johnson > wrote: 
>
> The simple approach is if there are more than one of the same 
> vnode in the outline you've grabbed to paste-as-template, those all 
> get a new vnode and become clone-nodes pointing to that instead.  If 
> they are loners within the outline you've grabbed, make them plain, 
> non-clone nodes.  
>

Hmm, this might work. I'm not 100% sure about that, but it just might work. 
Here is a function that would say if a node appears more than once in the 
outline or not. 

def is_unique(v):
while v is not c.hiddenRootNode:
if len(v.parents) == 1:
v = v.parents[0]
else:
return False
return True

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/02de850b-f2d1-4a60-ba74-0f4f5042ab66%40googlegroups.com.

Re: On the Semantics of Copying Clones

On Wednesday, May 27, 2020 at 2:09:37 PM UTC+2, Differance wrote:
>
> On 5/27/20, vitalije > wrote: 
> > @Differance 
>
> Speaking roughly, I think regardless of the structure of the tree 
> you've put in the clipboard, you can identify some that are clones of 
> each other within the tree -- can't you look at vnodes to see the 
> clones?  
>

Indeed we can look at vnodes and we'll see that each node found on the 
clipboard has a corresponding node with the exactly the same gnx among the 
known vnodes (providing that we copied from the same outline and that 
outline has not been modified since). If we use this information to decide 
whether node should be cloned or not then each node will be cloned and we 
would have just what paste-retaining-clones does. 

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/8336416d-1565-405c-928a-1a86afb09515%40googlegroups.com.

Re: On the Semantics of Copying Clones

Mark bits are really relevant only when copying node. After the copy has 
been made, the information which nodes were marked is on the clipboard and 
therefore is available to the paste-retaining-marked command. Even after 
unmark-all command as long as the clipboard is not changed, 
paste-retaining-marked will do its job correctly.

Of course there can be assigned another flag for this purpose but I've 
never used marks for anything else but for clone-marked-nodes command. 
Perhaps there is something else that this flag can be used for but I am not 
aware of it. If its main purpose is to mark nodes user wish to clone in the 
future, then it seems to me a fair use case for this command. 

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/20a753d3-1373-41b6-9dda-00fbbc808ad8%40googlegroups.com.

Re: On the Semantics of Copying Clones



On Wednesday, May 27, 2020 at 12:39:56 PM UTC+2, Edward K. Ream wrote:
>
> On Wed, May 27, 2020 at 4:56 AM Seth Johnson  > wrote:
>
> Couldn't a paste-as-template just traverse the tree and make new
>> clones where they already are clones and just copy as new nodes
>> everywhere else?
>>
>
> Much easier said than done, as I recently discovered.
>

He, he we have started our replies with the same words. 
  

> we need a new copy command, after which the paste-retaining-clones command 
> can be used as is.
>
>
I doubt that. The paste-retaining-clones will not change anything from the 
clipboard content, but the user will expect this command to create new 
nodes at least for some of the copied tree. If you devise a new 
copy-template function, then after the first paste-retaining-clones 
execution every next execution will create just the clone of the outline 
pasted the first time i.e. all nodes will be cloned not just some of them.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/4f15976a-daff-4190-aa8f-1e4fa72991ce%40googlegroups.com.

Re: On the Semantics of Copying Clones

@Differance

On Wednesday, May 27, 2020 at 11:56:57 AM UTC+2, Differance wrote:
>
> Couldn't a paste-as-template just traverse the tree and make new 
> clones where they already are clones and just copy as new nodes 
> everywhere else? 
>
> Might it be an easy enough rule to say that: 
>   
>

It is easier said than done. When you copy an outline part, Leo stores on 
the clipboard selected nodes along with their gnx fields. When pasting 
those gnx fields can be re-assigned to new values (when doing an ordinary 
paste command) or they can be left unchanged (when doing 
paste-retaining-clones). Nodes for which all clone instances are inside the 
copied tree have all the information (required to keep them as clones of 
each other) inside the clipboard content. However, nodes that have clones 
outside the copied outline have parents that the clipboard content doesn't 
know at all. Therefore some required information is lost. The 
paste-as-template command will have hard time to distinguish which of these 
nodes should be retained and which should be re-instantiated.

It is not possible to use just the gnx value to distinguish which nodes 
were previously clones because all of the copied nodes might have the 
original node still present in the outline or in the undo data and every 
node will look like it should be cloned.

It isn't possible either to just use the number of parents of the existing 
nodes to see if they should be retained because children of clones don't 
need to have more than one parent node, but they still can be found on 
several places in the outline and therefore they should be cloned.

Using marked bit of each node can help to simplify this task of 
distinguishing which nodes should retain their identity and which should 
not.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/eae72108-af6e-48a2-879e-43672a5614b4%40googlegroups.com.

Re: On the Semantics of Copying Clones