Re: SB: Please Read. Huge Ahas re unit testing

[tbp1...@gmail.com]

I'm glad to see you taking this approach.  I think that different 
development tasks should be done in separate branches.  In the past I've 
not always been careful about this, and by now I think I may finally have 
learned the lesson.

On Sunday, November 29, 2020 at 1:52:56 PM UTC-5 Edward K. Ream wrote:

> On Sunday, November 29, 2020 at 4:31:42 AM UTC-6 Edward K. Ream wrote:
>
> > The ekr-undo9 branch contains preliminary work on the new testing 
> framework.
>
> Not sure that's true. Anyway, work on this topic will be done in the 
> ekr-unit-test branch. PR 1762 
>  records the work.
>
> After some thought, I've decided to base ekr-unit-test on devel, as usual. 
> The goals are to minimize diffs, and to make the unit test work distinct 
> from the undo work. When both branches are ready, I'll merge ekr-undo9 into 
> devel first, merge devel into ekr-unit-test, and run all the new unit tests.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/f44d4cc7-9e71-40ce-912a-28e89c497ec1n%40googlegroups.com.

Re: SB: Please Read. Huge Ahas re unit testing

[Brian Theado]

Having measurable code coverage of the automated tests will be useful.

The coverage.py library is not at all tied to pytest. About a year ago I
tried running its standalone version against Leo's existing tests.

However, some of the coverage results were not making sense. Possibly the
way leo's unit test framework loads the code was causing confusion to
coverage.py. I didn't do any investigation to see if that is a reasonable
guess.

All this to say that your planned approach of extracting the tests into
standard pytests is probably the better path to reliable code coverage
measurements.

Have you considered using pytest features such as fixtures and
parameterized marks?

I tend to favor composition over inheritance and functions over classes.
The fixture feature helps with this preference.

I think your preference is the opposite, but I thought I'd ask.

On Sun, Nov 29, 2020 at 1:52 PM Edward K. Ream  wrote:

> On Sunday, November 29, 2020 at 4:31:42 AM UTC-6 Edward K. Ream wrote:
>
> > The ekr-undo9 branch contains preliminary work on the new testing
> framework.
>
> Not sure that's true. Anyway, work on this topic will be done in the
> ekr-unit-test branch. PR 1762
>  records the work.
>
> After some thought, I've decided to base ekr-unit-test on devel, as usual.
> The goals are to minimize diffs, and to make the unit test work distinct
> from the undo work. When both branches are ready, I'll merge ekr-undo9 into
> devel first, merge devel into ekr-unit-test, and run all the new unit tests.
>
> Edward
>
> --
> You received this message because you are subscribed to the Google Groups
> "leo-editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leo-editor+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/leo-editor/7debef12-60e1-4463-9dda-8bd6a1ab7dc4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAO5X8CywWr_fhJsAXh%3DGR-BfSjfdAbQ6ooYpnxmH_UZknNMn0A%40mail.gmail.com.

Re: PR's are a great success

[Félix]

Glad this workflow stuck ! its a lot simpler and easier that way! :)




On Sunday, November 29, 2020 at 10:24:01 AM UTC-5 Edward K. Ream wrote:

> There were some doubts when we first considered using PR's as part of 
> Leo's workflow. Those doubts have completely vanished. For me, PR's are all 
> gain and no pain:
>
> - There are huge advantages to having a permanent, easy-to-access, 
> authoritative record of all changes to Leo's code base. The first comment 
> of each PR gives an overview of the changes. The code diffs give the ground 
> truth of the changes. 
>
> - Creating a PR cost next to nothing. 
>
> - PR's impose discipline, including self-discipline, on code changes.
>
> I only rarely make cowboy commits (commits without a PR) to devel. Such 
> commits make trivial changes, like correcting misspellings, etc.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/8ce97f0d-558c-4a2d-bc74-0aa473b3d74an%40googlegroups.com.

Re: SB: Study units

[Edward K. Ream]

On Sun, Nov 29, 2020 at 12:36 PM Ricardo Pacheco  wrote:

I get the impression leo's user interface is more oriented towards
> individual units or nodes with their individual captions and bodies.


That's a fair statement. You can use @edit if you want to look at a file
all in one place.


> I think the amount of functionality in the code base is impressive.  It's
> nice to have an outliner with the possibility, after some work, to access
> and manipulate the tree structure, and to have options to do it thru the
> leo ui or the leobridge.
>

Thanks for the kind words. The Leonine world is like no other.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS3J3N3%2BPyaU%2BYNymG%3D-SGWTGiD3dQZodUBK0OFL%3D7k_mA%40mail.gmail.com.

Re: SB: Please Read. Huge Ahas re unit testing

[Edward K. Ream]

On Sunday, November 29, 2020 at 4:31:42 AM UTC-6 Edward K. Ream wrote:

> The ekr-undo9 branch contains preliminary work on the new testing 
framework.

Not sure that's true. Anyway, work on this topic will be done in the 
ekr-unit-test branch. PR 1762 
 records the work.

After some thought, I've decided to base ekr-unit-test on devel, as usual. 
The goals are to minimize diffs, and to make the unit test work distinct 
from the undo work. When both branches are ready, I'll merge ekr-undo9 into 
devel first, merge devel into ekr-unit-test, and run all the new unit tests.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/7debef12-60e1-4463-9dda-8bd6a1ab7dc4n%40googlegroups.com.

Re: SB: Study units

[Ricardo Pacheco]

I settled for the time being on Logseq and athens.  Logseq saves things 
to files in github and athens saves things to the local file system, just 
like leo does.  
I noticed their user interface is more like a simple text editor 
(Notepad if you use Windows 10, for example).  When making outlines, you 
can just arrow up or down and simply press tab to indent and shift-tab to 
de-indent.  I think this is very intuitive.
I get the impression leo's user interface is more oriented towards 
individual units or nodes with their individual captions and bodies.  I 
think the amount of functionality in the code base is impressive.  It's 
nice to have an outliner with the possibility, after some work, to access 
and manipulate the tree structure, and to have options to do it thru the 
leo ui or the leobridge.  

Ricardo

On Friday, November 27, 2020 at 1:47:25 PM UTC-8 Edward K. Ream wrote:

> On Thu, Nov 26, 2020 at 6:48 PM Ricardo Pacheco  wrote:
>
>> Personally, I'm curious about outliners in clojure.  These two seem 
>> interesting: 
>>
>> https://twitter.com/logseq
>>
>> https://yo-dave.com/2020/01/05/clown-a-clojure/script-outliner-with-notes/
>>
>
> Thanks for these links. Please let us know about your researches.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/e9a5a7ec-a3a9-4c3b-bd06-0d1dcb39dcffn%40googlegroups.com.

PR's are a great success

[Edward K. Ream]

There were some doubts when we first considered using PR's as part of Leo's 
workflow. Those doubts have completely vanished. For me, PR's are all gain 
and no pain:

- There are huge advantages to having a permanent, easy-to-access, 
authoritative record of all changes to Leo's code base. The first comment 
of each PR gives an overview of the changes. The code diffs give the ground 
truth of the changes. 

- Creating a PR cost next to nothing. 

- PR's impose discipline, including self-discipline, on code changes.

I only rarely make cowboy commits (commits without a PR) to devel. Such 
commits make trivial changes, like correcting misspellings, etc.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/89b3162c-32a2-4a97-b7f6-532dff258b71n%40googlegroups.com.

Re: SB: Please Read. Huge Ahas re unit testing

[Edward K. Ream]

On Sunday, November 29, 2020 at 4:31:42 AM UTC-6 Edward K. Ream wrote:

*tl;dr: *It will be *straightforward *to provide complete coverage tests 
> for most of Leo!
>

The various Ahas have straightforward consequences:

Leo will continue to support @test and @suite nodes. Eventually, leoTest.py 
will contain only the minimum code to support *generic* @test and @suite 
nodes.

Considerable code will migrate from leoTest.py to leoTest2.py. leoTest2.py 
will also contain new code and classes. As before, most of Leo's unit tests 
must be aware of Leo's DOM and API. To make that happen, leoTest2.py will 
define test classes that define off-screen *test commanders*. leoTest.py 
already does this. The corresponding code in leoTest2.py should be 
substantially simpler because the code will not be running *inside *Leo.

Oh joy. The distinction between running unit tests internally or externally 
will disappear.

leoTest2.py must provide a way of running all unit tests. It will likely do 
that by looking for source files with a top-level "pytest_main" method.

20+ years of hindsight should improve leoTest2.py. For example, it's now 
clear that most unit tests can and should be run with a "null" 
(string-based) gui. leoTest2.py must also support desperately needed unit 
tests of the Qt gui. I'll be experimenting with running tests in an 
"off-screen" Qt gui.

Let me emphasize several points:

1. The conversion script 
 
is dead simple, both in concept and in code. The script extracts data from 
a tree of nodes copied from unitTest.py, and uses that data as kwargs to a 
"run_test" method of a test class. Neither the run_test method nor the test 
class exists at present, but that's a nit :-) I'll adapt that script to 
convert other @test nodes in unitTest.py. Other @test nodes can just be 
copied to their own unit tests.

Aside: The new unit tests will not have automatic access to c and g and p. 
However, that won't change much, because the various setup and teardown 
methods can compensate. Surely, it's no big deal.

2. The sabbatical will continue. The main focus of my sabbatical remains 
learning new web technologies. But now a secondary focus will be to 
revolutionize Leo's unit tests.

In the next few weeks, I *will *create leoTest2.py and create new unit 
tests covering most of Leo's commands. PR 1759 
 tells which files were 
changed. To that list I'll add files changed in the ekr-undo9 branch.

Other work can wait. I am not going to commit to covering all of Leo's 
source code immediately. Some unit tests can wait. Otoh, full coverage 
testing of Leo's code base will be a defining feature of Leo 6.4.

*Summary*

No new significant invention is needed to make full coverage testing of 
Leo's code base a reality.

The sabbatical will continue, now with twin objectives.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/0c87a970-e16e-45be-9244-50fc8e2a012en%40googlegroups.com.

SB: Please Read. Huge Ahas re unit testing

[Edward K. Ream]

Yesterday will surely rank as one of the most important days in Leo's 
history.

*tl;dr: *It will be *straightforward *to provide complete coverage tests 
for most of Leo!

Like all momentous Ahas, it has suddenly become difficult to remember what 
the world looked like before the Aha. I am writing this post in the middle 
of the night, attempting to recreate the Aha, explain the way forward, and 
get back to sleep :-)

*A slow-motion Aha*

Yesterday I thought I was finally done with work on Leo. At last, I could 
start studying web technologies. Indeed, yesterday I completed the "Hello 
World" example for VS Code plugins 
. But 
then fate intervened.

My subconscious nagged me about completing the elimination of 
c.onBodyChanged. This is #1733 
. Yesterday I 
reopened this item, removed the "Won'tDo" label, and reassigned it to Leo 
6.4. I did this because I saw that after simplifying c.onBodyChanged it 
would be easy to replace c.onBodyChanged with a *small* amount of code, 
given in #1733.

I eliminated several calls to c.onBodyChanged using this strategy.  I ran 
all unit tests, and they all passed. So far, seemingly so good.

But now my subconscious started screaming at me: "Why are you doing this 
work? You have no idea whether your changes have been covered by unit 
tests. Admit it. You are flying blind."

According to github, I created #1758 
 (Cover most of Leo 
with unit tests) *two days *ago*.* I am quite sure that when I created this 
item I did not have any idea how to do it! But presumably, this item primed 
my mental pump. That's probably why my subconscious was screaming at me 
yesterday.

Yesterday afternoon Rebecca and I took a long walk in some lovely woods 
overlooking the University of Wisconsin. After the walk, I saw the start of 
the way forward.

Strangely, I don't remember the exact moment. I remember discussing unit 
testing with Rebecca after the walk. Before dinner, I wrote a script to 
convert unit tests for edit commands in unitTest.leo to "traditional" unit 
tests.

It was only after discussing that script with Rebecca at dinner that I 
started to realize what I had just done! As I talked, I got more and more 
excited. The building excitement was similar to the "hand-waving Aha" about 
synchronizing tokens in leoAst.py.


*The first conversion script*

The second comment of #1758 
 
shows the first conversion script. The setup is:

- Copy @file activeUnitTests.txt-->Organized by file-->leoEditCommands
  from unitTest.leo to the top-level node of leoPy.leo.
- Create a new top-level node called 'new-tests'.
- Run the script.

The second comment also shows the output of the script, which is a 
"traditional" unit test.

*Important:* The script creates a list of Leo *nodes*. I'll copy these 
nodes to leoEditCommands.py. But like all other nodes in @file trees, the 
nodes create *flat text* in the external file. This means...

*Aha 1*: pytest does not have to know about Leo outlines!

*Support classes*

A new file, leo/core/leoTest2.py, will define the classes and helpers 
needed to make the new tests work. I'll adjust the conversion script to 
bring the generated nodes into "alignment" with the code in leoTest.py.

*Aha 2*: The kinds of support that leoTest2.py is much simpler than that 
contained in leoTest.py!

leoTest.py *creates* "flat" unit tests for python's unittest module. It's 
messy. In contrast, leoTest2.py just needs to define some base classes and 
simple support helpers.

*Aha 3*: here is *no way* to run coverage tests from unitTest.leo. 
unitTest.leo must go.

None of these three Ahas were clear to me until my dinner with my muse :-)

*Summary*

Running coverage tests is the crucial first step. Coverage failures will 
guide the testing process. 

Scripts will convert unit tests in unitTest.leo to corresponding nodes 
containing traditional unit tests. I'll copy the generated nodes to the 
corresponding source files. The actual unit tests will be flat text in the 
external source file.

A new file, leoTest2.py, will provide support for the new testing 
framework. leoTest2.py will be *much* simpler than leoTest.py because it 
doesn't need to convert outlines to tests.

unitTest.leo must go. It was a useful dead end.

The *ekr-undo9 branch* contains preliminary work on the new testing 
framework.

I had expected that my sabbatical would be refreshing. I did not expect a 
huge breakthrough in Leo's world.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit