Re: Status report and a big surprise

[Edward K. Ream]

On Thursday, January 23, 2020 at 5:49:48 AM UTC-5, Edward K. Ream wrote:

*> A simple script uses asttokens to replace all of the previous several 
month's work!*

I have just updated the unit tests that compares running times of the TOG 
class and asttokens. The code in leoAst.py is almost three times faster 
than asttokens, after counting the time to run the "simple script".

leoAst.py does more "out of the box" than asttokens, so leoAst.py is still 
in the running as a major new python tool.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/08cf7af5-36b1-4c84-a42c-9b6f58158f7f%40googlegroups.com.

Re: ENB: Why didn't I, and black and fstring, use asttokens?

[Edward K. Ream]

On Thursday, January 23, 2020 at 6:58:33 AM UTC-5, Edward K. Ream wrote:

This Engineering Notebook post explains why deep knowledge of the problem 
> domain was needed to get to the surprising script. This post also explains 
> some parts of the script in detail.
>
...

The Aha: yesterday I saw that the code:
>
> if not isinstance(node, ast.Slice):
>
> could be replaced by:
>
> if not any(isinstance(z, ast.Slice) for z in self.token.node_list):
>

This might introduce subtle bugs. A cff on gen_token(':') shows 16 hits. 
Most of the hits are for nodes, like ast.ClassDef and ast.FunctionDef, that 
are not valid within slices.  But nodes like ast.Dict, ast.DictComp, 
ast.Lambda and ast.Slice itself *are* valid within slices.

So the proper code might be:

if isinstance(self.token.node_list[-1]), ast.Slice)

This relies on a property that *probably* is true, namely that 
asttokens.util.walk yields nodes in a "good enough" order. It likely does 
most of the time ;-).  However, the TOG class *guarantees *that 
self.token.node is correct.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/98d2f4df-12bb-49b7-ac51-7a41cadc1f60%40googlegroups.com.

Re: Leo for organizing notes?

[Matt Wilkie]

>
> I have one specific question: Can I have a Leo file appear as a 
> headline in another Leo file, so I can hotkey the headline and have the 
> separate file appear? If so, can I assume I can get infinite 
> "drilldown?" It would be great to have a "master" Leo file from which I 
> can get to other Leo files, directly or indirectly. 
>

You can get partway there using Uniform Node Location (UNL) links,

For example put something like this in headline or body and ctrl-2x-click 
on it:

unl://D:/code/studies/study-shortcutter.leo

The draw back is that they must be full path (it doesn't take into account 
any @path statements you might have in the current branch).

[later] After reading http://leoeditor.com/slides/plugins.html#unl-py I see 
there is also @url for use in headlines. It's smart enough to open a 
specific node, not just the file

@url 
D:/code/studies/study-shortcutter.leo#MyChanges-->create_shortcut-->Docs

I'm not sure what the relationship is between these two forms.

There was a lot of discusion a few years ago on "cross file links" that 
might yield other gems.

-matt

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/96000d57-6022-42f5-9ab9-4cc1a561d216%40googlegroups.com.

Re: Problem with "@@" headlines?

[Edward K. Ream]

On Tue, Jan 21, 2020 at 1:45 PM John Hutchinson 
wrote:

>
> Having a tree
>
> @asis _foo.txt
> @@ Headline
> Some text
>
> and trying to save the file gives the error [snip]
>

Thanks for this report. I have just created #1480
 for this.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS3Sk%3DPUUh9xSj%3DbVRii_BVXV2DmCMTPzNwj9j9TiDiOdQ%40mail.gmail.com.

Re: mini update and leoBridge Newbie question

[Edward K. Ream]

On Wed, Jan 22, 2020 at 7:07 PM Félix  wrote:

> (mini update for those following this vscode extension : Still working on
> leo integration into vscode I'm currently re-doing the clone/insert/delete
> functions to my satisfaction for the 3rd time before making a release to
> the master branch hopefully in jannuary!)
>

That's good news.

> How would I go about intercepting entries to the log pane in leobridge?

The g.es function in leoGlobals.py writes to the log pane. In the end, it
writes to app.log.

You will probably have to set up app.log so it writes to your log.

Aside: g.es has accumulated a lot of cruft. It might be best to start with
the simpler version in leoAst.py. To do that, you would likely have to
monkey-patch g.es.

HTH. Please feel free to ask more questions.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS1TC%3DYY4yfCMTRiXNUBPaRtj3dm9uoBvq3Afrswis_4Ww%40mail.gmail.com.

Re: Leo for organizing notes?

[Chris George]

Hi Steve,

I never discovered a way to do that. In fact, for the first seven years of
using Leo, I used only one Leo file to organize my life and handle all of
my different writing projects.

Chris

On Thu, Jan 23, 2020 at 12:26 AM Steve Litt 
wrote:

> On Sat, 18 Jan 2020 17:44:51 -0800
> Chris George  wrote:
>
> > On Sat, Jan 18, 2020 at 3:36 PM andyjim  wrote:
>
> > > My intended usage for Leo is organizing notes & ideas. I'm raising
> > > my hand here because perhaps Leo's outlining/organizing
> > > capabilities may be what I need. I'm hoping folks here can tell me
> > > if I'm even knocking on the right door by looking at Leo. But it
> > > looks like Leo's flexibility in outlining may be unsurpassed and
> > > may be what I'm looking for. Hope so.
> >
> > Leo can be without peer when it comes to organizing notes and ideas.
>
> Hi Chris,
>
> I have one specific question: Can I have a Leo file appear as a
> headline in another Leo file, so I can hotkey the headline and have the
> separate file appear? If so, can I assume I can get infinite
> "drilldown?" It would be great to have a "master" Leo file from which I
> can get to other Leo files, directly or indirectly.
>
> Thanks,
>
> SteveT
>
> Steve Litt
> January 2020 featured book: Troubleshooting: Just the Facts
> http://www.troubleshooters.com/tjust
>
> --
> You received this message because you are subscribed to the Google Groups
> "leo-editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leo-editor+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/leo-editor/20200123032620.336cf5a7%40mydesk.domain.cxm
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CADWQas3Y67OcT%3DJvkQaLMqx0D4bLo%2BgRpUoZiVQGxzLYjY5dEg%40mail.gmail.com.

Re: Programming now feels like playing a video game!

[Edward K. Ream]

On Thu, Jan 23, 2020 at 7:51 AM john lunzer  wrote:

> Not quite exact. The precise and important difference is that in almost
> all video games somebody else has written the unit and coverage tests for
> you. Writing the tests and coverage yourself would be akin to setting your
> own goals which is rare in video games.
>

To me, it feels like I have a guide who is always looking over my shoulders.

If the goal is to make programming more video game like you may have
> exposed a worthy programming pattern: write tests, but not for yourself;
> and write code, but not to pass tests you've written. This could be done in
> pairs or in a larger group. In fact I can see how this could also make
> writing tests for existing code more fun as there would be a challenge in
> finding holes in other people's code.
>

I like the pattern you propose. However, making programming more like a
video game is not my goal. I was only explaining why programming feels even
more immersive.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS03pBv3o%2BxhBSv7ecHj8ONuwHwwhYfaTpHRpc1CsdyTPg%40mail.gmail.com.

Re: Programming now feels like playing a video game!

[john lunzer]

Not quite exact. The precise and important difference is that in almost all 
video games somebody else has written the unit and coverage tests for you. 
Writing the tests and coverage yourself would be akin to setting your own 
goals which is rare in video games. 

If the goal is to make programming more video game like you may have 
exposed a worthy programming pattern: write tests, but not for yourself; 
and write code, but not to pass tests you've written. This could be done in 
pairs or in a larger group. In fact I can see how this could also make 
writing tests for existing code more fun as there would be a challenge in 
finding holes in other people's code.

On Saturday, January 18, 2020 at 7:07:38 AM UTC-5, Edward K. Ream wrote:
>
> The recent Aha's re the connection between unit testing and coverage 
> testing have turned computer programming into an utterly absorbing 
> adventure. Computer programming now feels like playing a video game!
>
> The analogy is exact!
>
> In a video game, you have more or less clearly defined goals. (In some 
> games the goal is to figure out what the goal is.) To attain those goals, 
> there is a well defined set of actions you must take. In well-crafted 
> games, those actions are pleasurable in themselves, or at least not utterly 
> tedious. There are also a set of obstacles, which make attaining the goals 
> non-trivial. There is also usually a score that measures progress. All 
> these combine to make a computer game it's own little universe.
>
> Likewise with computer programming. You have more or less clearly defined 
> goals. There are also actions, obstacles, and now, with coverage testing, a 
> score. Programming is now it's own *structured* universe. The goal gets 
> translated to unit tests, which all must pass with 100% coverage.
>
> *Summary*
>
> Unit tests translate a possibly ill-defined goal into something concrete, 
> specific and well defined. The task is, by definition, complete when all 
> unit tests pass with complete coverage.
>
> Coverage testing allows uncovered code to be discarded when all relevant 
> tests pass. This was a completely unexpected, and most welcome, development.
>
> The entwined tasks of creating unit tests and ensuring complete coverage 
> almost completely structures the task of programming. As a result, 
> programming is *more* engaging and compelling than ever before.
>
> Folks, this is a big deal.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/6c2df975-00b4-4ac9-859f-73a77393b8cb%40googlegroups.com.

Travelling, and other plans

[Edward K. Ream]

Rebecca and I will be leaving Florida tomorrow. I expect to spend about 5 
days on the road, so we should arrive back in Madison by February 1.

When I arrive home the first task will be to complete the split/join logic 
in the Orange class. This will complete #1440. It would be unbearable to 
leave this work unfinished. I'll then decide whether to announce the work 
to the python devs.

I shall then complete all the other items scheduled for Leo 6.1.1. There 
are too many outstanding bugs for my liking.

Next, I want to create proper unit tests (outside of unitTest.leo, inside 
the respective files) for Leo's most important files: leoNodes.py, 
leoAtFile.py, and leoFileCommands.py.

When all this is complete I'll start looking around for juicy new projects. 
Investigating web assembly and its relationship to python is high on the 
list.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/c0bc6819-2a5b-43ca-a8b7-fc45e06a4f42%40googlegroups.com.

ENB: Why didn't I, and black and fstring, use asttokens?

[Edward K. Ream]

I said that all the work on #1440 has been valuable, even though a simple 
script might use asttokens to do everything that the code in leoAst.py does.

This Engineering Notebook post explains why deep knowledge of the problem 
domain was needed to get to the surprising script. This post also explains 
some parts of the script in detail. As with all ENB posts, feel free to 
ignore it.

At no time was I upset by the surprise. I immediately treated it as *good* 
news. asttokens now provides a valuable point of comparison and context. 
The work I have done has given me deep insights into the subtle, 
behind-the-scenes, complications involved.

*Why did I, and black, and fstringify miss this possibility?*

In retrospect, it's clear why the Aha is easy to miss:

1. I didn't know until yesterday what data would be needed. It's impossible 
to know what would work until you know exactly what data will be needed. 
It's just all too confusing.

2. I have been assuming all along that *exact* traversal order would 
(ultimately) be required. But that not at all true. Indeed, in some cases 
*random* traversal suffices.

The Fstringify code in leoAst.py is an example. The ast.BinOp visitor would 
work if visited in *any* order, because potential f-strings are disjoint. 
However, we actually want the BinOp visitor to be visited in the 
approximate source-code order those ops appear in the sources, because 
Fstringify produces log messages, and we don't want *those* messages to be 
scrambled ;-)

3. [The big one]. I have been assuming that an exact, 1-to-1, 
correspondence between tokens and ast nodes is needed. Wrong, wrong, wrong! 
We can tolerate many-to-many links between tokens and nodes. That is, many 
nodes might point at a single token, and a single token might point at many 
nodes.

This is what I saw yesterday while discussing links with Rebecca. Iirc, I 
saw that crucial test in o.colon would work just fine with a many-to-many 
mapping between tokens and nodes. I've shown this crucial code before.  
Here it is again:

def colon(self, val):
"""Handle a colon."""
node = self.token.node
self.clean('blank')
if not isinstance(node, ast.Slice):
self.add_token('op', val)
self.blank()
return
# A slice.
[snip]

The Aha: yesterday I saw that the code:

if not isinstance(node, ast.Slice):

could be replaced by:

if not any(isinstance(z, ast.Slice) for z in self.token.node_list):

Let's see how token.node_list can be computed...

*The asttokens script*

First, we create a list of *mutable *Token objects.  asttokens uses only 
the named tuples provided by tokenize.tokenize. Named tuples are immutable, 
so the script must create an auxiliary list. The Token class is simple. No 
need to show it here.

atok = asttokens.ASTTokens(source, parse=True)
tokens = [Token(atok_name(z), atok_value(z)) for z in atok.tokens]

Given this list of Token objects, it's a snap to create the token lists:

for node in asttokens.util.walk(atok.tree):
for ast_token in atok.get_tokens(node, include_extra=True):
i = ast_token.index
token = tokens[i]
token.node_list.append(node)

That's all there is to it. It's also straightforward to inject parent/child 
links into ast nodes. See the actual script for details.

*Summary*

It takes deep insight to realize that asttokens could replace the TOG and 
TOT classes. This is the reason I was happy to see this possibility.

In any event, the TOG and TOT classes are still valuable. They are faster 
and clearer (in most ways) than the asttokens code. Otoh, the asttokens 
code could be said to be more clever. The new insights promise new ways to 
simplify the code in leoAst.py using clever asttokens code.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/efae61fa-f3bb-4826-8f1f-446045545ae7%40googlegroups.com.

Status report and a big surprise

[Edward K. Ream]

Work is going well on the last part of #1440 
.

The last task is to use this data while joining several lines into one 
longer line. The Orange class now have access to all needed data.  Some of 
this data comes from tokens, some from the ast (parse tree). This was 
tricky work. Happily, the code continues to simplify.

More importantly, I now understand completely what data are actually 
needed. As I was explaining the situation to Rebecca at breakfast 
yesterday, I realized that that the asttokens tool might provide *all* this 
data. An hour's work showed that this suspicion was correct. *A simple 
script uses asttokens to replace all of the previous several month's work!*

The show_asttokens_script unit test contains this script. This script is 
not run automatically because its name does not start with 'test'.

This is *good* news. All the work on #1440 has been valuable, for many 
reasons. The insights that lead to the script are subtle, as I'll explain 
in another post.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/06f9870b-0390-40f0-9e09-964daa9eea7f%40googlegroups.com.

Re: Leo for organizing notes?

[Steve Litt]

On Tue, 21 Jan 2020 10:04:39 -0800 (PST)
Israel Hands  wrote:

> Hi Andy,
> 
> File Menu / Read/Write Files / Read-file-into-node will suck in the
> file you point it at. Leo won't understand a word file so start with
> your text files.

What if he wants the original file to be separate, and just hotkey a
Leo headline to open that file in its natural program (LibreOffice for
instance)? Can you just have a headline with the filename and the
program to open it up with, and hotkey that headline?

Thanks,
 
SteveT

Steve Litt 
January 2020 featured book: Troubleshooting: Just the Facts
http://www.troubleshooters.com/tjust

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/20200123033009.32417bb8%40mydesk.domain.cxm.

Re: Leo for organizing notes?

[Steve Litt]

On Sat, 18 Jan 2020 17:44:51 -0800
Chris George  wrote:

> On Sat, Jan 18, 2020 at 3:36 PM andyjim  wrote:

> > My intended usage for Leo is organizing notes & ideas. I'm raising
> > my hand here because perhaps Leo's outlining/organizing
> > capabilities may be what I need. I'm hoping folks here can tell me
> > if I'm even knocking on the right door by looking at Leo. But it
> > looks like Leo's flexibility in outlining may be unsurpassed and
> > may be what I'm looking for. Hope so. 
> 
> Leo can be without peer when it comes to organizing notes and ideas.

Hi Chris,

I have one specific question: Can I have a Leo file appear as a
headline in another Leo file, so I can hotkey the headline and have the
separate file appear? If so, can I assume I can get infinite
"drilldown?" It would be great to have a "master" Leo file from which I
can get to other Leo files, directly or indirectly.

Thanks,

SteveT

Steve Litt 
January 2020 featured book: Troubleshooting: Just the Facts
http://www.troubleshooters.com/tjust

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/20200123032620.336cf5a7%40mydesk.domain.cxm.