Re: Discuss: a proposed answer to python issue #33337

[Edward K. Ream]

On Wed, Jan 15, 2020 at 2:45 PM Brad  wrote:

This is very interesting work.
>

Thank you.

> As you probably know, the Python core developers are a bit curmudgeonly
> when it comes to suggestions that haven't come from themselves.
>

Hehe. I have some experience with that :-)

Be aware that the responses could focus exclusively on all of the potential
> problems, with the inevitable: "Why don't you put this on PyPI first and
> see how it is received?"
>

An excellent idea, even if it does come from the python devs ;-).  Matt is
working on the packaging.

Thanks for your feedback.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS1OnELGp7eL8ou2p6z_nZHHRfy2tVSmr2VTVjAiXWD9-w%40mail.gmail.com.

Re: Discuss: a proposed answer to python issue #33337

[Brad]

Edward, 

This is very interesting work. 

As you probably know, the Python core developers are a bit curmudgeonly 
when it comes to suggestions that haven't come from themselves. 
Be aware that the responses could focus exclusively on all of the potential 
problems, with the inevitable: "Why don't you put this on PyPI first and 
see how it is received?"

-Brad

On Wednesday, January 15, 2020 at 12:42:31 AM UTC-7, Edward K. Ream wrote:
>
> On Tue, Jan 14, 2020 at 8:00 PM Matt Wilkie  > wrote:
>
> > you've worked incredibly hard to this point, and it must be really 
> exciting/enticing to be near a point of release and of shouting "hear ye, 
> hear ye" loud enough to attract attention, but don't make noise just yet!
>
> I agree that more work is needed.
>
> At the very least I think leoAst needs to be run on it's own (e.g. not 
>> rely on `import leo ...`), and then have a concrete quick start example or 
>> two like Terry and Btheado mentioned.
>>
>
> A quick-start example is a good idea. The first that comes to mind is the 
> code that would fstringify a file. leoBeautify.py contains three fstringify 
> commands that could be reworked. However, those commands do "too much" in 
> some sense.
>
> The challenge is to create a motivating example, which can be solved with 
> only a few lines of code. That's a big ask, because I imagine anything 
> useful might run to a few pages of code
>
> I think the Python devs (and everyone else) need to be told in more detail 
> why the tool is useful. That involves a discussion of what "unify the token 
> and ast worlds" means, and why it would be useful. That's what I'll discuss 
> in an upcoming post.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/69f49c9f-0058-4ef9-93f7-91c80abcf81e%40googlegroups.com.

Re: Discuss: a proposed answer to python issue #33337

[Edward K. Ream]

On Tuesday, January 14, 2020 at 3:00:44 PM UTC-5, Terry Brown wrote:

I wonder if a couple of demos would help
>

I'd like to thank you, Brian and Matt for your comments. Those comments 
have helped me not to make a fool of myself straightaway :-)

Something shocking has just happened. I'll discuss what's in a separate 
thread.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/1693b98c-0748-4562-88ba-183e65acf99a%40googlegroups.com.

Re: Discuss: a proposed answer to python issue #33337

[Edward K. Ream]

On Tue, Jan 14, 2020 at 8:00 PM Matt Wilkie  wrote:

> you've worked incredibly hard to this point, and it must be really
exciting/enticing to be near a point of release and of shouting "hear ye,
hear ye" loud enough to attract attention, but don't make noise just yet!

I agree that more work is needed.

At the very least I think leoAst needs to be run on it's own (e.g. not rely
> on `import leo ...`), and then have a concrete quick start example or two
> like Terry and Btheado mentioned.
>

A quick-start example is a good idea. The first that comes to mind is the
code that would fstringify a file. leoBeautify.py contains three fstringify
commands that could be reworked. However, those commands do "too much" in
some sense.

The challenge is to create a motivating example, which can be solved with
only a few lines of code. That's a big ask, because I imagine anything
useful might run to a few pages of code

I think the Python devs (and everyone else) need to be told in more detail
why the tool is useful. That involves a discussion of what "unify the token
and ast worlds" means, and why it would be useful. That's what I'll discuss
in an upcoming post.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS1YL8gYXxAwD_rN4Qo0y3XVCu3zfS5vg3gq1py22G35yQ%40mail.gmail.com.

Re: Discuss: a proposed answer to python issue #33337

[Edward K. Ream]

> Thanks for this suggestion.  Imo, the demo is the TOG class compared with
>> similar code in asttokens, fstringify and black. The differences are
>> striking.
>>
>
> This puts the burden on all the readers to go look at the two versions of
> fstringify (and other) code and be able to figure out on their own exactly
> what striking differences you are referring to.
>

Yeah, I agree with you and Terry that there is a hole in the docs, and in
my proposed letter.

> If you do the work to present the differences in an easy to read way, then
> all readers will benefit. Are there small pieces you can put side-by-side
> which show the improvements?
>

I don't think snippets will plug the hole. What's needed is some discussion
of what this tool is actually good for. This is something that I probably
have never discussed, because it's so clear to me. I'll attempt a proper
answer in an upcoming post.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS1DOo9kf85rPP5X_OBkGCJOOZ%2B0zYDPsZ2mnJrt4Fw1og%40mail.gmail.com.

Re: Discuss: a proposed answer to python issue #33337

[Matt Wilkie]

>
> Here is the first draft of a reply to Python issue #7 
> . This long-dormant issue discusses 
> possible improvements to python tools.
>
> I would appreciate it if you would read the thread in detail, and let me 
> know if the following reply seems on topic and useful. Thanks.
>

I still need to read through with attention, but my first blush feedback 
is: you've worked incredibly hard to this point, and it must be really 
exciting/enticing to be near a point of release and of shouting "hear ye, 
hear ye" loud enough to attract attention, but don't make noise just yet!

At the very least I think leoAst needs to be run on it's own (e.g. not rely 
on `import leo ...`), and then have a concrete quick start example or two 
like Terry and Btheado mentioned.

-matt

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/03595ad7-a789-4c7b-8dda-907fa703a3d4%40googlegroups.com.

Re: Discuss: a proposed answer to python issue #33337

[Brian Theado]

On Tue, Jan 14, 2020 at 4:05 PM Edward K. Ream  wrote:

> On Tue, Jan 14, 2020 at 3:00 PM Terry Brown  wrote:
>
>> I wonder if a couple of demos would help,
>>
>
> Thanks for this suggestion.  Imo, the demo is the TOG class compared with
> similar code in asttokens, fstringify and black. The differences are
> striking.
>

This puts the burden on all the readers to go look at the two versions of
fstringify (and other) code and be able to figure out on their own exactly
what striking differences you are referring to. If you do the work to
present the differences in an easy to read way, then all readers will
benefit. Are there small pieces you can put side-by-side which show the
improvements? But that's extra work you may not want to do and maybe it is
hard to do. I'm just sharing my thoughts.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAO5X8Cz%3DM3b4sJDerTwy%2BfmFPs_xDeSDrrXhr2XgDjvzS1BVLg%40mail.gmail.com.

Re: Discuss: a proposed answer to python issue #33337

[Edward K. Ream]

On Tue, Jan 14, 2020 at 3:00 PM Terry Brown  wrote:

> I wonder if a couple of demos would help,
>

Thanks for this suggestion.  Imo, the demo is the TOG class compared with
similar code in asttokens, fstringify and black. The differences are
striking.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS3W0PNQyE7Sn2Vag5ftkXGCzyUjtraPMv9u-Eg0ZZ8jSw%40mail.gmail.com.

Re: Discuss: a proposed answer to python issue #33337

[Terry Brown]

I wonder if a couple of demos would help, "see how much better this is"
rather than "ask me about how much better this is".  Maybe a comparison
like: https://gist.github.com/kennethreitz/973705
which compares `urllib2` to `requests` code to do the same thing.
And perhaps some context like "I want to know if this seems like a
candidate for inclusion before I put together all the integrated docs. /
tests / implementation pull request" to make it obvious you not expecting
someone else to do all that.

A slightly different route might be to do the above demo / comparison stuff
in a fully fledged package on GitHub / pypi / readthedocs so that you can
get some feedback from people trying it out then propose getting into core
libs by referencing the package.  Maybe you've already started in that
direction, not sure.

Cheers -Terry

On Tue, Jan 14, 2020 at 12:30 PM Edward K. Ream  wrote:

> Here is the first draft of a reply to Python issue #7
> . This long-dormant issue discusses
> possible improvements to python tools.
>
> I would appreciate it if you would read the thread in detail, and let me
> know if the following reply seems on topic and useful. Thanks.
>
> = Start Post
>
> > So if users want to write tools that manipulate Python code, the standard
> library doesn't provide them with great options.
>
> > I suggest the following plan:
>
> Hello all,
>
> I would like to suggest another possibility.
>
> For about three months I have been obsessed with the general problem of
> manipulating Python source code *easily*. I have long experience with such
> tools.  Along the way I have studied the
> [asttokens](https://pypi.org/project/asttokens/),
> [fstringify](https://pypi.org/project/fstringify/), and
> [black](https://pypi.org/project/black/) projects.
> Imo, each in its own way illustrates that something significantly better
> would be welcome. Each project uses complex, idiosyncratic, opaque
> generators to generate results. What if something much simpler were
> possible?
>
> In late October, 2019 I completed a token-based version of fstringify. I
> could have stopped there, but then a new thought intruded. What if we could
> define a **token-order traversal** of parse trees? This new idea sparked an
> avalanche of new ideas. I couldn't let them go.
>
> After three months of literally day and night development this single idea
> has turned into a major new tool, contained at present in [leoAst.py](
> https://github.com/leo-editor/leo-editor/blob/fstrings/leo/core/leoAst.py).
> The code is simple, general, flexible, and fast, in stark contrast to the
> tools mentioned above.
>
> Leo's [issue #1440](https://github.com/leo-editor/leo-editor/issues/1440)
> (Unify the ast and token worlds)
> contains extensive documentation for this project. The first comment
> contains an overview of the project, the second comment contains a Theory
> of Operation, and the third contains a History of the Project, with links
> to the many Engineering Notebook (ENB) post that I wrote along the way.
>
> Imo, the TokenOrderGenerator (TOG) and TokenOrderTraverser (TOT) might be
> candidates for inclusion in the standard library, but first you Python devs
> will want to decide whether this project is actually something that is even
> vaguely of interest to you :-)
>
> I will be happy to answer any questions you may have. Please let me know
> your thoughts.
>
> Edward K. Ream
> An old man, crazy about computer programming
>
> = End Post
>
> Edward
>
> --
> You received this message because you are subscribed to the Google Groups
> "leo-editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leo-editor+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/leo-editor/c70a9009-31d5-4cc4-b8d4-6e3af34d6d9f%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAFPg46RmMgwVzuvRdU6G%2BgWR346rkNy61scpSPc0F4BUNqDzuw%40mail.gmail.com.