Re: Proposed change to setup.py

[Thomas Passin]

On Sunday, March 29, 2020 at 10:29:36 PM UTC-4, Matt Wilkie wrote:
>
>
> b) merge early, merge often: as in daily. This way any given upstream 
> change can (usually) be inspected and accepted/changed in small easy to 
> understand pieces. Use an interactice tool like WinMerge or Kmeld to do the 
> merging. (WinMerge has been easiest for me to understand, the keyboard 
> shortcuts make sense to me.)
>

Thanks for your advice, Matt.  I really don't want to spend time doing 
daily merges if I can avoid it.  And I'll look at Winmerge.  I think my 
ideal would be to create a branch in my fork that would only contain the 
files I'm working on.  But Git (or at least Github desktop) doesn't seem to 
work that way.  Otherwise I suppose that frequent updates from upstream 
would be the best way to go.
 

> Really though if what you're doing now is successful keep doing it! VR3 is 
> awesome. Better you spend your precious attention on building cool things 
> like that than figuring out git arcana. ;-)
>

Well, thanks!  Though I feel that I'm just resurrecting Peter Mills's work 
and giving it a few tweaks.
 

>
> -matt
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/5fad1849-6dc6-4a3c-a467-35bb2ec38b0b%40googlegroups.com.

Re: Proposed change to setup.py

[Matt Wilkie]

>
> With the confusions I've had using github forks, I think I'd like to work 
> in my own Mercurial repo, where I would only need my own code and not the 
> entire Leo distro, and copy the results to my Github fork of Leo.  For that 
> to work, the most convenient way would be for Leo to load plugins from the 
> PYTHONPATH instead of its own location.
>
> One of the github things that I haven't mastered is when I start a branch 
> for a project - say based off devel - and then that upstream branch changes 
> before I'm ready to merge or do a pull request.  Of course, this happens 
> all the time.  If I don't update my fork to upstream, then the merge or 
> pull request could involve all those other changed files that I wasn't 
> working on.  If I do update to upstream, then the update would step all 
> over my work, and I would have to copy my work back into the fork.
>
> Do you know how to make this all work?
>

One option is to enable the hg-git plugin, so you can continue to use 
Mercurial and push/pull from a git repo. I used this happily for a couple 
of years until a change in network policy at work blocked me from being to 
push outside the network.

As for not being clobbered by upstream changes or getting too far behind, I 
use 2 main strategies: 

a) keep your work in it's own folder within the repo, a place no one else 
is likely to touch. This is not possible if you need to touch core though. 
However this problem isn't solved by the off-campus route you're using now 
either.

b) merge early, merge often: as in daily. This way any given upstream 
change can (usually) be inspected and accepted/changed in small easy to 
understand pieces. Use an interactice tool like WinMerge or Kmeld to do the 
merging. (WinMerge has been easiest for me to understand, the keyboard 
shortcuts make sense to me.)

Really though if what you're doing now is successful keep doing it! VR3 is 
awesome. Better you spend your precious attention on building cool things 
like that than figuring out git arcana. ;-)

-matt

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/c4b7be62-c11e-452e-994f-1d1fdb2b0f90%40googlegroups.com.

Re: Proposed change to setup.py

[Thomas Passin]

On Sunday, March 29, 2020 at 2:01:53 PM UTC-4, Matt Wilkie wrote:
>
>  
>
>> Then you could keep your development code entirely out of the repo until 
>> you got it to the point it was worth adding it in.  I'd like that.
>>
>
> A fork would accomplish this and then be easier to add to the main repo 
> later. I think so anyway, but there  might be things I'm not considering 
> that could make it more work than is desireable.
>

With the confusions I've had using github forks, I think I'd like to work 
in my own Mercurial repo, where I would only need my own code and not the 
entire Leo distro, and copy the results to my Github fork of Leo.  For that 
to work, the most convenient way would be for Leo to load plugins from the 
PYTHONPATH instead of its own location.

One of the github things that I haven't mastered is when I start a branch 
for a project - say based off devel - and then that upstream branch changes 
before I'm ready to merge or do a pull request.  Of course, this happens 
all the time.  If I don't update my fork to upstream, then the merge or 
pull request could involve all those other changed files that I wasn't 
working on.  If I do update to upstream, then the update would step all 
over my work, and I would have to copy my work back into the fork.

Do you know how to make this all work?

So until I learn how to avoid this problem, I need a place to develop my 
stuff that's outside of the fork.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/37f95744-6f13-4882-9da0-3fc459b567eb%40googlegroups.com.

Re: Proposed change to setup.py

[Matt Wilkie]

 

> Then you could keep your development code entirely out of the repo until 
> you got it to the point it was worth adding it in.  I'd like that.
>

A fork would accomplish this and then be easier to add to the main repo 
later. I think so anyway, but there  might be things I'm not considering 
that could make it more work than is desireable.

 

>  
>
>> Also for sde-byside it's best not to use the same HOME, as files in the 
>> .leo folder might get messed up. 
>>
>
> That's a problem that I sometimes have, not only with side-by-side.  I 
> don't like to use side-by-side anyway except for, let's say, checking that 
> something still works in a different version of Leo.  Even virtual 
> environments don't solve this, do they? 
>

Correct, virtuals don't help with this unless you also virtualize HOME 
(which is certainly possible by the way. There's a mechanism in conda and 
probably others to run scripts automatically when the env is activated.)

-matt

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/e09af8d4-3d4a-4f84-816f-10e1b30bfcf9%40googlegroups.com.

Re: Proposed change to setup.py

[Thomas Passin]

On Sunday, March 29, 2020 at 12:34:56 AM UTC-4, Matt Wilkie wrote:
>
> So if I had a clone at d:\leo-clone, I'd use the following?
>>
>> python -m pip install --editable d:\leo-clone
>>
>  
> Yes.
>
> I  have been setting PYTHONPATH to d:\leo-clone.  This has always seemed 
>> to work well, as long as the dependencies have been installed in my main 
>> Python install (because of a previous Leo install, for example).  This way, 
>> I can always switch back to using the previous install just by launching 
>> from another terminal without setting PYTHONPATH.
>>
>
> If your diferent versions are in the same SCM repo (eg. hg, git) then 
> resettng to or checking out your desired version can be done without 
> messing with PYTHONPATH. This is the primary benefit of editable install, 
> it always uses what's in that directory tree *right now, *allowing you to 
> test code changes immediately.
>
> pip install -e d:\leo-clone
> pushd d:\leo-clone
> git checkout v6.1
> python -m leo.core.runLeo
> # hack, hack. do stuff in Leo 6.1. exit.
> git checkout v6.2
> python -m leo.core.runLeo
> # hack, hack. do stuff in Leo 6.2. exit.
> ...etc
>

You get the same result if you point PYTHONPATH to the clone location.  
 

> If you want to run different versions of Leo side by side then *don't *pip 
> install Leo at all, editable or not, and set PYTHONPATH the way you have 
> been. I think it's cleaner to not to change PYTHONPATH though and to use 
> `python d:\leo-clone-2\launchLeo.py` for starting. (It might mess with some 
> imports or make them unpredictable as they search in Leo's tree instead of 
> the real python tree.)
>

Yes, that's a problem because Leo plugins aren't found the same way as 
ordinary imports.  I've been thinking of asking for a change so that the 
plugins are found using the PYTHONPATH instead of the current method.  Then 
you could keep your development code entirely out of the repo until you got 
it to the point it was worth adding it in.  I'd like that.
 

> Also for sde-byside it's best not to use the same HOME, as files in the 
> .leo folder might get messed up. 
>

That's a problem that I sometimes have, not only with side-by-side.  I 
don't like to use side-by-side anyway except for, let's say, checking that 
something still works in a different version of Leo.  Even virtual 
environments don't solve this, do they? 

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/9c5422b1-adef-4bc7-8ff3-54f01e49ddd7%40googlegroups.com.

Re: Proposed change to setup.py

[Matt Wilkie]

>
> So if I had a clone at d:\leo-clone, I'd use the following?
>
> python -m pip install --editable d:\leo-clone
>
 
Yes.

I  have been setting PYTHONPATH to d:\leo-clone.  This has always seemed to 
> work well, as long as the dependencies have been installed in my main 
> Python install (because of a previous Leo install, for example).  This way, 
> I can always switch back to using the previous install just by launching 
> from another terminal without setting PYTHONPATH.
>

If your diferent versions are in the same SCM repo (eg. hg, git) then 
resettng to or checking out your desired version can be done without 
messing with PYTHONPATH. This is the primary benefit of editable install, 
it always uses what's in that directory tree *right now, *allowing you to 
test code changes immediately.

pip install -e d:\leo-clone
pushd d:\leo-clone
git checkout v6.1
python -m leo.core.runLeo
# hack, hack. do stuff in Leo 6.1. exit.
git checkout v6.2
python -m leo.core.runLeo
# hack, hack. do stuff in Leo 6.2. exit.
...etc


If you want to run different versions of Leo side by side then *don't *pip 
install Leo at all, editable or not, and set PYTHONPATH the way you have 
been. I think it's cleaner to not to change PYTHONPATH though and to use 
`python d:\leo-clone-2\launchLeo.py` for starting. (It might mess with some 
imports or make them unpredictable as they search in Leo's tree instead of 
the real python tree.)

Also for sde-byside it's best not to use the same HOME, as files in the 
.leo folder might get messed up. 

-matt

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/b382c3f8-7554-4e21-be6e-106bb165d1f5%40googlegroups.com.

Re: Proposed change to setup.py

[Thomas Passin]

On Saturday, March 28, 2020 at 4:45:01 PM UTC-4, Matt Wilkie wrote:
>
>
> I've always installed Leo using pip.  I still don't understand what 
>> "--editable" accomplishes, so I've never used it.
>>
>
> *Pip install* puts the package and all the files it says it needs under 
> PYTHONHOME/Lib/site-packages and creates a launch wrapper in 
> PYTHONHOME/Scripts.
>
> Pip install *--editable* doesn't copy anything, puts link files in Lib 
> pointing to the code location, and creates the launch wrapper to code 
> location. 
>

So if I had a clone at d:\leo-clone, I'd use the following?

python -m pip install --editable d:\leo-clone

I  have been setting PYTHONPATH to d:\leo-clone.  This has always seemed to 
work well, as long as the dependencies have been installed in my main 
Python install (because of a previous Leo install, for example).  This way, 
I can always switch back to using the previous install just by launching 
from another terminal without setting PYTHONPATH.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/a3d7c458-edc5-4d04-968e-c5d0e131ccf5%40googlegroups.com.

Re: Proposed change to setup.py

[Matt Wilkie]

> I've always installed Leo using pip.  I still don't understand what 
> "--editable" accomplishes, so I've never used it.
>

*Pip install* puts the package and all the files it says it needs under 
PYTHONHOME/Lib/site-packages and creates a launch wrapper in 
PYTHONHOME/Scripts.

Pip install *--editable* doesn't copy anything, puts link files in Lib 
pointing to the code location, and creates the launch wrapper to code 
location.  

At present editable is the most reliable because it's guaranteed all files 
are present. The first way leaves some files out. Some are unavoidable 
without deep restructuring of Leo's code repo [#603 
, #573 
] 
and others I'm slowly tracking down and fixing as the bug reports come in 
(and I figure out how to remedy). [#1512 
 and probably #1547 
] 


I recall reading that launchLeo acts different somehow than leo.core.runLeo, 
> but I don't remember the differences. For what it's worth, I normally start 
> Leo with python -m leo.core.runLeo
>

These two methods should be identical. I'm interested in more info if this 
isn't the case.

 -matt

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/dd289e1c-c036-4ba4-a78d-033cfe611f93%40googlegroups.com.

Re: Proposed change to setup.py

[Matt Wilkie]

> Anyway, I expanded the docstring of setup.py and put in a guard to prevent 
>> anyone from using it naively like I did.   I've attached below.
>>
>
> Looks good to me. Matt, any comments?
>

Thank you tfer. In all this time it never occured to me to add a message to 
guide people to usng pip instead! I'll use your code with some modification 
to do that. (We have at least one project downstream that uses Leo's 
setup.py directly so we should be mindful not to block that.)

I created #1547  for 
this.

-matt


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/def3279e-8b88-4104-8a6c-7c0c0bf6c681%40googlegroups.com.

Re: Proposed change to setup.py

[Thomas Passin]

I still don't understand what "--editable" accomplishes, so I've never used 
it.

What I do, when I want to run from a git clone, is to set PYTHONPATH to the 
top directory of the clone.  Then Python will use, say, the Leo clone but 
it find all its other packages from the regular installed site-packages 
directory.  If I'm working on code for Leo, I keep the working code in a 
separate development directory, and bind a hot key to a little script that 
copies that file or files to the git clone.

This way, if the clone gets trashed somehow, I haven't lost any work.  I 
can just copy it to a new clone.

This approach is also compatible with using a virtual environment.

On Thursday, March 26, 2020 at 2:00:41 AM UTC-4, tfer wrote:
>
> I've been making a mistake in setting up Leo and using it for years!
>
> I've always run Leo from my local git clone of it, (usually the develop 
> branch), never realizing I should have been using setup.py to make sure the 
> python it is running under is fully ready to support it.  I just used 
> launchleo.py to start it, adding any packages that it complained about as 
> "missing" and carrying on.
>
> I was using the very "fat" anaconda as my system python and not using any 
> virtual environments.  Now that their conda utility program has been made 
> to work under powershell, (it only worked "cmd", before), --note use 
> "/conda init" to integrate it into powershell, I realiize 
> this wasn't the way to go.
>
> I got rid of Anaconda, (not easy, uninstall and getting rid of 
> directories...), then just went with Miniconda.  I now can create conda 
> env(s), leobase, leobase1, leobase2, and so on.  This is were I finally 
> tumbled on what setup.py was for!  
>
> My first naive approach was to run the script directly with the leobase 
> enviroment active, but it does not run completely this way, (it does a 
> bunch of stuff, then exits with a failed import).  I found (and Matt 
> pointed out to me), that I should be using pip, specifically: "pip install 
> --editable ." --while in the git directory of leo.
>
> Anyway, I expanded the docstring of setup.py and put in a guard to prevent 
> anyone from using it naively like I did.   I've attached below.
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/0116e10a-0448-442e-a009-73ca5874eb1b%40googlegroups.com.

Re: Proposed change to setup.py

[Edward K. Ream]

On Thu, Mar 26, 2020 at 1:00 AM 'tfer' via leo-editor <
leo-editor@googlegroups.com> wrote:

> I've been making a mistake in setting up Leo and using it for years!
>

Yeah, setup.py has confused a lot of people.

Anyway, I expanded the docstring of setup.py and put in a guard to prevent
> anyone from using it naively like I did.   I've attached below.
>

Looks good to me. Matt, any comments?

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS1JZzbFCG4C084146H%3DqEFju3CBN0pzUHhdyJse6_Vq0w%40mail.gmail.com.

Proposed change to setup.py

['tfer' via leo-editor]

I've been making a mistake in setting up Leo and using it for years!

I've always run Leo from my local git clone of it, (usually the develop 
branch), never realizing I should have been using setup.py to make sure the 
python it is running under is fully ready to support it.  I just used 
launchleo.py to start it, adding any packages that it complained about as 
"missing" and carrying on.

I was using the very "fat" anaconda as my system python and not using any 
virtual environments.  Now that their conda utility program has been made 
to work under powershell, (it only worked "cmd", before), --note use 
"/conda init" to integrate it into powershell, I realiize 
this wasn't the way to go.

I got rid of Anaconda, (not easy, uninstall and getting rid of 
directories...), then just went with Miniconda.  I now can create conda 
env(s), leobase, leobase1, leobase2, and so on.  This is were I finally 
tumbled on what setup.py was for!  

My first naive approach was to run the script directly with the leobase 
enviroment active, but it does not run completely this way, (it does a 
bunch of stuff, then exits with a failed import).  I found (and Matt 
pointed out to me), that I should be using pip, specifically: "pip install 
--editable ." --while in the git directory of leo.

Anyway, I expanded the docstring of setup.py and put in a guard to prevent 
anyone from using it naively like I did.   I've attached below.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/d3d56934-0699-4f7a-abfa-d2c1a35b1bc4%40googlegroups.com.
# -*- coding: utf-8 -*-
#@+leo-ver=5-thin
#@+node:maphew.20180224170853.1: * @file C:/VC/git/leo-editr/setup.py
#@@first
"""setup.py for leo

As the top level setup script in a package, library, or application -- as here, for the leo-editor Application, it is meant to make the package installable in a 'ready to use' state.  If you 'pip install leo-editor' this script will be run.

Note: if you are running Leo out of you own local git copy of leo-editor, you'll need to run the command:
pip install --editable  
   or
pip install --editable .
   if you are in the repository.
"""
#@+others
#@+node:trf.20200325085135.1: ** guard to prevent running at top-level
# This file is only meant to be run by pip, not by itself.
# Below is a guard to prevent that and offer advice.

if __name__ == "__main__" :
print('"setup.py" is only meant to be run by pip.  ')
print('From inside the Leo package directory, please run: ')
print('   pip install --editable .   ')
print('To make Leo fully ready to run')
quit()
#@+node:maphew.20180305124637.1: ** imports
from codecs import open  # To use a consistent encoding
import os
import platform
# from shutil import rmtree
from setuptools import setup, find_packages  # Always prefer setuptools over distutils
import sys


# Ensure setup.py's folder is in module search path else import leo fails
# required for pip >v10 and pyproject.toml
sys.path.append(os.path.dirname(os.path.realpath(__file__)))
import leo.core.leoGlobals as g
import leo.core.leoVersion as leoVersion
#@+node:mhw-nc.20190126224021.1: ** setup janitor
try:
   from setupext_janitor import janitor
   CleanCommand = janitor.CleanCommand
except ImportError:
   CleanCommand = None

cmd_classes = {}
if CleanCommand is not None:
   cmd_classes['clean'] = CleanCommand
#@+node:maphew.20181010203342.385: ** get_version & helper (setup.py)
def get_version(file, version=None):
"""Determine current Leo version. Use git if in checkout, else internal Leo"""
root = os.path.dirname(os.path.realpath(file))
if os.path.exists(os.path.join(root, '.git')):
version = git_version(file)
if not version:
version = get_semver(leoVersion.version)
return version
#@+node:maphew.20181010203342.386: *3* git_version
def git_version(file, version=None):
"""
Fetch from Git: {tag} {distance-from-tag} {current commit hash}
Return as semantic version string compliant with PEP440
"""
root = os.path.dirname(os.path.realpath(file))
try:
tag, distance, commit = g.gitDescribe(root)
# 5.6b2, 55, e1129da
ctag = clean_git_tag(tag)
#version = get_semver(ctag)
version = ctag
if int(distance) > 0:
version = '{}-dev{}'.format(version, distance)
except IndexError:
print('Attempt to `git describe` failed with IndexError')
except FileNotFoundError:
print('Attempt to `git describe` failed with FileNotFoundError')
return version
#@+node:maphew.20180224170257.1: *4* clean_git_tag
def clean_git_tag(tag):
"""Return only version number from tag name. Ignore unknown formats.
   Is specific to tags in Leo's repository.
5.7b1