[issue26993] Copy idlelib *.py files with new names

[Terry J. Reedy]

Terry J. Reedy added the comment:

Files were renamed later in May.  Some things were backported to 3.5 and even 
2.7 for 3.5.3 and 2.7.13.  Anything involving ttk, which will soon be nearly 
all tkinter code, could not and cannot be backported.

--
resolution: postponed -> rejected
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Terry J. Reedy]

Terry J. Reedy added the comment:

I finished reopening #24225 (renaming), for 3.6 only.  I will open a new issue 
with Larry Hastings nosy, should I proposed (next fall) to backport the 
revamped idlelib to 3.5.

--
resolution:  -> postponed
superseder:  -> Idlelib: changing file names

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Terry J. Reedy]

Terry J. Reedy added the comment:

Both Serhiy and Ned have expressed concern about the difficulty of merging 
patches from 3.5 to 3.6 once the 3.6 files are renamed.  To me, this is a 
non-issue since I don't expect to do that more than a few times, and some may 
be null merges of backports of fixes originally developed for and applied to 
3.6.  And I don't expect another core developer to start applying patches to 
3.5.

Hence my opinion that I should use hg rename, to keep continuity of revision 
history as much as possible, and that I should push a complete rename patch as 
soom as possible.

Two things that I *will* apply to 3.5, if and when possible, are fixing the 
third regression (Idle preferences not opening on Linux with tk 8.6, #26673) 
and fixing freezing of Macs when clicking on a completion window (#15786).

To me, this issue is a start on paying down IDLE's technical debt.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Terry J. Reedy]

Terry J. Reedy added the comment:

In response to Nick's point 4: To me, these are important aspects of 'in the 
stdlib'.

First is having issues on this tracker.  As Guido said in his King Day speech, 
non-trivial software is a collective project.  IDLE needs continued access to 
dependency experts, such as Serhiy for tkinter, and platform experts, such as 
Ned for Mac.  In April, I nosyed Kristján Valur Jónsson to ask an IDLE question 
that seemed related to a socket patch he pushed, and he helpfully answered.  On 
the other side, users are much more likely to participate here than on a 
separate tracker.

Second is being easily installable with the rest of Python.  I think some 
out-of-the-box augment to the console is crucial on Windows.

I am aware that Linux distributions have a separate tkinter-gui install instead 
of just a checkbox option.  I think it should consistently, on all plateforms, 
comprise _tkinter, tkinter, idlelib, turtle, and turtledemo.

All of those except idlelib are documented even though optional.  So I think 
idlelib could be also.  But documenting direct usage, which Nick recommended in 
#24225, could be relegated to the modules themselves and the idlelib README or 
a new text file.  Ditto for indicating replacements for thing known or strongly 
 suspected of external use.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Terry J. Reedy]

Terry J. Reedy added the comment:

I should have explained my usage of 'idle2' and 'idle3'.  'Idle1' was the 
original IDLE that executed user code in the idle process.  'Idle2' was the 
re-write, still in use, that executes user code in a separate process.  'Idle3' 
is intended to be a re-write of similar scope that uses ttk, but also refactors 
to make automated tests and other changes easier.  (Adding multi-pane windows 
might be called 'idle4'.)  It is the tests and other changes that require 
refactoring.  Hence 'ttk-idle' partly misses the point unless understood as 
including name changes and refactoring.

For the idle2 re-writes, all the new code needed to execute remotely, by 
default, while keeping local execution as an option, had to be in place for any 
of it to be useful. The idle3 changes can be broken up into multiple usable and 
testable pieces.  I wish the renames could be in alpha1 and definitely want 
them in alpha2, along with a notice in What's New.  Anyone who has code that 
will break because of the changes should have opportunity to find out as soon 
as possible.  I would like to do simple ttk replacements by alpha 3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Nick Coghlan]

Nick Coghlan added the comment:

+1 for Ned's point that PEP 434 grants the IDLE developers *permission* to 
backport IDLE enhancements to maintenance releases, but doesn't *oblige* them 
to provide such backports.

For Python 3.6, another point to keep in mind is that if anyone particularly 
wants to keep the old idlelib API alive, they can either fork the existing code 
as a PyPI project, or create a compatibility shim that runs atop the new 
ttk-based API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Ned Deily]

Ned Deily added the comment:

Glad to be of help with regards to 8.4 :=)  With my 3.6 release manager hat on, 
I think the best way to think of this is as a new feature: call it "Ttk IDLE" 
or some such (and not idle2 vs idle3 which I think is confusing) and, rather 
than inventing something new, I suggest you should just follow the standard 
procedure for new feature development as documented in the Developers Guide.  
That is, create a temporary repository on hg.python.org and make all the 
changes in a new branch in that repo.  Or, if you'd rather work in private, 
just create your own local repo clone and branch within it.  When you are 
satisfied that it is ready to be integrated into the release, we can either 
merge the branch in or extract the changes as one big patch to be applied to 
the main cpython default branch.  And I'll be happy to help with making that 
happen when the time comes.  That way you and any one else wanting to work on 
IDLE can work unimpeded, you don't have to worry about temporary changes and h
 acks to accommodate two versions of IDLE in the same repo, the old version of 
IDLE continues to work in the main cpython default branch until TtkIDLE is 
ready, and the cpython repo is not cluttered with all the intermediate commits 
from the feature dev cycle (although they could be preserved during 
integration, if desired).  Plus, during feature development you should be able 
to use TtkIDLE from the feature repo with a standard 3.6 (or 3.5) binary 
install of Python.

As for timing, I would prefer to get such a major feature in by feature code 
cutoff but, as you say, IDLE is pretty self-contained so we can be flexible.  A 
similar argument would apply to 3.5.  But I think it makes sense to focus on 
3.6 first and then later separately consider dropping it all into 3.5.x, a 
decision that would be up to Larry as the 3.5 release manager.  For 3.6, I 
don't think there's a need for further discussion or to wait to begin, as long 
as the development work is done in a feature repo.  Have at it!

https://docs.python.org/devguide/committing.html#long-term-development-of-features

By the way, if you are essentially replacing the IDLE files, then (contrary to 
what I suggested earlier) you probably don't want to use hg mv to rename files. 
 Otherwise you can end up with needless merge conflicts between 3.x branches.  
In the feature branch, you can just do the equivalent of: hg rm Lib/idlelib and 
hg commit then copy any old idlelib files you want to retain back into 
Lib/idlelib, and do an hg add and hg commit.  Then the feature branch files are 
no longer "connected" to the old IDLE files from hg's perspective.  On the 
other hand, if you go the feature repo route, you won't have two copies of IDLE 
in the same branch at a time, so you don't *have* to rename files if you don't 
otherwise want to.  In any case, if the feature is integrated as one big 
commit, all of this can be handled at integration time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Terry J. Reedy]

Terry J. Reedy added the comment:

Ned, thank you for the helpful response.  Yes, I *was* proposing two versions 
in 3.5 and subsequent releases until the current version could be deleted.  Its 
an ugly, least bad alternative that only made sense under the presumption that 
'idle2' had to remain in 3.6.   With that assumption dropped ...

I would be delighted to start renaming (and refactoring) existing files in 
default *now*.  Whether to renames all at once (the current #24225 proposal) or 
in batches would be a sub-issue of #24225.

I believe that the PEP 434 exemption from separating 'behavior' from 
'enhancement' issue applies during the beta period as well as after.  For IDLE, 
danger of accidental regression and thoroughness of testing are more relevant 
for deciding timing of commits.

As for 3.5, if we assume that substantial numbers of people, including 
instructors, will use the final 3.5 (likely 3.5.3) instead of 3.6.0, then it 
might be better for users to include 'idle3' in 3.5.3 as an option.  This could 
be done a couple of weeks before the release by copying 3.6 idlelib into, for 
instance, 3.5 idlelib._, editing imports, and adding a switch to the startup 
code.  If Ned's alternative is accepted, the backport decision should be 
deferred until there is something worth backporting.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Ned Deily]

Ned Deily added the comment:

Terry, I'm all for having IDLE using the newer Ttk widgets and for refactoring 
to make it easier to enhance and maintain IDLE.  I'm not sure I understand the 
proposal to make two sets of IDLE files: does this mean there would be two 
versions of IDLE in one branch, e.g. Python 3.6?  That doesn't make sense to 
me.  One issue you mention is continuing to support Tk 8.4 (which lacks Ttk) on 
legacy systems.  Tk 8.4 support ended some years ago by the Tcl Core team. I 
think it is time for us to draw a line in the sand and say we no longer need to 
continue to support it for IDLE.  (It would be nice to not intentionally break 
basic non-IDLE Tkinter support for 8.4.)  I haven't yet announced any proposals 
for Mac installer support for 3.6 but I'm willing to commit to not requiring 
any Tcl/Tk 8.4 dependencies.   So, if we assume 8.4 is no longer an issue, what 
other issues remain?  I believe you've already stated that you will no longer 
attempt to keep the 2.7 version of IDLE in sync with the 
 latest IDLE 3; I thinks that's the right approach at this stage.  That means 
there will be only some backporting of selected IDLE bugfixes to 2.7 and we 
don't do any direct VCS merging between 2.7 and default anyway.  So it seems to 
me there is no reason why the default branch (e.g. what is going into 3.6) 
needs to retain any copies of IDLE files from the 2.7 branch.  In other words, 
all the file renamings (via hg mv or otherwise) and deletions just take place 
in default without disturbing 2.7.  If there are any external 
incompatibilities, it's fine to announce them as part of the 3.6 feature 
release.  This is assuming that the externally incompatible work is in place in 
time for the 3.6 feature code cutoff (3.6.0 beta 1, currently scheduled for 
2016-09-08).  As far as 3.5.n is concerned, the last bugfix release for 3.5 is 
likely to be less than a year away (assuming 3.6.0 releases by the end of 2016 
as planned).  So I would also recommend not making any refactoring changes for 
3.5,
  just important bugfixes similar to 2.7.  We should try to make it easier for 
you and the others working on IDLE.  And try to minimize the risks of 
inadvertent breakages going into maintenance releases.  Does that make sense?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Terry J. Reedy]

Terry J. Reedy added the comment:

In response to 1.  I considered copying idlelib as a whole as a serious 
possibility, meant to include it as an alternative, and am willing to 
reconsider my choice.  I specifically thought of something like idlelib3 or 
idlelib/idle3, but not _idlelib,  A complete copy in a new directory would be, 
as you say, a cleaner separation but it seemed like it would make import 
changes and startup more complicated.  It is also not obviously covered by PEP 
434 -- unless you say so.

For users, making idle2 the copy would immediately break current external 
imports.  Making idle3 the copy would break new external imports sometime in 
the future.  But using _idlelib would warn of that possibility.

For importat, IDLE itself uses uses all of 'from idlelib import module', 'from 
idlelib.module import callable', 'from idlelib.idle_test.htest import run', and 
unittest.main('idlelib.idle_test.test_module' (but no relative imports).  A new 
directory would greatly increase the number of changes and mean changing 
'idlelib' to 'something' now and 'something' back to 'idlelib' in the future.

I thought of using relative imports, but I have not used them and have read 
that they are limited, tricky, and discouraged.  I just did some experiments 
and it seems that they do not work when a file is run as __main__ instead of 
imported.  (Whatever the limitation, it is not obvious to me from the doc.)

I agree with 2. and 3. and will response to 4. in another message.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Nick Coghlan]

Nick Coghlan added the comment:

Terry, have you considered also doing a top level idlelib -> _idlelib rename in 
addition to the file level name changes?

My rationale for suggesting that:

1. The idle2 vs idle3 distinction will be clear at a glance (as they'll be in 
different directories)
2. The package level rename handles bifurcation of files that aren't themselves 
being renamed
3. You can subsequently iterate on the internal APIs without worrying about 
third party breakage
4. You set the stage for eventually moving IDLE to being an independently 
versioned implicitly pip-installed component (perhaps with an 
"ensure_default_ide" module akin to "ensurepip")

In relation to that last point, while I think it's important for CPython to 
come with a default IDE, I *don't* think it's important for the Python standard 
library to provide a default IDE-building toolkit (and, indeed, Linux distros 
tend to separate out not only IDLE, but also Tcl/Tk support in general, from 
the default Python installation).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26993] Copy idlelib *.py files with new names

[Terry J. Reedy]

New submission from Terry J. Reedy:

Proposal: duplicate nearly all of the existing idlelib *.py files, with new 
names.  This would implement the intention of PEP 434 with respect to freely 
changing idlelib APIs.  I view it as a prerequisite for most anticipated 
patches.  A longer than usual explanation follows.

Last August/September, I realized that the way to modernize and improve IDLE, 
while maintaining backward compatibility, would be to add copies of existing 
.py files.  The copies with new names (the idle3 set) could be freely edited  
while existing files (the idle2 set) could be pretty much left alone until 
deleted.  IDLE would temporarily, for a time yet to be decided, have two 
operating modes.  The idle2 mode would only use exiting files and tk widgets.  
The idle3 mode would use new files and ttk widgets.

Renaming and refactoring files, including PEP 8 updates, is needed for bug 
fixing, testing, and feature upgrades, including addition of a single window 
multiple pane design.  However, doing so with existing files would break direct 
imports and subsequent calls (see #24225, for some discussion).  Ttk widgets 
cannot be used on systems with tcl/tk 8.4. (This and related issues was 
discussed with Nick, Ned, and release managers, as well as in #24759, 
especially msg247993 & msg248031, and in #24750).

I originally planned to add new files one or a few at a time.  I did not start 
doing do immediately, before the November-December releases, because I wanted 
to fix some bugs in the existing files before they were bifurcated. I did not 
start doing so after the releases because I could not figure out how to safely 
manage an 'idle3/ttk' mode that mixed a shrinking subset of idle2 files and the 
growing set of new idle3 files.  It seemed like a recipe for introducing bugs.  
I decided instead that the best solution would be to create a complete, tested 
idle3 set all at once, using the following steps.  (Idlelib/idle_test/*.py 
files will be handled similarly, but with differences discussed later.)

0. Decide on the new names.
1. Copy idlelib/*.py files and hg add them.
2. Revise imports in new idle3 files.
3. Test to make sure that the idle3 set works right. 
4. Upload patch for review, and commit.

Possible variation: push after step 1, so that the revisions in step 2 are a 
separate patch instead of being folded into a combined patch.  I do not propose 
that because the new files would be useless until revised.


DETAILS OF CURRENT PLAN:

0. @newnames.txt, https://bugs.python.org/file42678/%40newnames.txt is attached 
to issue #24225.  It lists old name, Sweigart's new name, and my current new 
name.  Bikeshedding names can be done on that issue.

Leaving aside the unused and deprecated 'idlever' and the startup files, all of 
which are omitted from the list, there are currently three short lowercase 
names: 'help', 'rpc', and 'run'.  To reuse these names for the idle3 set, these 
could be renamed, in a preliminary patch, to have a captital letter.  I think 
this should be okay because  'help' specifies that is it subject to change, and 
the other two are extremely unlikely to be used outside of IDLE.  If this were 
done, implementation files could be identified as 'idle2' versus 'idle3' by the 
presence of an uppercase letter in the name.

Serhiy requested on #24225 that renames of existing 3.5 files be backported to 
2.7.  Since I do not anticipate many patches to idle2 files, let alone 
backports to 2.7, I am not sure that this is needed.

Alternatively, I could name the copies help3, rpc3, and run3.

1. Use the name list from step 0 to create copyall.bat, to be run in the 3.5 
repository idlelib directory.  It will have 'copy src dst' for each file and 
end with 'hg add ../idlelib' (or 'cd ..; hg add idlelib') to 'add' them all.  I 
am aware of 'hg copy', but the docs are not clear on whether there is any 
lingering effect after pushing the adds.  The pre-commit merge effect seems 
unneeded as no one else is pushing idlelib commits (except Serhiy ocassionally, 
as part of multifile patches, and he should be reading this ;-).

2. Review Al Sweigart's 'idle_updating_imports.patch' from #24225.  Al ran 
revised unittests, but the coverage is too spotty to adequately test.  So 
review is needed.

The tricky part is that many modules and classes have the same name, such as 
AutoComplete the module and AutoComplete the class.  Module references must be 
changed to the new name; class references must not be touched.  I would use 
IDLE's File in Files (grep) with each module name to find references to check.  
The import in each client module ('import X' versus 'from X import X' 
determines how bare 'X' is used in rest of the module.

If the patch seems correct, or mostly so, perhaps after a few corrections,  
write a script, using the step 0 name list, to replace Al's new names with my 
new names, where they are different.

Apply the patch.  Since Al's patch is a year old, some chunks w