[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Terry J. Reedy]

Change by Terry J. Reedy :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[STINNER Victor]

STINNER Victor  added the comment:

The 12 latest build on https://buildbot.python.org/all/#/builders/464 are green 
(success). Can this issue be closed now? We can reopen it later if the issue 
strikes back.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9c89d62073fa0bcfe68e59add5b55fbcbf7672ab by Miss Islington (bot) 
in branch '3.10':
bpo-44282: Fix occasional test_incremental_editing failures on buildbots 
(GH-26491) (GH-26499)
https://github.com/python/cpython/commit/9c89d62073fa0bcfe68e59add5b55fbcbf7672ab


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +25095
pull_request: https://github.com/python/cpython/pull/26499

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[STINNER Victor]

STINNER Victor  added the comment:


New changeset adef445dc34685648bd0ea1c125df2ef143912ed by Tal Einat in branch 
'main':
bpo-44282: Fix occasional test_incremental_editing failures on buildbots 
(GH-26491)
https://github.com/python/cpython/commit/adef445dc34685648bd0ea1c125df2ef143912ed


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Tal Einat  added the comment:

See PR GH-26491.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Change by Tal Einat :


--
pull_requests: +25087
pull_request: https://github.com/python/cpython/pull/26491

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Tal Einat  added the comment:

On my machine it running the test many times with parallelization no longer 
causes any failures with this change.  Also, running other tests which us the 
@run_in_tk_mainloop decorator, with the same 1ms delay as before, doesn't cause 
any such failures.  PR forthcoming.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Tal Einat  added the comment:

> Running 12 duplicate tests in parallel on Windows on a 6 core (12 CPU) 
> machine with [...] resulted in nearly all failing.

Yes, running test_idle in parallel with itself usually causes some failures due 
to UI focus being stolen from one instance by another.  That's why I just ran 
this test rather than the whole test suite.

> I am reluctant to add a delay that is mostly not needed, so i tried the 
> uploaded 'adaptive' code to only add a long delay if needed.  Uploaded.  But 
> when a tested failed once, it failed again even with the delay, and contrary 
> to the results above.

The delay needs to be placed carefully.  Changing root.after(1, ...) to use 50 
rather than 1 (these are milliseconds) seems to resolve the issue entirely on 
my machine, at the cost of this single test taking 0.4 seconds to run rather 
than about 0.045 seconds.

> The other thing I thought is parameterizing the decorator to pass in a delay 
> only for this test, but that require another layer of wrapping, which I 
> cannot do right now.

I can do that.  Though I fear this may happen for other such tests as well...  
Worth a shot I guess.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The other thing I thought is parameterizing the decorator to pass in a delay 
only for this test, but that require another layer of wrapping, which I cannot 
do right now.

So I think we should just add skiptest for now.  In fact, we only need to run 
it when editing colordelagator.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The confusion is my fault: PR-26404 is where I first saw this failure, for the 
previously added tests, not where the test was added.  I unlinked the erroneous 
message.

Running 12 duplicate tests in parallel on Windows on a 6 core (12 CPU) machine 
with

f:\dev\3x>python -m test -j12  -ugui -m test_incremental_editing test_idle 
test_idle test_idle test_idle test_idle test_idle test_idle test_idle test_idle 
test_idle test_idle test_idle

resulted in nearly all failing.  Half or less failed with -j6, usually one with 
-j3, none without -j.  Are the processess failing because of interaction with 
each other?  Or from be switched with other processes?

Back with -j12: increasing the after delay in run_in_tk_mainloop from 1 to 5 
cut the failures down to 0-4 in 12.  Increasing further to 8 resulting in no 
failures.  Note that human keystrokes are at least, say, 50 milliseconds (1/20 
second) apart.

I am reluctant to add a delay that is mostly not needed, so i tried the 
uploaded 'adaptive' code to only add a long delay if needed.  Uploaded.  But 
when a tested failed once, it failed again even with the delay, and contrary to 
the results above.

--
Added file: https://bugs.python.org/file50084/tkinter_testing_utils.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Terry J. Reedy]

Change by Terry J. Reedy :


--
Removed message: https://bugs.python.org/msg394871

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Tal Einat  added the comment:

On Ubuntu 20.04, I've managed to occasionally get this issue to reproduce 
locally by running the test many times in parallel, repeatedly:

parallel -n8 -N0 './python -m test -ugui test_idle -m test_colorizer' ::: $(seq 
20)

This appears to be an issue with our new method of running tests with a Tk 
mainloop in the background, using the @run_in_tk_mainloop decorator.

Investigating further.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Tal Einat  added the comment:

I'm investigating this now. If nothing obvious comes up, I'll revert that PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Tal Einat  added the comment:

The latest PR where changes were made to IDLE's colorizer and its tests, 
including specifically adding the problematic test, is GH-25851.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Tal Einat  added the comment:

Please note that the referenced PR is not the correct one.

I'll add a link to the right one shortly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[STINNER Victor]

STINNER Victor  added the comment:

Buildbots are very unstable these days. I propose to revert the change to give 
more time to IDLE developers to investigate the issue, and to be able to 
identify other regressions on buildbots.
https://pythondev.readthedocs.io/ci.html#revert-on-fail

I created https://github.com/python/cpython/pull/26487 to revert

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Terry J. Reedy]

New submission from Terry J. Reedy :

idlelib.idle_test.test_colordelagator.ColorDelegatorTest.test_incremental_editing
 has failed on at least on the following machines.  

Pip32  CI Pipelines Win32
Gen3x  https://buildbot.python.org/all/#/builders/464  x86 Gentoo Installed 
with X 3.x
GenPR  https://buildbot.python.org/all/#/builders/465  x86 Gentoo Installed 
with X PR

Failures seen.  Since tests are sequential without subtesting, failure means 
successors are not tested.

line  keyword start   wantgot  
569   ('BUILTIN', '1.0'), ('1.0', '1.3')  ()  Gen3x, x3; GenPR
573   ('BUILTIN', '1.0'), ('1.0', '1.3')  ()  Gen3x, x5
586   ('BUILTIN', '1.0'), ('1.0', '1.3')  ()  GenPR
597   ('KEYWORD', '1.0'), ()  ('1.0', '1.1')  Pip32, x2

As far as I can tell, looking at Windows buildbots, the 1 failure on Win32 has 
not been repeated.  If there have been any on CI machines, I have not been 
informed.

A few years ago, I believe Gentoo was the only *nix buildbot running tkinter 
and IDLE tests. It has a problem with a builtin at the beginning of the line 
about 10-20% of test runs.  This are the majority of its failures in the last 
week.  Perhaps there is a timing issue we can fix.

--
assignee: terry.reedy
components: IDLE, Tests
messages: 394870
nosy: taleinat, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE: ColorDelegatorTest test_incremental_editing failures
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Tal Einat]

Change by Tal Einat :


--
keywords: +patch
pull_requests: +25076
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/26404

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44282] IDLE: ColorDelegatorTest test_incremental_editing failures

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Followup to PR-26404, which added the tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com