[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 676201a59f90caace606d11d4172aa74c1cd4992 by Victor Stinner in 
branch 'main':
bpo-45410: regrtest replaces print_warning.orig_stderr (GH-28926)
https://github.com/python/cpython/commit/676201a59f90caace606d11d4172aa74c1cd4992


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +27215
pull_request: https://github.com/python/cpython/pull/28926

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 773330773968f211c77abc7b5b525faa7b3c35a2 by Victor Stinner in 
branch 'main':
bpo-45410: regrtest -W leaves stdout/err FD unchanged (GH-28915)
https://github.com/python/cpython/commit/773330773968f211c77abc7b5b525faa7b3c35a2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +27205
pull_request: https://github.com/python/cpython/pull/28915

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

STINNER Victor  added the comment:

> bpo-45410: Enhance libregrtest -W/--verbose3 option (GH-28908)

Oh. Sadly, with this change, faulthandler is no longer able to dump a traceback 
on a crash :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

STINNER Victor  added the comment:


New changeset dbe213de7ef28712bbfdb9d94a33abb9c33ef0c2 by Victor Stinner in 
branch 'main':
bpo-45410: Enhance libregrtest -W/--verbose3 option (GH-28908)
https://github.com/python/cpython/commit/dbe213de7ef28712bbfdb9d94a33abb9c33ef0c2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +27199
pull_request: https://github.com/python/cpython/pull/28908

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

STINNER Victor  added the comment:

Calling sys.stdout doesn't help :-(

Fail at commit ce3489cfdb9f0e050bdc45ce5d3902c2577ea683:
https://buildbot.python.org/all/#/builders/73/builds/788
---
0:00:10 load avg: 7.59 [306/427/1] test_ftplib failed (env changed)
Warning -- Uncaught thread exception: Exception
Exception in thread Thread-67:
(...)
Exception
test__all__ (test.test_ftplib.MiscTestCase) ... ok
test_abort (test.test_ftplib.TestFTPClass) ... ok
test_acct (test.test_ftplib.TestFTPClass) ... ok
(...)
---

The warning cannot come from MiscTestCase.test__all__(), so the warning is 
still logged at the wrong place.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 1ebd798fddef51e1f6fd40a4860278a1851ac268 by Victor Stinner in 
branch 'main':
bpo-45410: Add test.support.flush_std_streams() (GH-28885)
https://github.com/python/cpython/commit/1ebd798fddef51e1f6fd40a4860278a1851ac268


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +27179
pull_request: https://github.com/python/cpython/pull/28885

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

STINNER Victor  added the comment:

Sadly, a recent build failure (at commit 
560a79f94e94de66a18f2a5e4194c2fe51e2adf1) still write "Uncaught thread 
exception" at the top of logs:
---
0:00:11 load avg: 6.07 [315/427/1] test_ftplib failed (env changed)
Warning -- Uncaught thread exception: Exception
Exception in thread Thread-67:
(...)
Exception
test__all__ (test.test_ftplib.MiscTestCase) ... ok
test_abort (test.test_ftplib.TestFTPClass) ... ok
(...)
Ran 94 tests in 3.234s
OK (skipped=1)
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b108db63e02797a795840152b82cab9792fd3ed2 by Victor Stinner in 
branch 'main':
bpo-45410: libregrtest -jN writes stderr into stdout (GH-28819)
https://github.com/python/cpython/commit/b108db63e02797a795840152b82cab9792fd3ed2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +27138
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28819

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45410] python -m test -jN: write stderr in stdout to get messages in order

[STINNER Victor]

New submission from STINNER Victor :

In buildbot logs, when a worker process is run, two pipes are created for 
stdout and stderr. The problem is that using two pipes produce an output which 
is hard to read.

Pseudo output:
---
stdout time 1
stdout time 2
stdout time 3
stderr time 1
stderr time 2
---

I would prefer:
---
stdout time 1
stderr time 1
stdout time 2
stderr time 2
stdout time 3
---

Real logs:
https://buildbot.python.org/all/#/builders/79/builds/870
---
0:01:10 load avg: 4.48 [236/427/1] test_ftplib failed (env changed) -- (...)
Warning -- Uncaught thread exception: Exception
Exception in thread Thread-67:
Traceback (most recent call last):
  (...)
Exception
test__all__ (test.test_ftplib.MiscTestCase) ... ok
test_abort (test.test_ftplib.TestFTPClass) ... ok
test_acct (test.test_ftplib.TestFTPClass) ... ok
test_all_errors (test.test_ftplib.TestFTPClass) ... ok
(...)
Ran 94 tests in 2.326s
---

The Warning is logged at the beginning, it's not possible to know which test 
emitted this warning.

I propose to write stdout and stderr of a worker process into a single pipe to 
keep messages order.

In the past, I saw buildbot logging stderr messages in a different color. But 
it no longer seems to be the case.

I'm working on a PR.

--
components: Tests
messages: 403467
nosy: vstinner
priority: normal
severity: normal
status: open
title: python -m test -jN: write stderr in stdout to get messages in order
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com