[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

I'm not 100% sure that all issues are fixed, but the tests seem much more 
reliable yet. I close the issue. If the bug reoccurs, I will reopen the issue 
or open a new one.

--
priority: deferred blocker -> 
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

Should we close this issue now?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[miss-islington]

Change by miss-islington :


--
pull_requests: +6805

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[miss-islington]

miss-islington  added the comment:


New changeset 4b828467a3fcec0c1947e8326f67b8db12a4f303 by Miss Islington (bot) 
in branch '3.7':
bpo-32458: Further tune start_tls tests (GH-7166)
https://github.com/python/cpython/commit/4b828467a3fcec0c1947e8326f67b8db12a4f303


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

While Yury seems unable to reproduce the bug, it's easy for me to reproduce it 
on Linux. After 2 hours of debugging, I found the root issue: a race condition 
not in the test, but in asyncio itself! => bpo-33674 "asyncio: race condition 
in SSLProtocol".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 8267ea2e84d355f00654dec3ad782fc7b1f680f1 by Yury Selivanov in 
branch 'master':
bpo-32458: Further tune start_tls tests (#7166)
https://github.com/python/cpython/commit/8267ea2e84d355f00654dec3ad782fc7b1f680f1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +6801

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

https://github.com/python/cpython/pull/7130 is in.  It includes many tests for 
TLS tests as well as bugfixes to asyncio code.  Hopefully this all will make 
tests more stable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
nosy:  -giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

> OK, sounds good, will do that.

Race condition! I created https://github.com/python/cpython/pull/7157 to 
increase the timeout from 5 seconds to 60 seconds.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +6793

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

> If the purpose is to detect when the test hangs: use a timeout of 5 minutes, 
> or at least 1 minute (60 seconds).

OK, sounds good, will do that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

> Yeah, I agree. The current timeout for the test is 5 seconds and it tries to 
> transfer 2mb of data, which isn't that much actually.  I'll bump the timeout 
> and reduce the amount of data transferred in 
> https://github.com/python/cpython/pull/7130

What is the purpose of a timeout of 5 seconds? Some buildbots are really slow.

If the purpose is to detect when the test hangs: use a timeout of 5 minutes, or 
at least 1 minute (60 seconds).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

> It's a race condition which doesn't depend on the OS, but on the system load.

Yeah, I agree. The current timeout for the test is 5 seconds and it tries to 
transfer 2mb of data, which isn't that much actually.  I'll bump the timeout 
and reduce the amount of data transferred in 
https://github.com/python/cpython/pull/7130

Thing is, these are the very first *functional* tests for asyncio. Before it 
was mostly tested with mocks. So we'll need some adjustment period to learn how 
slow/loaded the CI servers are and what kind of timeouts are safe to use.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

I just reproduced the issue on Linux. Open 3 terminals and run the commands in 
parallel:

(1) ./python -m test test_asyncio -m test_start_tls_server_1 -F
(2) ./python -m test -j16 -r
(3) ./python -m test -j16 -r

It's a race condition which doesn't depend on the OS, but on the system load.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

Christian, I believe 
https://github.com/python/cpython/pull/7130/commits/bd17a5593f03e3f2d39a64f5afd3ffb77c97e914
 fixes the issue with OpenSSL 1.1.1 (socket.shutdown was missing and asyncio's 
protocol didn't receive EOF, instead the connection was terminated (as expected 
though))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Terry J. Reedy]

Terry J. Reedy  added the comment:

'deleted somehow': Christian added himself while I had this page open, so when 
I submitted without refreshing, the nosy list did not include him.  When I saw 
the red warning, I did not notice the nosy list change; I should have.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Christian Heimes]

Christian Heimes  added the comment:

Linux

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

Terry, you somehow deleted Christian from the nosy list.

Christian,

> It's failing reproducible with OpenSSL 1.1.1 and TLS 1.3 enabled. I haven't 
> seen it failing with TLS 1.2 yet.

On Linux or Windows?

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I presume the above is using the month-old openssl-bin-1.1.0h.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I have not seen test_asyncio fail on AppVeyor since about Sunday.  
https://ci.appveyor.com/project/python/cpython/history
shows only a couple of failures, which could very well be real.
I have seen test_asyncio fail on Travis since Sunday.

With my local repository 32-bit debug build of master branch:

If I run python -m test.test_asyncio -v,
test_sock_sendfile_not_regular_file 
(test.test_asyncio.test_proactor_events.ProactorEventLoopUnixSockSendfileTests) 
... (and the test process) aborts with a failed debug assertion:
minkernel\crts\ucrt\src\appcrt\lowio\osfinfo.cpp, line 257
  fh >=  0 && (unsigned)fh < (unsigned)_nhandle

If I run python -m test test_asyncio, I get
F:\dev\3x\lib\asyncio\sslproto.py:320: ResourceWarning: unclosed transport 

  source=self)
F:\dev\3x\lib\asyncio\sslproto.py:320: ResourceWarning: unclosed transport 

  source=self)
Exception in callback 
_ProactorReadPipeTransport._loop_reading__data_received(<_OverlappedFuture 
cancelled>)
handle: )>
Traceback (most recent call last):
  File "F:\dev\3x\lib\asyncio\events.py", line 88, in _run
self._context.run(self._callback, *self._args)
  File "F:\dev\3x\lib\asyncio\proactor_events.py", line 222, in 
_loop_reading__data_received
self._closing)
AssertionError
test_asyncio passed in 35 sec
1 test OK.

With -v, above is scattered in verbose output, but test_start_tls_server_1 
passed both times.

Specifically, I ran the following 10 times with no failures.
f:\dev\3x>python -m test -v -m test_start_tls_server_1 test_asyncio
...
test_start_tls_server_1 (test.test_asyncio.test_sslproto.ProactorStartTLSTests) 
... ok
test_start_tls_server_1 (test.test_asyncio.test_sslproto.SelectorStartTLSTests) 
... ok

--
nosy:  -christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Christian Heimes]

Christian Heimes  added the comment:

It's failing reproducible with OpenSSL 1.1.1 and TLS 1.3 enabled. I haven't 
seen it failing with TLS 1.2 yet.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

> But in msg317468 Victor asserts that it sometimes fails on Linux, too?

Hm, I missed that. I'll definitely take a look.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Ned Deily]

Ned Deily  added the comment:

> Last time I looked into this I couldn't reproduce the failures on my Windows 
> 10 VM, so it seems like an AppVeyor-specific problem. I'll take another look 
> on Monday.

But in msg317468 Victor asserts that it sometimes fails on Linux, too?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

Last time I looked into this I couldn't reproduce the failures on my Windows 10 
VM, so it seems like an AppVeyor-specific problem. I'll take another look on 
Monday.

Andrew, if you have a Windows environment, could you please try to run this 
test?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Ned Deily]

Ned Deily  added the comment:

Yury, are you still planning to address this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

Not at my computer right now, can do it tomorrow.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

> I'm OK to skip it for now. Writing functional tests is super hard because 
> some buildbots are super slow and unpredictable.

Would you mind to write a PR to skip the PR? So you will feel guilty and will 
try to remind to fix it!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Yury Selivanov]

Yury Selivanov  added the comment:

I'm OK to skip it for now. Writing functional tests is super hard because some 
buildbots are super slow and unpredictable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

It would be nice to fix this bug before Python 3.7.0 final: either skip the 
test, or fix it. Flaky tests can be very annoying. For example, the full test 
suite is run to build a Red Hat package. If a single test fails, the package 
build fails and should be retried whereas it's slow.

I understood that fixing the root cause might require to rewrite the test, so I 
don't expect a quick fix on this test. The issue is open since last December... 
Maybe we can skip the test but fix it in Python 3.7.1?

--
priority: high -> deferred blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

test_start_tls_server_1() just failed on my Linux. It likely depends on the 
system load.

--
nosy: +vstinner
title: test_asyncio failures on Windows -> test_asyncio: 
test_start_tls_server_1() fails randomly

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com