[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9012

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9012, 9013

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[STINNER Victor]

STINNER Victor  added the comment:

Is test_socket of Python 2.7 affected by the issue? If not, I suggest to remove 
2.7 from this issue and close it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

It looks like the test_socket is fixed but some related issues are still left 
in multiprocessing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[STINNER Victor]

STINNER Victor  added the comment:

Commit merged into master:

commit 7291108d88ea31d205da4db19d202d6cbffc6d93
Author: Pablo Galindo 
Date:   Thu Sep 27 10:25:03 2018 +0100

Fix tests in test_socket to use correctly CMSG_LEN (GH-9594)

After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots
regarding tests in test_socket that are using
testFDPassSeparateMinSpace(), FreeBDS revision 337423 was pointed
out to be the reason the test started to fail.

A close examination of the manpage for cmsg_space(3) reveals that
the number of file descriptors needs to be taken into account when
using CMSG_LEN().

This commit fixes tests in test_socket to use correctly CMSG_LEN, taking
into account the number of FDs.

3.7:

commit addef07ca7d7b6971d59c062c3229e91a99e5f5e (upstream/3.7, 3.7)
Author: Miss Islington (bot) <31488909+miss-isling...@users.noreply.github.com>
Date:   Thu Sep 27 06:30:47 2018 -0700

3.6:

commit fe48b6df101aac10dc846fa6fd1a41f877e77025 (upstream/3.6, 3.6)
Author: Miss Islington (bot) <31488909+miss-isling...@users.noreply.github.com>
Date:   Thu Sep 27 06:30:55 2018 -0700

(the commit message doesn't contain bpo-34521, so bots didn't added comments 
here about the commits)

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[miss-islington]

Change by miss-islington :


--
pull_requests: +8999

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[miss-islington]

Change by miss-islington :


--
pull_requests: +8998
stage: backport needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[Kubilay Kocak]

Kubilay Kocak  added the comment:

Copying in my original email to Pablo that elucidated the test failure cause 
and resolution, including upstream commit/review/issue references for the 
recvmsg(2) changes IN CURRENT.

Full buildbot worker log attached

While I'm here, update list of versions (branches) affected.

--

The test suite fails:

  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/multiprocessing/reduction.py",
line 164, in recvfds
raise RuntimeError('received %d items of ancdata' %
RuntimeError: received 0 items of ancdata

For these two tests (further up in the log [1]) ...

test_fd_transfer
(test.test_multiprocessing_forkserver.WithProcessesTestConnection) ... FAIL
test_large_fd_transfer
(test.test_multiprocessing_forkserver.WithProcessesTestConnection) ... FAIL

Further up we see (in the initial test run) ...

4 tests failed:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn test_socket

Then further up we see:

==
FAIL: testFDPassSeparateMinSpace
(test.test_socket.RecvmsgSCMRightsStreamTest)
--
Traceback (most recent call last):
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3186, in testFDPassSeparateMinSpace
self.checkRecvmsgFDs(2,
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3107, in checkRecvmsgFDs
self.assertEqual(len(fds), numfds)
AssertionError: 1 != 2
==
FAIL: testFDPassSeparateMinSpace
(test.test_socket.RecvmsgIntoSCMRightsStreamTest)
--
Traceback (most recent call last):
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3186, in testFDPassSeparateMinSpace
self.checkRecvmsgFDs(2,
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3107, in checkRecvmsgFDs
self.assertEqual(len(fds), numfds)
AssertionError: 1 != 2
--
Ran 554 tests in 27.636s
FAILED (failures=2, skipped=57)

I looked for recvmsg() changes in FreeBSD recently, and found ...

Commit: https://svnweb.freebsd.org/base?view=revision&revision=337423
Review: https://reviews.freebsd.org/D16561
Bug ID: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=131876
Bug Desc:[socket] FD leak by receiving SCM_RIGHTS by recvmsg with small
control message buffer

The commit/bug/review details above point to recvmsg specific changes,
specifically, but maybe not *just*, for SCM_RIGHTS messages, which (in
my opinion) is way too similar to the test case specifics not to be related.

Also, the commit date of Aug 7 16:36:48 2018  seems right around the
time I updated the machine and the tests began to fail.

I can't say yet whether or not a bug crept in with the change, *but*,
the commit does say that it *fixes* bugs, which the python tests, or the
socket code in Python may be have been relying on.

It may also be that FreeBSD specific-behaviour isn't being
handled by the code and/or tests (truncated now where it wasn't before?)

It could also be the buffer sizes the socket code/tests use.

--

--
keywords: +buildbot
stage: patch review -> backport needed
title: test_socket.RecvmsgIntoSCMRightsStreamTest fails on AMD64 FreeBSD 
CURRENT Debug 3.x -> test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on 
AMD64 FreeBSD CURRENT
versions: +Python 2.7, Python 3.6, Python 3.7, Python 3.8
Added file: 
https://bugs.python.org/file47835/freebsd-current.python.3.x.build.301.stdio.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com