[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

STINNER Victor  added the comment:

I merged my fix. I close the issue and hope that the test will not fail again 
;-)

Python 2.7 is not affected: it doesn't have test_chown().

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 1d4b6ba19466aba0eb91c4ba01ba509acf18c723 by Victor Stinner (Miss 
Islington (bot)) in branch '3.7':
bpo-37400: Fix test_os.test_chown() (GH-14374) (GH-14378)
https://github.com/python/cpython/commit/1d4b6ba19466aba0eb91c4ba01ba509acf18c723


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[miss-islington]

miss-islington  added the comment:


New changeset 12d174bed9960ded1d072035c57f82e10a89f0d6 by Miss Islington (bot) 
in branch '3.8':
bpo-37400: Fix test_os.test_chown() (GH-14374)
https://github.com/python/cpython/commit/12d174bed9960ded1d072035c57f82e10a89f0d6


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[miss-islington]

Change by miss-islington :


--
pull_requests: +14192
pull_request: https://github.com/python/cpython/pull/14377

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[miss-islington]

Change by miss-islington :


--
pull_requests: +14193
pull_request: https://github.com/python/cpython/pull/14378

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

STINNER Victor  added the comment:


New changeset d7c87d982d4ec4ba201bcee14324ae5e0e90581f by Victor Stinner in 
branch 'master':
bpo-37400: Fix test_os.test_chown() (GH-14374)
https://github.com/python/cpython/commit/d7c87d982d4ec4ba201bcee14324ae5e0e90581f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

STINNER Victor  added the comment:

I'm connected to the FreeBSD CURRENT buildbot as the user "haypo". I'm unable 
to reproduce the bug:

CURRENT-amd64% id
uid=1003(haypo) gid=1003(haypo) groups=1003(haypo)

CURRENT-amd64% ./python -m test test_os -m test_chown
(...)
Tests result: SUCCESS

Moreover, it seems like something changes on the buildbot, since the user 
"buildbot" now has 0 group!?

$ ./python
>>> [g.gr_gid for g in grp.getgrall() if 'buildbot' in g.gr_mem]
[]

Or maybe as the user "haypo", I cannot see *all* groups.

In case of doubt, I will blindly apply my fix PR 14374.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +14189
pull_request: https://github.com/python/cpython/pull/14374

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

STINNER Victor  added the comment:

I merged my pythoninfo change, more complete pythoninfo from the buildbot:

os.getgid: 1002
os.getgrouplist: 1002
os.getgroups: 1002
os.getuid: 1002

pwd.getpwuid(1002): pwd.struct_passwd(pw_name='buildbot', pw_passwd='*', 
pw_uid=1002, pw_gid=1002, pw_gecos='FreeBSD BuildBot', pw_dir='/home/buildbot', 
pw_shell='/bin/sh')

IMHO the test is wrong: it should rely on os.getgroups() rather tan "discover" 
groups from pw.getpwall().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9cb274114c844f9b1c13028f812926c987a7b4a7 by Victor Stinner in 
branch 'master':
bpo-37400: pythoninfo logs getpwuid and getgrouplist (GH-14373)
https://github.com/python/cpython/commit/9cb274114c844f9b1c13028f812926c987a7b4a7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[Kubilay Kocak]

Kubilay Kocak  added the comment:

And I just remembered that I had to restart the build worker service on that 
host (koobs-freebsd10) this morning as I was receiving messaging that the 
worker had gone missing.

I ran `[koobs@10-STABLE-amd64:~] sudo /usr/local/etc/rc.d/buildslave restart`

which is the same as the prevailing reproduction case in #27838

I could restart the worker to make the issue go away, but I think the 
underlying issue should be fixed instead.

Should we close this as a dupe and reopen the original?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[Kubilay Kocak]

Kubilay Kocak  added the comment:

This looks like a reincarnation of #27838

--
nosy: +koobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +14188
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14373

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37400] test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[STINNER Victor]

New submission from STINNER Victor :

https://buildbot.python.org/all/#/builders/167/builds/1265

test_chown (test.test_os.ChownFileTests) ... ERROR

==
ERROR: test_chown (test.test_os.ChownFileTests)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_os.py",
 line 1327, in test_chown
os.chown(support.TESTFN, uid, gid_1)
PermissionError: [Errno 1] Operation not permitted: '@test_95158_tmp'

Extract of the test:

import grp
groups = [g.gr_gid for g in grp.getgrall() if getpass.getuser() in g.gr_mem]
if hasattr(os, 'getgid'):
process_gid = os.getgid()
if process_gid not in groups:
groups.append(process_gid)

@unittest.skipUnless(len(groups) > 1, "test needs more than one group")
def test_chown(self):
gid_1, gid_2 = groups[:2]
uid = os.stat(support.TESTFN).st_uid
os.chown(support.TESTFN, uid, gid_1)  # < FAIL HERE
gid = os.stat(support.TESTFN).st_gid
self.assertEqual(gid, gid_1)
os.chown(support.TESTFN, uid, gid_2)
gid = os.stat(support.TESTFN).st_gid
self.assertEqual(gid, gid_2)

Extract of test.pythoninfo:

os.uid: 1002
os.umask: 077
os.gid: 1002
os.groups: 1002


I'm not sure that the code to manually get groups from grp.getgrall() is 
correct. Why not relying on the *current* groups, os.getgroups()?

Note: there is also os.getgrouplist(), I'm not sure of the difference between 
os.getgroups() and os.getgrouplist(). I know that os.getgrouplist() was 
modified recently to use:

/*
 * NGROUPS_MAX is defined by POSIX.1 as the maximum
 * number of supplimental groups a users can belong to.
 * We have to increment it by one because
 * getgrouplist() returns both the supplemental groups
 * and the primary group, i.e. all of the groups the
 * user belongs to.
 */
ngroups = 1 + MAX_GROUPS;

--
components: Tests
messages: 346508
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE 
Non-Debug 3.* buildbots
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com