[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Olexa Bilaniuk]

Olexa Bilaniuk  added the comment:

Yes, it may be closed. The fix was merged in time to make it into Python 3.7.4 
and 3.8.0, and solved the problems we were facing with fancy filesystem xattrs 
on read-only files.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Irit Katriel]

Irit Katriel  added the comment:

This seems complete, can it be closed?

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:


New changeset 0a5b88e7f23b671d63896619b13148b0e4e2b5dd by Giampaolo Rodola 
(Miss Islington (bot)) in branch '3.7':
bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only 
files. (PR-13212) (#13234)
https://github.com/python/cpython/commit/0a5b88e7f23b671d63896619b13148b0e4e2b5dd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[miss-islington]

Change by miss-islington :


--
pull_requests: +13144

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:


New changeset 79efbb719383386051c72f2ee932eeca8e033e6b by Giampaolo Rodola 
(Olexa Bilaniuk) in branch 'master':
bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only 
files. (PR-13212)
https://github.com/python/cpython/commit/79efbb719383386051c72f2ee932eeca8e033e6b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
versions: +Python 3.7, Python 3.8 -Python 3.2, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Patch LGTM. I'd like to point out one thing for posterity. Current shutil code 
catches EPERM but not EACCES. While reviewing the patch I wondered whether 
simply catching (and ignoring) both error codes instead, but it turns out they 
are supposed to have 2 different meanings:
https://stackoverflow.com/a/35879961/376587
Especially after _copyxattr is moved before chmod() EACCES (permission denied) 
should propagate instead of being silenced, and EPERM should be left in place 
because in this case EPERM acts more like an ENOTSUP (operation not supported) 
than EACCES (permission denied) and as such should rightly be ignored.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Olexa Bilaniuk]

Olexa Bilaniuk  added the comment:

This old bug now has a PR on Github authored by myself that may be reviewed.

--
nosy: +giampaolo.rodola, obilaniu, tarek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +13123
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Maxime Boissonneault]

Changes by Maxime Boissonneault :


--
nosy: +Maxime Boissonneault

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Bart Oldeman]

Bart Oldeman added the comment:

The issue can be avoided by calling _copyxattr *before* instead of after 
os.chmod in shutil.copystat. That way the file is still writable.

The same issue happens on the Lustre parallel file system, with the lustre.lov 
extended attribute.

--
nosy: +Bart Oldeman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Gerrit Holl]

Changes by Gerrit Holl topjakl...@gmail.com:


--
type:  - crash

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24538
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Gerrit Holl]

New submission from Gerrit Holl:

`shutil.copystat` fails on [panfs](https://en.wikipedia.org/wiki/Panasas#PanFS) 
if the source file lacks u+w, because setting extended attributes results in a 
`PermissionError`.  This leads to higher end functions such as 
`shutil.copytree` to fail.  More seriously, it leads to `pip install .` to 
fail, as I [reported here](https://github.com/pypa/pip/issues/2941):

In [55]: shutil.copystat(/src/on/panfs, /tmp/fubar)
---
PermissionError   Traceback (most recent call last)
ipython-input-55-139c9fc77184 in module()
 1 
shutil.copystat(/home/users/gholl/checkouts/pyatmlab/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.pack,
 /tmp/fubar)

/home/users/gholl/lib/python3.4/shutil.py in copystat(src, dst, 
follow_symlinks)
211 else:
212 raise
-- 213 _copyxattr(src, dst, follow_symlinks=follow)
214
215 def copy(src, dst, *, follow_symlinks=True):

/home/users/gholl/lib/python3.4/shutil.py in _copyxattr(src, dst, 
follow_symlinks)
151 try:
152 value = os.getxattr(src, name, 
follow_symlinks=follow_symlinks)
-- 153 os.setxattr(dst, name, value, 
follow_symlinks=follow_symlinks)
154 except OSError as e:
155 if e.errno not in (errno.EPERM, errno.ENOTSUP, 
errno.ENODATA):

PermissionError: [Errno 13] Permission denied: '/tmp/fubar'

This occurs for any source file where the user write bit is not set.

Now, I'm not sure if this should be addressed in `pip`, `shutil.copytree`, 
`shutil.copystat`, `setxattr`, `panfs`, or in none of the above.  I'm reporting 
it in different places; please close this issue if this is the wrong place.

--
components: Library (Lib)
messages: 245985
nosy: Gerrit.Holl
priority: normal
severity: normal
status: open
title: os.setxattr PermissionError on panfs propagates up causing `copystat`, 
`copytree`, and `pip install .` to fail unhepfully
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24538
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[Gerrit Holl]

Gerrit Holl added the comment:

Perhaps the solution would be some kind of flag, at least for copytree and 
possibly others, on what to do when attributes cannot be completely copied — a 
bit like numpys options to raise, warn, or ignore?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24538
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

[R. David Murray]

R. David Murray added the comment:

There are a couple of related open issues.  I think there is an stdlib problem 
here, but Im not sure what the solution is.

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24538
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com