[issue42014] shutil.rmtree calls onerror with different function than failed

2020-11-10 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
nosy:  -giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue42014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30044] shutil.copystat should (allow to) copy ownership, and other attributes

2020-07-20 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Sorry, after re-reading Eryk's comment, it seems I'm not correct about 
CopyFileEx.

--

___
Python tracker 
<https://bugs.python.org/issue30044>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30044] shutil.copystat should (allow to) copy ownership, and other attributes

2020-07-20 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Since the need to copy file ownership is common, I think there could be space 
for a new copy3() function which copies ownership + extended attributes (where 
possible). In detail:

- on Windows this can be achieved by using CopyFileEx, which would also provide 
better overall performances than the current readinto() implementation. This 
was the original approach used in BPO-33671, discarded after Eryk pointed out 
that ACLs should not be copied.

- on macOS we can use fcopyfile() (which is already exposed privately) + 
COPYFILE_ALL, see http://www.manpagez.com/man/3/fcopyfile/

- on all other UNIX we can use the current shutil.copystat() (which already 
copies xattrs) + os.chown(). I am not entirely sure this would achieve a "full 
ACLs copy" though, or what would be needed exactly to achieve that.

--

___
Python tracker 
<https://bugs.python.org/issue30044>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue41271] Add support for io_uring to cpython

2020-07-17 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue41271>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39673] Map errno==ETIME to TimeoutError

2020-05-26 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Sigh! I misread the OP's post and thought the proposal was to add TimeoutError 
which I forgot existed. Sorry for the noise and please disregard my previous 
comment.

--

___
Python tracker 
<https://bugs.python.org/issue39673>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39673] Map errno==ETIME to TimeoutError

2020-05-25 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I'm -1 about TimeoutError because the concept of "timeout" is generic enough to 
be often implemented as a custom exception, which poses questions re. 
backward/forward compatibilty. E.g. in psutil I have "TimeoutExpired", also 
providing a "seconds" attribute. Also I've probably never seen ETIME / 
ETIMEDOUT happening, whereas AFAIU the point of PEP 3151 was to create mappings 
for the most common errnos.

--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue39673>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37873] unittest: execute tests in parallel

2020-04-30 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue37873>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39380] ftplib uses latin-1 as default encoding

2020-03-20 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

+1 from me as well. @SebastianGPedersen could you update the PR (constructor + 
doc changes)?

--

___
Python tracker 
<https://bugs.python.org/issue39380>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4080] unittest: display time used by each test case

2020-02-16 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I updated the PR for Python 3.9. I think it's ready to be merged. Can someone 
involved in unittest take a look at it?

--

___
Python tracker 
<https://bugs.python.org/issue4080>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4080] unittest: display time used by each test case

2020-02-15 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
versions: +Python 3.9 -Python 3.8

___
Python tracker 
<https://bugs.python.org/issue4080>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39488] test_largefile: TestSocketSendfile.test_it() uses too much disk space

2020-01-29 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +17637
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18261

___
Python tracker 
<https://bugs.python.org/issue39488>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39390] shutil.copytree - 3.8 changed argument types of the ignore callback

2020-01-27 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
assignee: docs@python -> giampaolo.rodola
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39390] shutil.copytree - 3.8 changed argument types of the ignore callback

2020-01-27 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset cf9d00554715febf21cf94950da4f42723b80498 by Giampaolo Rodola 
(mbarkhau) in branch '3.8':
[3.8] bpo-39390 shutil: fix argument types for ignore callback (GH-18122)
https://github.com/python/cpython/commit/cf9d00554715febf21cf94950da4f42723b80498


--

___
Python tracker 
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39380] ftplib uses latin-1 as default encoding

2020-01-25 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

It's been a long time since I implemented UTF-8 support in pyftpdlib, but long 
story short is that:
- most recent servers are supposed to use UTF-8 by default
- such servers must include "UTF-8" in the FEAT command response
- some servers may enable UTF-8 only if client explicitly sends "OPTS UTF-8 ON" 
first, but that is based on an draft RFC. Server implementors usually treat 
this command as a no-op and simply assume UTF-8 as the default.

With that said, I am -1 about implementing logic based on FEAT/OPTS: that 
should be done before login, and at that point some servers may erroneously 
reject any command other than USER, PASS and ACCT. 

Personally I think it makes more sense to just use UTF-8 without going through 
a deprecation period, document *encoding* attribute and mention that in order 
to deal with servers not supporting UTF8 you can pre-emptively check FEAT 
response and set ASCII encoding. But I am not a unicode expert, so I would like 
to hear some other opinion re. the implications of going from latin-1 to utf8 
in terms of potential code breakage.

--
nosy: +benjamin.peterson, ezio.melotti, lemburg

___
Python tracker 
<https://bugs.python.org/issue39380>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39390] shutil.copytree - 3.8 changed argument types of the ignore callback

2020-01-24 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I don't think we need to change anything on < 3.8, but 3.8 and 3.9 will always 
convert *src* to str via os.fspath(), which IMO is more consistent (e.g. 
os.path.* functions and others do the same).

--

___
Python tracker 
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39390] shutil.copytree - 3.8 changed argument types of the ignore callback

2020-01-24 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

For completeness, a similar problem is present also on python < 3.8 if passing 
a pathlib.Path type as *src*: the callback function will receive a pathlib.Path 
type once, and then string types.

--

___
Python tracker 
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39390] shutil.copytree - 3.8 changed argument types of the ignore callback

2020-01-24 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 88704334e5262c6cd395a0809d4ef810f33f3ca5 by Giampaolo Rodola 
(mbarkhau) in branch 'master':
bpo-39390 shutil: fix argument types for ignore callback (GH-18122)
https://github.com/python/cpython/commit/88704334e5262c6cd395a0809d4ef810f33f3ca5


--

___
Python tracker 
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39390] shutil.copytree - 3.8 changed argument types of the ignore callback

2020-01-21 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
nosy: +vstinner

___
Python tracker 
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39390] shutil.copytree - 3.8 changed argument types of the ignore callback

2020-01-21 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Yes, thanks. Whoever got bit by this is either getting an exception or not the 
intended behavior (due to failed string comparison). I doubt anybody is relying 
on the new type checking since it's not documented. If they are, they are 
probably just doing:

def callback(name, names):
if not isinstance(name, str):  # bugfix 3.8
name = name.name
...

--

___
Python tracker 
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39390] shutil.copytree - 3.8 changed argument types of the ignore callback

2020-01-21 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

> Should not copytree convert arguments of the ignore callback to str and list 
> correspondingly?

It should. I think it makes sense to just do this for Python 3.8.2 instead of 
updating the doc.

--

___
Python tracker 
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39099] scandir.dirfd() method

2019-12-19 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Good point, I didn't consider that. I suppose you're right. =)
Closing.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39099>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39099] scandir.dirfd() method

2019-12-19 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +17131
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17664

___
Python tracker 
<https://bugs.python.org/issue39099>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39099] scandir.dirfd() method

2019-12-19 Thread Giampaolo Rodola'


New submission from Giampaolo Rodola' :

PR in attachment adds a new dirfd() method to the scandir() object (POSIX 
only). This can be be passed to os.* functions supporting the "dir_fd" 
parameter, and avoid opening a new fd as in:

>>> dirfd = os.open("basename", os.O_RDONLY, dir_fd=topfd)

At the moment I am not sure if it's possible to also support Windows.

--
components: Library (Lib)
messages: 358686
nosy: giampaolo.rodola
priority: normal
severity: normal
status: open
title: scandir.dirfd() method
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39099>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39004] test_largefile: TestSocketSendfile.test_it() failed on AMD64 Fedora Stable Refleaks 3.x

2019-12-10 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 82374979ec7e01e23385dca1d02b2aa3de16fea5 by Giampaolo Rodola in 
branch 'master':
bpo-39004: increment large sendfile() test timeout (GH-17552)
https://github.com/python/cpython/commit/82374979ec7e01e23385dca1d02b2aa3de16fea5


--

___
Python tracker 
<https://bugs.python.org/issue39004>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39004] test_largefile: TestSocketSendfile.test_it() failed on AMD64 Fedora Stable Refleaks 3.x

2019-12-10 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I suspect it's a timing problem (client disconnects, threaded server fails to 
detect that in time). Let's see what happens by incrementing the timeout when 
waiting for the thread to be join()ed (PR-17552).

--

___
Python tracker 
<https://bugs.python.org/issue39004>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39004] test_largefile: TestSocketSendfile.test_it() failed on AMD64 Fedora Stable Refleaks 3.x

2019-12-10 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +17026
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17552

___
Python tracker 
<https://bugs.python.org/issue39004>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38906] copy2 doesn't copy metadata on Windows and MacOS

2019-11-27 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

If we use CopyFileExW in copy2(), then also copystat() and copymode() should be 
able to copy the same metadata/security-bits/etc as CopyFileExW. I don't know 
which Windows APIs should be used though.

I sort of agree with Steven that CopyFileExW could be used everywhere (meaning 
copyfile()), but that basically means breaking backward compatibility re. what 
is promised in the doc, and am not sure how to deal with that. For that reason 
it's probably it's better to leave copyfile() alone. 

On macOS it seems we can use fcopyfile(3) syscall (which is already exposed) 
and its flag argument (COPYFILE_METADATA, COPYFILE_DATA. COPYFILE_XATTR, etc,) 
to implement both copy2() and copystat() / copymode().

--

___
Python tracker 
<https://bugs.python.org/issue38906>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38688] Python 3.8 regression: endless loop in shutil.copytree

2019-11-26 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38688>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38688] Python 3.8 regression: endless loop in shutil.copytree

2019-11-26 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 65c92c5870944b972a879031abd4c20c4f0d7981 by Giampaolo Rodola 
(Bruno P. Kinoshita) in branch '3.8':
[3.8] bpo-38688, shutil.copytree: consume iterator and create list of entries 
to prevent infinite recursion (GH-17397)
https://github.com/python/cpython/commit/65c92c5870944b972a879031abd4c20c4f0d7981


--

___
Python tracker 
<https://bugs.python.org/issue38688>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38688] Python 3.8 regression: endless loop in shutil.copytree

2019-11-26 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
stage: patch review -> commit review
versions: +Python 3.9

___
Python tracker 
<https://bugs.python.org/issue38688>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38688] Python 3.8 regression: endless loop in shutil.copytree

2019-11-26 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 9bbcbc9f6dfe1368fe7330b117707f828e6a2c18 by Giampaolo Rodola 
(Bruno P. Kinoshita) in branch 'master':
bpo-38688, shutil.copytree: consume iterator and create list of entries to 
prevent infinite recursion (GH-17098)
https://github.com/python/cpython/commit/9bbcbc9f6dfe1368fe7330b117707f828e6a2c18


--

___
Python tracker 
<https://bugs.python.org/issue38688>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38688] Python 3.8 regression: endless loop in shutil.copytree

2019-11-21 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

PR-17098 as it stands re-introduces some stat() syscall. I suggest to just 
consume the iterator: it's a small change and it should fix the issue.

--

___
Python tracker 
<https://bugs.python.org/issue38688>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38688] Python 3.8 regression: endless loop in shutil.copytree

2019-11-21 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

The speedup introduced in issue33695 is mostly because the number of os.stat() 
syscall was reduced from 6 to 1 per file (both by using scandir() and because 
stat() results are cached and passed around between function calls). As such, 
even if we immediately consume scandir() iterator I believe it won't have a 
meaningful impact in terms of speed. FWIW issue33695 has a benchmark script 
attached (but it's not very stable).

--

___
Python tracker 
<https://bugs.python.org/issue38688>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38688] Python 3.8 regression: endless loop in shutil.copytree

2019-11-20 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

PR-17098 basically reverts https://bugs.python.org/issue33695. Not good. =) 
I think we can simply consume the iterator immediately as in:

def copytree(src, ...):
with os.scandir(src) as itr:
entries = list(itr)
return _copytree(entries=entries, ...)

--

___
Python tracker 
<https://bugs.python.org/issue38688>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38630] subprocess.Popen.send_signal() should poll the process

2019-11-13 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

To further elaborate on the creation time solution, the idea in pseudo-code is 
the following:

class Popen:

def __init__(self, ...):
self._execute_child(...)
try:
self._ctime = get_create_time(self.pid)
except ProcessLookupError:
self._ctime = None

def send_signal(self, sig):
if self._ctime is not None:
if self._ctime != get_create_time(self.pid):
raise ProecssLookupError("PID has been reused")
os.kill(self.pid, sig)

Technically there is still a race condition between _execute_child() and 
get_create_time(), but:

1) the time between the two calls is supposed to be extremely short
2) an OS should not reuse a PID that fast, precisely in order to minimize the 
chances of such a race condition to occur
3) as a general rule, on most systems PIDs are supposed to be assigned 
sequentially and wrap around. E.g. on Linux:

~$ for i in $(seq 20); do ps; done | grep ps
3199 pts/000:00:00 ps
3200 pts/000:00:00 ps
3201 pts/000:00:00 ps
3202 pts/000:00:00 ps
3203 pts/000:00:00 ps
3204 pts/000:00:00 ps
3205 pts/000:00:00 ps
3207 pts/000:00:00 ps
3208 pts/000:00:00 ps
3209 pts/000:00:00 ps
3210 pts/000:00:00 ps
3213 pts/000:00:00 ps
3214 pts/000:00:00 ps
3215 pts/000:00:00 ps
3216 pts/000:00:00 ps
3217 pts/000:00:00 ps
3218 pts/000:00:00 ps
3219 pts/000:00:00 ps
3221 pts/000:00:00 ps
3223 pts/000:00:00 ps

As for Windows, the termination is based on the process handle instead of the 
PID, so I assume that means Windows is safe in this regard. There was a prior 
discussion about this actually:
https://bugs.python.org/issue36067#msg336243
Getting process creation time is relatively easy, even though 
platform-dependent (and should be done in C).
pidfd API would be nicer to use, but it seems it's not portable and relatively 
recent.

--

___
Python tracker 
<https://bugs.python.org/issue38630>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38630] subprocess.Popen.send_signal() should poll the process

2019-11-03 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

> Did someone propose a pull request to fix this issue by ignoring 
> ProcessLookupError?

I misread your PR, sorry. I thought that was the effect.

--

___
Python tracker 
<https://bugs.python.org/issue38630>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38630] subprocess.Popen.send_signal() should poll the process

2019-11-01 Thread Giampaolo Rodola'

Giampaolo Rodola'  added the comment:

-1 about the PR solution to suppress ProcessLookupError in case the process is 
gone. In psutil I solved the “pid reused problem” by using process creation 
time to identify a process uniquely (on start). 
A decorator can be used to protect the sensibile methods interacting with the 
PID/handle (communicate(), send_signal(), terminate(), kill()) and raise an 
exception (maybe ProcessLooKupError(“PID has been reused”)?).

--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue38630>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38480] resource.setrlimit() should raise PermissionError

2019-10-15 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

It turns out there is a similar precedent which was solved in the same way: 
issue18787.

--

___
Python tracker 
<https://bugs.python.org/issue38480>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38480] resource.setrlimit() should raise PermissionError

2019-10-15 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +16360
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16804

___
Python tracker 
<https://bugs.python.org/issue38480>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38480] resource.setrlimit() should raise PermissionError

2019-10-15 Thread Giampaolo Rodola'


New submission from Giampaolo Rodola' :

>>> import resource
>>> high = 300 * 1024 * 1024
>>> resource.setrlimit(resource.RLIMIT_MEMLOCK, (high, high))
Traceback (most recent call last):
  File "", line 1, in 
ValueError: not allowed to raise maximum limit
>>> 

Internally EPERM is translated into ValueError (should have been 
PermissionError). resource.prlimit(), on the other hand, is not affected 
(correctly raises PermissionError):

>>> resource.prlimit(os.getpid(), resource.RLIMIT_MEMLOCK, (high, high))
Traceback (most recent call last):
  File "", line 1, in 
PermissionError: [Errno 1] Operation not permitted


It seems 'ValueError' was used in order to provide a more informative error 
message, but IMO it was a bad call.
Proposal is to change this in 3.9 only, and document it under whatsnew/porting.

--
components: Library (Lib)
messages: 354701
nosy: giampaolo.rodola
priority: normal
severity: normal
status: open
title: resource.setrlimit() should raise PermissionError
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue38480>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38378] os.sendfile() has improperly named parameter

2019-10-11 Thread Giampaolo Rodola'

Giampaolo Rodola'  added the comment:

I’m for renaming both. Since the function is about transmitting or copying (on 
Linux) files src_fd and dst_fd could also be good candidates.

--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue38378>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38416] test_largefile.TestSocketSendfile.test_it() failed with 15 min timeout on AMD64 FreeBSD 10-STABLE Non-Debug 3.x

2019-10-10 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Thanks.

--
resolution: fixed -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue38416>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38416] test_largefile.TestSocketSendfile.test_it() failed with 15 min timeout on AMD64 FreeBSD 10-STABLE Non-Debug 3.x

2019-10-10 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

> Another approach is to change the timeout of 15 min for this super slow 
> buildbot. For example, set it to 20 min or 30 min.

I think this makes more sense (set timeout to 25 min).

--

___
Python tracker 
<https://bugs.python.org/issue38416>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37096] Add large-file tests for modules using sendfile(2)

2019-10-01 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Sorry about that. I'm attaching a patch which skips those tests if there's not 
enough free disk space. If you confirm it works I'll make a PR.

--
Added file: https://bugs.python.org/file48636/disk-space.patch

___
Python tracker 
<https://bugs.python.org/issue37096>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38319] shutil.copyfile(): os.sendfile() fails with OverflowError on 32-bit system

2019-09-30 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Looks like it worked:
https://buildbot.python.org/all/#/builders/176/builds/1383

--

___
Python tracker 
<https://bugs.python.org/issue38319>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38319] shutil.copyfile(): os.sendfile() fails with OverflowError on 32-bit system

2019-09-30 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
versions: +Python 3.8

___
Python tracker 
<https://bugs.python.org/issue38319>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38319] shutil.copyfile(): os.sendfile() fails with OverflowError on 32-bit system

2019-09-30 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 94e165096fd65e8237e60de570fb609604ab94c9 by Giampaolo Rodola in 
branch 'master':
bpo-38319: Fix shutil._fastcopy_sendfile(): set sendfile() max block size 
(GH-16491)
https://github.com/python/cpython/commit/94e165096fd65e8237e60de570fb609604ab94c9


--

___
Python tracker 
<https://bugs.python.org/issue38319>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38319] shutil.copyfile(): os.sendfile() fails with OverflowError on 32-bit system

2019-09-30 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +16077
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16491

___
Python tracker 
<https://bugs.python.org/issue38319>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37096] Add large-file tests for modules using sendfile(2)

2019-09-29 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
components: +Tests -Library (Lib)
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> enhancement
versions: +Python 3.9 -Python 3.8

___
Python tracker 
<https://bugs.python.org/issue37096>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37096] Add large-file tests for modules using sendfile(2)

2019-09-29 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 5bcc6d89bcb622a6786fff632fabdcaf67dbb4e2 by Giampaolo Rodola in 
branch 'master':
bpo-37096: Add large-file tests for modules using sendfile(2) (GH-13676)
https://github.com/python/cpython/commit/5bcc6d89bcb622a6786fff632fabdcaf67dbb4e2


--

___
Python tracker 
<https://bugs.python.org/issue37096>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38244] example in ftplib.rst uses ftp.debian.org but it's not reachable

2019-09-26 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue38244>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38244] example in ftplib.rst uses ftp.debian.org but it's not reachable

2019-09-26 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset a7414571a113815033fdc9f7abb3f5a24e44c472 by Giampaolo Rodola 
(Prateek Nayak) in branch 'master':
bpo-38244: [Documentation] Fix unreliable link to sever in ftplib.rst (#16319)
https://github.com/python/cpython/commit/a7414571a113815033fdc9f7abb3f5a24e44c472


--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue38244>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37791] Propose to deprecate `ignore_errors` and `onerror` parameters of `shutil.rmtree()`

2019-08-08 Thread Giampaolo Rodola'

Giampaolo Rodola'  added the comment:

There’s too much code out there relying on both args (especially ignore_errors 
in unit tests) that would break for no real benefit.

--

___
Python tracker 
<https://bugs.python.org/issue37791>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33408] Enable AF_UNIX support in Windows

2019-07-10 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue33408>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37366] Add an "onitem" callback parameter to shutil.rmtree()

2019-06-29 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I'm gonna reject this one, sorry Jeffrey. Reasons given in 
https://bugs.python.org/issue36422#msg346878 also apply here.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue37366>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36422] tempfile.TemporaryDirectory() removes entire directory tree even if it's a mount-point

2019-06-29 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

> in case a filesystem has been mounted on the temporary directory, this can 
> lead to the entire filesystem being removed

-1

That is expected behavior and the use case looks pretty unusual. Such a new 
parameter wouldn't even be supported by other "batteries" since there's no 
portable/standard way to either mount or unmount a directory (in fact you had 
to use a subprocess in your unit-tests).

A `delete=bool` parameter would be a more reasonable proposal in principle but:
1) if you want to keep the directory around then you can just use 
tempfile.mkdtemp() (see "there should preferably be only one way to do it")
2) it would conflict with the context manager usage which is expected to delete 
the dir on ctx manager exit

In summary, I think this would over-complicate the API for no good reason. 
I'm closing this out as rejected.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.6

___
Python tracker 
<https://bugs.python.org/issue36422>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37366] Add an "onitem" callback parameter to shutil.rmtree()

2019-06-27 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I am not sure I understand the use case. Other than "unmounting a mount point" 
the other scenarios look unrealistic to me and also require a fd instead of a 
path. Also, if you want to unmount a path you can do so via *onerror* instead 
(unmount + delete). Pre-emptively checking if you have the permission to do 
something is sort of unpythonic and racy.

--
versions:  -Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue37366>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37157] shutil: add reflink=False to file copy functions to control clone/CoW copies (use copy_file_range)

2019-06-05 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Adding a new patch (still a PoC, will create a PR when I have something more 
solid).

--
Added file: https://bugs.python.org/file48393/cow2.diff

___
Python tracker 
<https://bugs.python.org/issue37157>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37157] shutil: add reflink=False to file copy functions to control clone/CoW copies (use copy_file_range)

2019-06-05 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

> I'm not sure that attempt to call unlink() if FICLONE fails is a good idea
Agreed.


> I dislike the *fallback* parameter of reflink().

Me too. A specific exception is better.


> Why not exposing clonefile() as os.clonefile() but os._clonefile()?

Mmm... I'm not sure it's worth it. The only reason one may want to use 
clonefile() directly is for passing CLONE_NOFOLLOW and CLONE_NOOWNERCOPY flags 
(the only possible ones):
 
- CLONE_NOFOLLOW can be exposed via "follow_symlinks=True" (like other shutil.* 
functions) and used internally
- CLONE_NOOWNERCOPY should also be passed internally by default because all 
other functions of shutil do not copy ownership (there's a warning at the top 
of the doc), so I think it makes sense for reflink() to do the same.
 

> +#if defined(MAC_OS_X_VERSION_10_12): Would it be possible to use a runtime 
> check?

Good point. It should definitively be loaded at runtime. I will look into that 
(but not soon).

--

___
Python tracker 
<https://bugs.python.org/issue37157>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37159] Use copy_file_range() in shutil.copyfile() (server-side copy)

2019-06-05 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Nope, it doesn't (see man page). We can simply use FICLONE (cp does the same).

--

___
Python tracker 
<https://bugs.python.org/issue37159>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37159] Use copy_file_range() in shutil.copyfile() (server-side copy)

2019-06-05 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

issue37157 is for reflink / CoW copy, this one is not.

--

___
Python tracker 
<https://bugs.python.org/issue37159>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37159] Use copy_file_range() in shutil.copyfile() (server-side copy)

2019-06-04 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
title: Have shutil.copyfile() use copy_file_range() -> Use copy_file_range() in 
shutil.copyfile() (server-side copy)

___
Python tracker 
<https://bugs.python.org/issue37159>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37159] Have shutil.copyfile() use copy_file_range()

2019-06-04 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
nosy: +StyXman, desbma, facundobatista, martin.panter, ncoghlan, neologix, 
pablogsal, petr.viktorin, vstinner

___
Python tracker 
<https://bugs.python.org/issue37159>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26826] Expose new copy_file_range() syscall in os module.

2019-06-04 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

> Please open a new issue to discuss how it can used in shutil ;-)

Use copy_file_range() in shutil.copyfile():
https://bugs.python.org/issue37159

--

___
Python tracker 
<https://bugs.python.org/issue26826>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37159] Have shutil.copyfile() use copy_file_range()

2019-06-04 Thread Giampaolo Rodola'


New submission from Giampaolo Rodola' :

This is a follow up of issue33639 (zero-copy via sendfile()) and issue26828 
(os.copy_file_range()). On [Linux 4.5 / glib 2.27] shutil.copyfile() will use 
os.copy_file_range() instead of os.sendfile(). According to my benchmarks 
performances are the same but when dealing with NFS copy_file_range() is 
supposed to attempt doing a server-side copy, meaning there will be no exchange 
of data between client and server, making the copy operation an order of 
magnitude faster.

Before proceeding unit-tests for big-file support should be added first 
(issue37096). We didn't hit the 3.8 deadline but I actually prefer to land this 
in 3.9 as I want to experiment with it a bit (copy_file_range() is quite new, 
issue26828 is still a WIP).

--
components: Library (Lib)
files: patch.diff
keywords: patch
messages: 344671
nosy: giampaolo.rodola
priority: normal
severity: normal
status: open
title: Have shutil.copyfile() use copy_file_range()
type: performance
versions: Python 3.9
Added file: https://bugs.python.org/file48392/patch.diff

___
Python tracker 
<https://bugs.python.org/issue37159>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37157] shutil: add reflink=False to file copy functions to control clone/CoW copies (use copy_file_range)

2019-06-04 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I'm attaching an initial PoC using FICLONE on Linux and clonefile(3) on OSX. It 
is also possible to support Windows but it requires a ReFS partition to test 
against which I currently don't have.  I opted for exposing reflink() as a 
separate function, mostly because:

- conceptually standard copy and CoW copy are 2 different things
- shutil already provides a distinction between copy functions (copy(), 
copy2(), copyfile()) which can be used as callbacks for copytree() and move(). 
As such one can follow the same approach and do:

   >>> copytree(src, dst, copy_function=reflink).

This initial patch provides a callback=None parameter in case the CoW operation 
fails because not supported by the underlying filesystems but this is debatable 
because we can get different errors depending on the platform (which is not 
good). As such a more generic ReflinkNotSupportedError exception is probably a 
better choice.

--
keywords: +patch
Added file: https://bugs.python.org/file48390/cow.diff

___
Python tracker 
<https://bugs.python.org/issue37157>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26826] Expose new copy_file_range() syscall in os module.

2019-06-04 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Actually "man copy_file_range" claims it can do server-side copy, meaning no 
network traffic between client and server if *src* and *dst* live on the same 
network fs. So I agree copy_file_range() should be preferred over sendfile() 
after all. =)
I have a wrapper for copy_file_range() similar to what I did in shutil in 
issue33671 which I can easily integrate, but I wanted to land this one first:
https://bugs.python.org/issue37096
Also, I suppose we cannot land this in time for 3.8?

--

___
Python tracker 
<https://bugs.python.org/issue26826>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26826] Expose new copy_file_range() syscall in os module.

2019-06-04 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I think data deduplication / CoW / reflink copy is better implemented via 
FICLONE. "cp --reflink" uses it, I presume because it's older than 
copy_file_range(). I have a working patch adding CoW copy support for Linux and 
OSX (but not Windows). I think that should be exposed as a separate 
shutil.reflink() though, and copyfile() should just do a standard copy.

--

___
Python tracker 
<https://bugs.python.org/issue26826>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26826] Expose new copy_file_range() syscall in os module.

2019-06-04 Thread Giampaolo Rodola'

Giampaolo Rodola'  added the comment:

shutil.copyfile() already uses sendfile() which basically provides the same 
performances. sendfile() should be preferred though because it’s supported 
since Linux 2.6.33.

--

___
Python tracker 
<https://bugs.python.org/issue26826>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34368] ftplib __init__ function can't handle 120 or 4xy reply when connect to the server

2019-05-31 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

>From RFC-959:

  If the server is unable to accept input right away, a
  120 "expected delay" reply should be sent immediately and a 220
  reply when ready.  The user will then know not to hang up if there
  is a delay.

Seems to make sense. Willing to provide a patch?

--

___
Python tracker 
<https://bugs.python.org/issue34368>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37096] Add large-file tests for modules using sendfile(2)

2019-05-30 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +13564
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/13676

___
Python tracker 
<https://bugs.python.org/issue37096>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37096] Add large-file tests for modules using sendfile(2)

2019-05-30 Thread Giampaolo Rodola'


New submission from Giampaolo Rodola' :

The need for this emerged in:
https://bugs.python.org/issue36610#msg343948.
We currently use sendfile(2) syscall in high-level shutil.copyfile() on Linux 
and socket.sendfile() on UNIX. In addition this PR also tests shutil.copyfile() 
implementation on OSX which takes advantage of fcopyfile(2) syscall. The goal 
is to make sure "fast-copy" syscalls can handle files > 2G. On my Ubuntu 18.04 
with SSD disk this introduces a 5 secs slowdown to the test run.

According to:
https://github.com/golang/go/issues/13892
...problems may arise on SunOS, so if a BB will turn red this is expected.

--
components: Library (Lib)
messages: 343951
nosy: giampaolo.rodola, rosslagerwall
priority: normal
severity: normal
status: open
title: Add large-file tests for modules using sendfile(2)
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue37096>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36610] os.sendfile can return EINVAL on Solaris

2019-05-30 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
assignee:  -> giampaolo.rodola
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue36610>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36610] os.sendfile can return EINVAL on Solaris

2019-05-30 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 413d955f8ec88a7183f91d7ad8b0ff7def803de3 by Giampaolo Rodola in 
branch 'master':
bpo-36610: shutil.copyfile(): use sendfile() on Linux only (GH-13675)
https://github.com/python/cpython/commit/413d955f8ec88a7183f91d7ad8b0ff7def803de3


--

___
Python tracker 
<https://bugs.python.org/issue36610>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24564] shutil.copytree fails when copying NFS to NFS

2019-05-30 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 2.7

___
Python tracker 
<https://bugs.python.org/issue24564>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24564] shutil.copytree fails when copying NFS to NFS

2019-05-29 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset f1487b323549e2360460383b4304f6592fb38e27 by Giampaolo Rodola 
(Miss Islington (bot)) in branch '3.7':
bpo-24564: shutil.copystat(): ignore EINVAL on os.setxattr() (GH-13369)
https://github.com/python/cpython/commit/f1487b323549e2360460383b4304f6592fb38e27


--

___
Python tracker 
<https://bugs.python.org/issue24564>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36610] os.sendfile can return EINVAL on Solaris

2019-05-29 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

I currently have no Solaris box to test this against and am currently short on 
time but I'm of the opinion that because of:

   > Sendfile on Solaris can raise EINVAL if offset is equal or bigger than the 
size
   > of the file (Python expects that it will return 0 bytes sent in that case).

...and the fact that beta1 will happen soon it's safer to use sendfile() on 
Linux only. Googling for "solaris + sendfile" also raises suspicions that 
sendfile() may be broken on Solaris. shutil.copyfile() is too central to risk 
breaking it. 

We may also want to look at socket.sendfile() implementation and add tests for 
large files for both functions (socket.sendfile() and shutil.copyfile()). I'm 
adding that to my TODO list.

--
versions:  -Python 3.7

___
Python tracker 
<https://bugs.python.org/issue36610>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36610] os.sendfile can return EINVAL on Solaris

2019-05-29 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
pull_requests: +13563
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/13675

___
Python tracker 
<https://bugs.python.org/issue36610>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24564] shutil.copytree fails when copying NFS to NFS

2019-05-29 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset a16387ab2d85f19665920bb6ff91a7e57f59dd2a by Giampaolo Rodola 
(Ying Wang) in branch 'master':
bpo-24564: shutil.copystat(): ignore EINVAL on os.setxattr() (GH-13369)
https://github.com/python/cpython/commit/a16387ab2d85f19665920bb6ff91a7e57f59dd2a


--

___
Python tracker 
<https://bugs.python.org/issue24564>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36997] Document that spwd is considered harmful

2019-05-21 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue36997>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

2019-05-13 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 0a5b88e7f23b671d63896619b13148b0e4e2b5dd by Giampaolo Rodola 
(Miss Islington (bot)) in branch '3.7':
bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only 
files. (PR-13212) (#13234)
https://github.com/python/cpython/commit/0a5b88e7f23b671d63896619b13148b0e4e2b5dd


--

___
Python tracker 
<https://bugs.python.org/issue24538>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22021] shutil.make_archive() root_dir do not work

2019-05-13 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

@cheryl.sabella don't have the bandwidth right now (traveling), sorry.

--

___
Python tracker 
<https://bugs.python.org/issue22021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

2019-05-09 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 79efbb719383386051c72f2ee932eeca8e033e6b by Giampaolo Rodola 
(Olexa Bilaniuk) in branch 'master':
bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only 
files. (PR-13212)
https://github.com/python/cpython/commit/79efbb719383386051c72f2ee932eeca8e033e6b


--

___
Python tracker 
<https://bugs.python.org/issue24538>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

2019-05-09 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
versions: +Python 3.7, Python 3.8 -Python 3.2, Python 3.3, Python 3.4

___
Python tracker 
<https://bugs.python.org/issue24538>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24538] os.setxattr PermissionError on panfs propagates up causing `copystat`, `copytree`, and `pip install .` to fail unhepfully

2019-05-09 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Patch LGTM. I'd like to point out one thing for posterity. Current shutil code 
catches EPERM but not EACCES. While reviewing the patch I wondered whether 
simply catching (and ignoring) both error codes instead, but it turns out they 
are supposed to have 2 different meanings:
https://stackoverflow.com/a/35879961/376587
Especially after _copyxattr is moved before chmod() EACCES (permission denied) 
should propagate instead of being silenced, and EPERM should be left in place 
because in this case EPERM acts more like an ENOTSUP (operation not supported) 
than EACCES (permission denied) and as such should rightly be ignored.

--

___
Python tracker 
<https://bugs.python.org/issue24538>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36823] shutil.copytree copies directories and files but fails with that same directory with '[Errno 1] Operation not permitted')

2019-05-09 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

You should try to copy those offending files individually via shutil.copy2() 
instead copytree() in order to get the full exception.

--

___
Python tracker 
<https://bugs.python.org/issue36823>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36850] shutil.copy2 fails with even with source network filesystem not supporting extended attributes

2019-05-09 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +13133
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue36850>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36488] os.sendfile() on BSD, macOS don't return bytes sent on EINTR

2019-04-12 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +12734
stage: needs patch -> patch review

___
Python tracker 
<https://bugs.python.org/issue36488>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36610] os.sendfile can return EINVAL on Solaris

2019-04-12 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Can you paste the traceback or are you able to reproduce the bug via a script? 
sendfile implementation is supposed to giveup if no data was sent on first 
call, so I suppose this happen later? If for any reason it turns out sendfile() 
is broken on Solaris or behave differently than on Linux I think I prefer to be 
safe than sorry and support it on Linux only.

--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue36610>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35934] Add socket.create_server() utility function

2019-04-09 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

No problem. My bad that I mistyped the "BPO" thing.

--

___
Python tracker 
<https://bugs.python.org/issue35934>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35934] Add socket.create_server() utility function

2019-04-09 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Those are old failures (+/- 17 hours ago). I committed:
https://github.com/python/cpython/pull/12735
...which fixed them.

--

___
Python tracker 
<https://bugs.python.org/issue35934>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17561] Add socket.bind_socket() convenience function

2019-04-09 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Fixed. There's a remaining failing BB:
https://buildbot.python.org/all/#/builders/176/builds/185/steps/4/logs/stdio
...but the failure appears unrelated and it has been red for a while.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue17561>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36570] ftplib timeouts for misconfigured server

2019-04-09 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Please no. The mistake should be fixed in the misconfigured FTP server, not in 
ftplib, which should not make this event pass unnoticed or take arbitrary 
decisions. Filezilla is different than ftplib in that it's an end-user app. 
ftplib is not (e.g. it can be used for testing FTP server implementations, like 
pyftpdlib).

A possible alternative is making socket.create_connection() show the address in 
case of error, like we do in socket.create_server:
https://github.com/python/cpython/blob/8702b67dad62a9084f6c1823dce10653743667c8/Lib/socket.py#L803-L805
That would at least provide a hint on what's the real issue. If you want to 
provide a patch for that you have my blessing.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue36570>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1756] -m broken in trunk

2019-04-08 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Sorry, I accidentally referenced this BPO issue in my commit.

--

___
Python tracker 
<https://bugs.python.org/issue1756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17561] Add socket.bind_socket() convenience function

2019-04-08 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 8702b67dad62a9084f6c1823dce10653743667c8 by Giampaolo Rodola in 
branch 'master':
BPO-17561: set create_server backlog default to None (GH-12735)
https://github.com/python/cpython/commit/8702b67dad62a9084f6c1823dce10653743667c8


--

___
Python tracker 
<https://bugs.python.org/issue17561>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1756] -m broken in trunk

2019-04-08 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset 8702b67dad62a9084f6c1823dce10653743667c8 by Giampaolo Rodola in 
branch 'master':
BPO-17561: set create_server backlog default to None (GH-12735)
https://github.com/python/cpython/commit/8702b67dad62a9084f6c1823dce10653743667c8


--
nosy: +giampaolo.rodola

___
Python tracker 
<https://bugs.python.org/issue1756>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17561] Add socket.bind_socket() convenience function

2019-04-08 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
pull_requests: +12658
stage: resolved -> patch review

___
Python tracker 
<https://bugs.python.org/issue17561>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17561] Add socket.bind_socket() convenience function

2019-04-08 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

Patch committed as of:
https://github.com/python/cpython/commit/eb7e29f2a9d075accc1ab3faf3612ac44f5e2183
For posterity, since the API evolved since the original proposal (as per PR 
review/suggestions), this is the final incarnation:

# IPv4 only
>>> socket.create_server(addr)  
# IPv6 only
>>> socket.create_server(addr, family=socket.AF_INET6)
# IPv4 + IPv6
>>> socket.create_server(addr, family=socket.AF_INET6, dualstack_ipv6=True)
# IPv4/6 if possible and don't care about IPv4 mapped addresses, else IPv4
>>> if socket.has_dualstack_ipv6():
...s = socket.create_server(addr, family=socket.AF_INET6, 
dualstack_ipv6=True)
... else:
...s = socket.create_server(addr)

--

___
Python tracker 
<https://bugs.python.org/issue17561>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17561] Add socket.bind_socket() convenience function

2019-04-08 Thread Giampaolo Rodola'


Change by Giampaolo Rodola' :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue17561>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35934] Add socket.create_server() utility function

2019-04-08 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:


New changeset eb7e29f2a9d075accc1ab3faf3612ac44f5e2183 by Giampaolo Rodola in 
branch 'master':
bpo-35934: Add socket.create_server() utility function (GH-11784)
https://github.com/python/cpython/commit/eb7e29f2a9d075accc1ab3faf3612ac44f5e2183


--

___
Python tracker 
<https://bugs.python.org/issue35934>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   9   10   >