Giampaolo Rodola' added the comment:
Hello.
I added some initial comments to the PR, but I'm sort of skeptical about this.
It must be noted that:
1) very few FTP servers probably support this feature
(https://en.wikipedia.org/wiki/File_Transfer_Protocol#Data_transfer_modes)
2) the specs
Giampaolo Rodola' added the comment:
> So using copy_file_range (if available) should be the most generic solution,
> which includes copy-on-write support, and server-side copy support.
Doesn't this imply to pass some flag to copy_file_range()? "man
copy_file_range" sa
Change by Giampaolo Rodola' :
--
pull_requests: +24674
pull_request: https://github.com/python/cpython/pull/26024
___
Python tracker
<https://bugs.python.org/issue43
Giampaolo Rodola' added the comment:
> The question seems to be is if it should be okay to _GiveUpOnFastCopy after a
> partial (incomplete) copy has already occurred via sendfile.
I think it should not. For posterity: my rationale for introducing
_USE_CP_SENDFILE was to allow
Change by Giampaolo Rodola' :
--
nosy: -giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue42014>
___
___
Python-bugs-list mailing list
Unsub
Giampaolo Rodola' added the comment:
Sorry, after re-reading Eryk's comment, it seems I'm not correct about
CopyFileEx.
--
___
Python tracker
<https://bugs.python.org/issue30
Giampaolo Rodola' added the comment:
Since the need to copy file ownership is common, I think there could be space
for a new copy3() function which copies ownership + extended attributes (where
possible). In detail:
- on Windows this can be achieved by using CopyFileEx, which would also
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue41271>
___
___
Python-bugs-list mailing list
Unsub
Giampaolo Rodola' added the comment:
Sigh! I misread the OP's post and thought the proposal was to add TimeoutError
which I forgot existed. Sorry for the noise and please disregard my previous
comment.
--
___
Python tracker
<ht
Giampaolo Rodola' added the comment:
I'm -1 about TimeoutError because the concept of "timeout" is generic enough to
be often implemented as a custom exception, which poses questions re.
backward/forward compatibilty. E.g. in psutil I have "TimeoutExpired", also
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue37873>
___
___
Python-bugs-list mailing list
Unsub
Giampaolo Rodola' added the comment:
+1 from me as well. @SebastianGPedersen could you update the PR (constructor +
doc changes)?
--
___
Python tracker
<https://bugs.python.org/issue39
Giampaolo Rodola' added the comment:
I updated the PR for Python 3.9. I think it's ready to be merged. Can someone
involved in unittest take a look at it?
--
___
Python tracker
<https://bugs.python.org/issue4
Change by Giampaolo Rodola' :
--
versions: +Python 3.9 -Python 3.8
___
Python tracker
<https://bugs.python.org/issue4080>
___
___
Python-bugs-list mailin
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +17637
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/18261
___
Python tracker
<https://bugs.python.org/issu
Change by Giampaolo Rodola' :
--
assignee: docs@python -> giampaolo.rodola
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python
Giampaolo Rodola' added the comment:
New changeset cf9d00554715febf21cf94950da4f42723b80498 by Giampaolo Rodola
(mbarkhau) in branch '3.8':
[3.8] bpo-39390 shutil: fix argument types for ignore callback (GH-18122)
https://github.com/python/cpython/commit
Giampaolo Rodola' added the comment:
It's been a long time since I implemented UTF-8 support in pyftpdlib, but long
story short is that:
- most recent servers are supposed to use UTF-8 by default
- such servers must include "UTF-8" in the FEAT command response
- some servers may en
Giampaolo Rodola' added the comment:
I don't think we need to change anything on < 3.8, but 3.8 and 3.9 will always
convert *src* to str via os.fspath(), which IMO is more consistent (e.g.
os.path.* functions and others do the s
Giampaolo Rodola' added the comment:
For completeness, a similar problem is present also on python < 3.8 if passing
a pathlib.Path type as *src*: the callback function will receive a pathlib.Path
type once, and then string types.
--
___
Pyt
Giampaolo Rodola' added the comment:
New changeset 88704334e5262c6cd395a0809d4ef810f33f3ca5 by Giampaolo Rodola
(mbarkhau) in branch 'master':
bpo-39390 shutil: fix argument types for ignore callback (GH-18122)
https://github.com/python/cpython/commit/88704334e5262c6cd395a0809d4ef810f33f3ca5
Change by Giampaolo Rodola' :
--
nosy: +vstinner
___
Python tracker
<https://bugs.python.org/issue39390>
___
___
Python-bugs-list mailing list
Unsubscribe:
Giampaolo Rodola' added the comment:
Yes, thanks. Whoever got bit by this is either getting an exception or not the
intended behavior (due to failed string comparison). I doubt anybody is relying
on the new type checking since it's not documented. If they are, they are
probably just doing
Giampaolo Rodola' added the comment:
> Should not copytree convert arguments of the ignore callback to str and list
> correspondingly?
It should. I think it makes sense to just do this for Python 3.8.2 instead of
updating the doc.
--
___
Giampaolo Rodola' added the comment:
Good point, I didn't consider that. I suppose you're right. =)
Closing.
--
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/i
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +17131
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/17664
___
Python tracker
<https://bugs.python.org/issu
New submission from Giampaolo Rodola' :
PR in attachment adds a new dirfd() method to the scandir() object (POSIX
only). This can be be passed to os.* functions supporting the "dir_fd"
parameter, and avoid opening a new fd as in:
>>> dirfd = os.open("basename&quo
Giampaolo Rodola' added the comment:
New changeset 82374979ec7e01e23385dca1d02b2aa3de16fea5 by Giampaolo Rodola in
branch 'master':
bpo-39004: increment large sendfile() test timeout (GH-17552)
https://github.com/python/cpython/commit/82374979ec7e01e23385dca1d02b2aa3de16fea5
Giampaolo Rodola' added the comment:
I suspect it's a timing problem (client disconnects, threaded server fails to
detect that in time). Let's see what happens by incrementing the timeout when
waiting for the thread to be join()ed (PR-17552
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +17026
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/17552
___
Python tracker
<https://bugs.python.org/issu
Giampaolo Rodola' added the comment:
If we use CopyFileExW in copy2(), then also copystat() and copymode() should be
able to copy the same metadata/security-bits/etc as CopyFileExW. I don't know
which Windows APIs should be used though.
I sort of agree with Steven that CopyFileExW could
Change by Giampaolo Rodola' :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Giampaolo Rodola' added the comment:
New changeset 65c92c5870944b972a879031abd4c20c4f0d7981 by Giampaolo Rodola
(Bruno P. Kinoshita) in branch '3.8':
[3.8] bpo-38688, shutil.copytree: consume iterator and create list of entries
to prevent infinite recursion (GH-17397)
https://github.com
Change by Giampaolo Rodola' :
--
stage: patch review -> commit review
versions: +Python 3.9
___
Python tracker
<https://bugs.python.org/issue38688>
___
___
Py
Giampaolo Rodola' added the comment:
New changeset 9bbcbc9f6dfe1368fe7330b117707f828e6a2c18 by Giampaolo Rodola
(Bruno P. Kinoshita) in branch 'master':
bpo-38688, shutil.copytree: consume iterator and create list of entries to
prevent infinite recursion (GH-17098)
https://github.com/python
Giampaolo Rodola' added the comment:
PR-17098 as it stands re-introduces some stat() syscall. I suggest to just
consume the iterator: it's a small change and it should fix the issue.
--
___
Python tracker
<https://bugs.python.org/issue38
Giampaolo Rodola' added the comment:
The speedup introduced in issue33695 is mostly because the number of os.stat()
syscall was reduced from 6 to 1 per file (both by using scandir() and because
stat() results are cached and passed around between function calls). As such,
even if we
Giampaolo Rodola' added the comment:
PR-17098 basically reverts https://bugs.python.org/issue33695. Not good. =)
I think we can simply consume the iterator immediately as in:
def copytree(src, ...):
with os.scandir(src) as itr:
entries = list(itr)
return
Giampaolo Rodola' added the comment:
To further elaborate on the creation time solution, the idea in pseudo-code is
the following:
class Popen:
def __init__(self, ...):
self._execute_child(...)
try:
self._ctime = get_create_time(self.pid)
except
Giampaolo Rodola' added the comment:
> Did someone propose a pull request to fix this issue by ignoring
> ProcessLookupError?
I misread your PR, sorry. I thought that was the effect.
--
___
Python tracker
<https://bugs.python.org/i
Giampaolo Rodola' added the comment:
-1 about the PR solution to suppress ProcessLookupError in case the process is
gone. In psutil I solved the “pid reused problem” by using process creation
time to identify a process uniquely (on start).
A decorator can be used to protect the sensibile
Giampaolo Rodola' added the comment:
It turns out there is a similar precedent which was solved in the same way:
issue18787.
--
___
Python tracker
<https://bugs.python.org/issue38
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +16360
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16804
___
Python tracker
<https://bugs.python.org/issu
New submission from Giampaolo Rodola' :
>>> import resource
>>> high = 300 * 1024 * 1024
>>> resource.setrlimit(resource.RLIMIT_MEMLOCK, (high, high))
Traceback (most recent call last):
File "", line 1, in
ValueErro
Giampaolo Rodola' added the comment:
I’m for renaming both. Since the function is about transmitting or copying (on
Linux) files src_fd and dst_fd could also be good candidates.
--
nosy: +giampaolo.rodola
___
Python tracker
<ht
Giampaolo Rodola' added the comment:
Thanks.
--
resolution: fixed ->
stage: resolved ->
status: closed -> open
___
Python tracker
<https://bugs.python.or
Giampaolo Rodola' added the comment:
> Another approach is to change the timeout of 15 min for this super slow
> buildbot. For example, set it to 20 min or 30 min.
I think this makes more sense (set timeout to 25 min).
--
___
Python t
Giampaolo Rodola' added the comment:
Sorry about that. I'm attaching a patch which skips those tests if there's not
enough free disk space. If you confirm it works I'll make a PR.
--
Added file: https://bugs.python.org/file48636/disk-space.patch
Giampaolo Rodola' added the comment:
Looks like it worked:
https://buildbot.python.org/all/#/builders/176/builds/1383
--
___
Python tracker
<https://bugs.python.org/issue38
Change by Giampaolo Rodola' :
--
versions: +Python 3.8
___
Python tracker
<https://bugs.python.org/issue38319>
___
___
Python-bugs-list mailing list
Unsubscribe:
Giampaolo Rodola' added the comment:
New changeset 94e165096fd65e8237e60de570fb609604ab94c9 by Giampaolo Rodola in
branch 'master':
bpo-38319: Fix shutil._fastcopy_sendfile(): set sendfile() max block size
(GH-16491)
https://github.com/python/cpython/commit
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +16077
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16491
___
Python tracker
<https://bugs.python.org/issu
Change by Giampaolo Rodola' :
--
components: +Tests -Library (Lib)
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
type: -> enhancement
versions: +Python 3.9 -Python 3.8
___
Python tracker
<https://
Giampaolo Rodola' added the comment:
New changeset 5bcc6d89bcb622a6786fff632fabdcaf67dbb4e2 by Giampaolo Rodola in
branch 'master':
bpo-37096: Add large-file tests for modules using sendfile(2) (GH-13676)
https://github.com/python/cpython/commit/5bcc6d89bcb622a6786fff632fabdcaf67dbb4e2
Change by Giampaolo Rodola' :
--
stage: patch review -> resolved
status: open -> closed
versions: -Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/i
Giampaolo Rodola' added the comment:
New changeset a7414571a113815033fdc9f7abb3f5a24e44c472 by Giampaolo Rodola
(Prateek Nayak) in branch 'master':
bpo-38244: [Documentation] Fix unreliable link to sever in ftplib.rst (#16319)
https://github.com/python/cpython/commit
Giampaolo Rodola' added the comment:
There’s too much code out there relying on both args (especially ignore_errors
in unit tests) that would break for no real benefit.
--
___
Python tracker
<https://bugs.python.org/issue37
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue33408>
___
___
Python-bugs-list mailing list
Unsub
Giampaolo Rodola' added the comment:
I'm gonna reject this one, sorry Jeffrey. Reasons given in
https://bugs.python.org/issue36422#msg346878 also apply here.
--
resolution: -> rejected
stage: patch review -> resolved
status: open -&g
Giampaolo Rodola' added the comment:
> in case a filesystem has been mounted on the temporary directory, this can
> lead to the entire filesystem being removed
-1
That is expected behavior and the use case looks pretty unusual. Such a new
parameter wouldn't even be supported by
Giampaolo Rodola' added the comment:
I am not sure I understand the use case. Other than "unmounting a mount point"
the other scenarios look unrealistic to me and also require a fd instead of a
path. Also, if you want to unmount a path you can do so via *onerror* instead
(unmoun
Giampaolo Rodola' added the comment:
Adding a new patch (still a PoC, will create a PR when I have something more
solid).
--
Added file: https://bugs.python.org/file48393/cow2.diff
___
Python tracker
<https://bugs.python.org/issue37
Giampaolo Rodola' added the comment:
> I'm not sure that attempt to call unlink() if FICLONE fails is a good idea
Agreed.
> I dislike the *fallback* parameter of reflink().
Me too. A specific exception is better.
> Why not exposing clonefile() as os.clonefile() but os._clonefil
Giampaolo Rodola' added the comment:
Nope, it doesn't (see man page). We can simply use FICLONE (cp does the same).
--
___
Python tracker
<https://bugs.python.org/issue37
Giampaolo Rodola' added the comment:
issue37157 is for reflink / CoW copy, this one is not.
--
___
Python tracker
<https://bugs.python.org/issue37159>
___
___
Change by Giampaolo Rodola' :
--
title: Have shutil.copyfile() use copy_file_range() -> Use copy_file_range() in
shutil.copyfile() (server-side copy)
___
Python tracker
<https://bugs.python.org/issu
Change by Giampaolo Rodola' :
--
nosy: +StyXman, desbma, facundobatista, martin.panter, ncoghlan, neologix,
pablogsal, petr.viktorin, vstinner
___
Python tracker
<https://bugs.python.org/issue37
Giampaolo Rodola' added the comment:
> Please open a new issue to discuss how it can used in shutil ;-)
Use copy_file_range() in shutil.copyfile():
https://bugs.python.org/issue37159
--
___
Python tracker
<https://bugs.python.org/issu
New submission from Giampaolo Rodola' :
This is a follow up of issue33639 (zero-copy via sendfile()) and issue26828
(os.copy_file_range()). On [Linux 4.5 / glib 2.27] shutil.copyfile() will use
os.copy_file_range() instead of os.sendfile(). According to my benchmarks
performances
Giampaolo Rodola' added the comment:
I'm attaching an initial PoC using FICLONE on Linux and clonefile(3) on OSX. It
is also possible to support Windows but it requires a ReFS partition to test
against which I currently don't have. I opted for exposing reflink() as a
separate function
Giampaolo Rodola' added the comment:
Actually "man copy_file_range" claims it can do server-side copy, meaning no
network traffic between client and server if *src* and *dst* live on the same
network fs. So I agree copy_file_range() should be preferred over sendfile()
after all.
Giampaolo Rodola' added the comment:
I think data deduplication / CoW / reflink copy is better implemented via
FICLONE. "cp --reflink" uses it, I presume because it's older than
copy_file_range(). I have a working patch adding CoW copy support for Linux and
OSX (but not Windows
Giampaolo Rodola' added the comment:
shutil.copyfile() already uses sendfile() which basically provides the same
performances. sendfile() should be preferred though because it’s supported
since Linux 2.6.33.
--
___
Python tracker
<ht
Giampaolo Rodola' added the comment:
>From RFC-959:
If the server is unable to accept input right away, a
120 "expected delay" reply should be sent immediately and a 220
reply when ready. The user will then know not to hang up if there
is a delay.
Seems
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +13564
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/13676
___
Python tracker
<https://bugs.python.org/issu
New submission from Giampaolo Rodola' :
The need for this emerged in:
https://bugs.python.org/issue36610#msg343948.
We currently use sendfile(2) syscall in high-level shutil.copyfile() on Linux
and socket.sendfile() on UNIX. In addition this PR also tests shutil.copyfile()
implementation
Change by Giampaolo Rodola' :
--
assignee: -> giampaolo.rodola
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python
Giampaolo Rodola' added the comment:
New changeset 413d955f8ec88a7183f91d7ad8b0ff7def803de3 by Giampaolo Rodola in
branch 'master':
bpo-36610: shutil.copyfile(): use sendfile() on Linux only (GH-13675)
https://github.com/python/cpython/commit/413d955f8ec88a7183f91d7ad8b0ff7def803de3
Change by Giampaolo Rodola' :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
versions: -Python 2.7
___
Python tracker
<https://bugs.python.or
Giampaolo Rodola' added the comment:
New changeset f1487b323549e2360460383b4304f6592fb38e27 by Giampaolo Rodola
(Miss Islington (bot)) in branch '3.7':
bpo-24564: shutil.copystat(): ignore EINVAL on os.setxattr() (GH-13369)
https://github.com/python/cpython/commit
Giampaolo Rodola' added the comment:
I currently have no Solaris box to test this against and am currently short on
time but I'm of the opinion that because of:
> Sendfile on Solaris can raise EINVAL if offset is equal or bigger than the
size
> of the file (Python e
Change by Giampaolo Rodola' :
--
pull_requests: +13563
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/13675
___
Python tracker
<https://bugs.python.org/issu
Giampaolo Rodola' added the comment:
New changeset a16387ab2d85f19665920bb6ff91a7e57f59dd2a by Giampaolo Rodola
(Ying Wang) in branch 'master':
bpo-24564: shutil.copystat(): ignore EINVAL on os.setxattr() (GH-13369)
https://github.com/python/cpython/commit
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue36997>
___
___
Python-bugs-list mailing list
Unsub
Giampaolo Rodola' added the comment:
New changeset 0a5b88e7f23b671d63896619b13148b0e4e2b5dd by Giampaolo Rodola
(Miss Islington (bot)) in branch '3.7':
bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only
files. (PR-13212) (#13234)
https://github.com/python/cpython
Giampaolo Rodola' added the comment:
@cheryl.sabella don't have the bandwidth right now (traveling), sorry.
--
___
Python tracker
<https://bugs.python.org/issue22
Giampaolo Rodola' added the comment:
New changeset 79efbb719383386051c72f2ee932eeca8e033e6b by Giampaolo Rodola
(Olexa Bilaniuk) in branch 'master':
bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only
files. (PR-13212)
https://github.com/python/cpython/commit
Change by Giampaolo Rodola' :
--
versions: +Python 3.7, Python 3.8 -Python 3.2, Python 3.3, Python 3.4
___
Python tracker
<https://bugs.python.org/issue24
Giampaolo Rodola' added the comment:
Patch LGTM. I'd like to point out one thing for posterity. Current shutil code
catches EPERM but not EACCES. While reviewing the patch I wondered whether
simply catching (and ignoring) both error codes instead, but it turns out they
are supposed to have
Giampaolo Rodola' added the comment:
You should try to copy those offending files individually via shutil.copy2()
instead copytree() in order to get the full exception.
--
___
Python tracker
<https://bugs.python.org/issue36
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +13133
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue36850>
___
_
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +12734
stage: needs patch -> patch review
___
Python tracker
<https://bugs.python.org/issu
Giampaolo Rodola' added the comment:
Can you paste the traceback or are you able to reproduce the bug via a script?
sendfile implementation is supposed to giveup if no data was sent on first
call, so I suppose this happen later? If for any reason it turns out sendfile()
is broken on Solaris
Giampaolo Rodola' added the comment:
No problem. My bad that I mistyped the "BPO" thing.
--
___
Python tracker
<https://bugs.python.org/issue35934>
___
___
Giampaolo Rodola' added the comment:
Those are old failures (+/- 17 hours ago). I committed:
https://github.com/python/cpython/pull/12735
...which fixed them.
--
___
Python tracker
<https://bugs.python.org/issue35
Giampaolo Rodola' added the comment:
Fixed. There's a remaining failing BB:
https://buildbot.python.org/all/#/builders/176/builds/185/steps/4/logs/stdio
...but the failure appears unrelated and it has been red for a while.
--
resolution: -> fixed
stage: patch review -> re
Giampaolo Rodola' added the comment:
Please no. The mistake should be fixed in the misconfigured FTP server, not in
ftplib, which should not make this event pass unnoticed or take arbitrary
decisions. Filezilla is different than ftplib in that it's an end-user app.
ftplib is not (e.g
Giampaolo Rodola' added the comment:
Sorry, I accidentally referenced this BPO issue in my commit.
--
___
Python tracker
<https://bugs.python.org/issue1
Giampaolo Rodola' added the comment:
New changeset 8702b67dad62a9084f6c1823dce10653743667c8 by Giampaolo Rodola in
branch 'master':
BPO-17561: set create_server backlog default to None (GH-12735)
https://github.com/python/cpython/commit/8702b67dad62a9084f6c1823dce10653743667c8
Giampaolo Rodola' added the comment:
New changeset 8702b67dad62a9084f6c1823dce10653743667c8 by Giampaolo Rodola in
branch 'master':
BPO-17561: set create_server backlog default to None (GH-12735)
https://github.com/python/cpython/commit/8702b67dad62a9084f6c1823dce10653743667c8
1 - 100 of 1713 matches
Mail list logo
