Change by Giampaolo Rodola' :
--
nosy: -giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue26415>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
assignee: -> giampaolo.rodola
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.p
Giampaolo Rodola' added the comment:
New changeset 19c46a4c96553b2a8390bf8a0e138f2b23e28ed6 by Giampaolo Rodola in
branch 'master':
bpo-33695 shutil.copytree() + os.scandir() cache (#7874)
https://github.com/python/cpython/commit/19c46a4c96553b2a8390bf8a0
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue34655>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
Do you mean raising an exception if "count" argument is passed and >
2,147,479,552? In that case I think asyncio's sendfile() should simply do the
math to transmit that many bytes by taking into account that os.sendfile() may
Giampaolo Rodola' added the comment:
@Serhiy: I would like to proceed with this. Do you have further comments? Do
you prefer to bring this up on python-dev for further discussion?
--
___
Python tracker
<https://bugs.python.org/is
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue34934>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
PR merged for 3.8 branch. I don't think such a simple change deserves a
backport for previous Python versions. Closing this out as resolved.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
ve
Giampaolo Rodola' added the comment:
New changeset 7b313971805ca9b53f181f7b97e5376d0b89dc06 by Giampaolo Rodola
(Romuald Brunet) in branch 'master':
bpo-32680 add default "sock" on SMTP objects (#5345)
https://github.com/python/cpython/commit/7b313971805ca9b
Change by Giampaolo Rodola' :
--
versions: -Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/is
Giampaolo Rodola' added the comment:
Can you paste a sample output showing how it looks like pre and post patch?
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/is
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue34605>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
If os.name == 'nt' is True on IronPython then why not simply do:
mswindows = sys.platform == "win32" or os.name == "nt"
For the record, both variants are used in different places in cPython source
code. It would n
Giampaolo Rodola' added the comment:
I think this feature request is reasonable for 2 reasons:
1) As it stands if dst directory exists copytree() cannot be used. The only
possible workaround is using rmtree(dst) first, but that doesn't seem to make
much sense. The change may look
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue31940>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
It's not clear where the exception originates from. Try to use copy2() instead
of copytree().
--
___
Python tracker
<https://bugs.python.o
Giampaolo Rodola' added the comment:
What function raises ENOTSUPP exactly (traceback would be welcome)?
--
___
Python tracker
<https://bugs.python.org/is
Giampaolo Rodola' added the comment:
Yes, file copy (open() + read() + write()) is of course more expensive than
just "reading" a tree (os.walk(), glob()) or deleting it (rmtree()) and the
"pure file copy" time adds up to the benchmark. And indeed it's not an
Giampaolo Rodola' added the comment:
I agree the provided benchmark on Linux should be more refined. And I'm not
sure if "echo 3 | sudo tee /proc/sys/vm/drop_caches" before running it is
enough honestly.
The main point here is the reduction of stat() syscalls (-38%)
Giampaolo Rodola' added the comment:
[ https://bugs.python.org/issue24564#msg246278 ]
> Adding `EINVAL` to the ignored errnos would fix the problem, but might hide
> real failures (I'm not sure about the real failures, but it seems logical).
I think this is an accep
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue34260>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
Closing as duplicate of #33671.
--
resolution: -> duplicate
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.pyth
Giampaolo Rodola' added the comment:
> PR 8287 seems short to me and it seems like psutils doesn't expose Processor
> Queue Length, so I'm not sure why we are talking about depending on psutils?
I'm not sure if you're strictly interested in getting system load
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue34157>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
psutil exposes this functionality as "psutil.cpu_percent()":
https://github.com/giampaolo/psutil/blob/ac9dccab6b038835b5d612f92cf4804ec2662c2e/psutil/_psutil_windows.c#L992
https://github.com/giampaolo/psutil/blob/ac9dccab6b038835b5d612f92
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue30480>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
Unless somebody has complaints I think I'm gonna merge this soon.
--
___
Python tracker
<https://bugs.python.o
Giampaolo Rodola' added the comment:
I suppose there is no way to emulate os.getloadavg() on Windows because that
would necessarily imply using a thread to call the necessary routine (WMI, PDH,
whatever...) every X secs or something, co
Giampaolo Rodola' added the comment:
> Giampaolo, on your psutil issue you specifically said, "(possibly without
> using WMI)" Is there any particular problem with using WMI?
Performance. In general WMI is (a lot) slower than the Windows API counterpart
(psutil never us
Giampaolo Rodola' added the comment:
> For example, os.stat() accepts both a filename (string) or a file descriptor
> (integer).
Interesting. I never realized that. Other functions involved in file copy are:
os.lchmod
os.chmod
os.listxattr
os.getxattr
I checked and it appea
Giampaolo Rodola' added the comment:
There is an old ticket for this in psutil with some (possible useful)
references in it:
https://github.com/giampaolo/psutil/issues/604
--
___
Python tracker
<https://bugs.python.org/is
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue34060>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue17180>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
All copy* functions and move() are subjects to race conditions (the only
exception is rmtree()). You may solve the race condition in copyfile() but then
you'd have the same problem in copystat() and copymode() which are used by
copy() and co
Change by Giampaolo Rodola' :
--
pull_requests: -7403
___
Python tracker
<https://bugs.python.org/issue33671>
___
___
Python-bugs-list mailing list
Unsubscr
Change by Giampaolo Rodola' :
--
pull_requests: -7525
___
Python tracker
<https://bugs.python.org/issue33671>
___
___
Python-bugs-list mailing list
Unsubscr
Change by Giampaolo Rodola' :
--
pull_requests: -7482
___
Python tracker
<https://bugs.python.org/issue33671>
___
___
Python-bugs-list mailing list
Unsubscr
Giampaolo Rodola' added the comment:
This was implemented in https://bugs.python.org/issue30014. Closing as
duplicate.
--
resolution: -> duplicate
stage: needs patch -> resolved
status: open -> closed
versions: +Python 2.7 -Python 3.5
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue33935>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue31861>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
> I re-ran benchmarks since shutil code changed after #33695.
Sorry, I meant #33671.
--
___
Python tracker
<https://bugs.python.org
Giampaolo Rodola' added the comment:
PR at: https://github.com/python/cpython/pull/7874.
I re-ran benchmarks since shutil code changed after #33695. Linux went from
+13.5% to 8.8% and Windows went from +17% to 20.7%.
In the PR I explicitly avoided using a context manager around os.sc
Change by Giampaolo Rodola' :
--
pull_requests: +7481
___
Python tracker
<https://bugs.python.org/issue33695>
___
___
Python-bugs-list mailing list
Unsubscr
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue32568>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
New changeset c7f02a965936f197354d7f4e6360f4cfc86817ed by Giampaolo Rodola in
branch 'master':
bpo-33671 / shutil.copyfile: use memoryview() with dynamic size on Windows
(#7681)
https://github.com/python
Change by Giampaolo Rodola' :
--
pull_requests: +7293
___
Python tracker
<https://bugs.python.org/issue33671>
___
___
Python-bugs-list mailing list
Unsubscr
Giampaolo Rodola' added the comment:
For future reference, as per https://github.com/python/cpython/pull/7160
discussion, we decided not to use CopyFileEx on Windows and instead increase
read() buffer size from 16KB to 1MB (Windows only) resulting in a 40.8% speedup
(instead of 48%).
Giampaolo Rodola' added the comment:
New changeset 4a172ccc739065bb658c75e8929774a8e94af9e9 by Giampaolo Rodola in
branch 'master':
bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win)
(#7160)
https://github.com/python
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue23346>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue27687>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue15100>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
If such a thing goes in then copytree() should also be taken into account and
expose the same parameter. That would probably be inconsistent with
copytree()'s copy_function parameter though.
--
nosy: +giampa
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue10016>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
+1. It looks reasonable to ignore FileNotFoundError on os.rmdir(), os.unlink()
and also os.open() and os.scandir().
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/is
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue30400>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
This could be achieved at least on Windows with CopyFileEx [1] and on OSX with
copyfile(3) + COPYFILE_ALL which copies ACLs (but not users/groups). These are
(were, in case of CopyFileEx) exposed in
https://github.com/python/cpython/pull/7160/. S
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue20849>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue32073>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
+1
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue32557>
___
___
Python-bugs-list m
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue32347>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue30235>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue32453>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
Gotcha. Thanks for clarifying.
--
___
Python tracker
<https://bugs.python.org/issue32561>
___
___
Python-bugs-list m
Giampaolo Rodola' added the comment:
os.preadv() and os.pwritev() are great but to my understanding one essential
piece is still missing in order to effectively do non-blocking file IO and
avoid using a thread pool: being notified when the file fd is
readable/writable. select() and
Giampaolo Rodola' added the comment:
Patch in attachment does the trick.
--
keywords: +patch
Added file: https://bugs.python.org/file47634/clinic.diff
___
Python tracker
<https://bugs.python.org/is
Change by Giampaolo Rodola' :
--
nosy: -giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue33743>
___
___
Python-bugs-list mailing list
New submission from Giampaolo Rodola' :
There currently are different functions in Modules/_winapi.c that deals with
paths: CreateFile, CreateJunction, CreateNamedPipe and I'm currently working on
a patch which will add 3 more. For those functions it would be convenient to be
a
New submission from Giampaolo Rodola' :
If I use "Tools\clinic\clinic.py Modules\somemodule.c" on Windows argument
clinic will modify the whole file and use Windows line endings ( "\r\n"). It
would be good to use "\n" instead.
--
assignee: larry
c
Change by Giampaolo Rodola' :
--
nosy: +benhoyt, benjamin.peterson, brett.cannon, ncoghlan, serhiy.storchaka,
stutzbach, tarek, vstinner, yselivanov
___
Python tracker
<https://bugs.python.org/is
Change by Giampaolo Rodola' :
--
keywords: +patch
Added file: https://bugs.python.org/file47625/bpo-33695.patch
___
Python tracker
<https://bugs.python.org/is
New submission from Giampaolo Rodola' :
Patch in attachment makes shutil.copytree() use os.scandir() and (differently
from #33414) DirEntry instances are passed around so that cached stat()s are
used also from within copy2() and copystat() functions. The number of times the
filesystem
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue33414>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
Yes, I tried changing benchmarks order and zero-copy variants are always
faster. As for instantaneous CoW copy, it is debatable. E.g. "cp" command does
not do it by default:
https://unix.stackexchange.com/questions/80351/why-is-cp-refli
Change by Giampaolo Rodola' :
--
nosy: +SilentGhost, StyXman, asvetlov, desbma, facundobatista, gps, josh.r,
martin.panter, ncoghlan, neologix, petr.viktorin, pitrou, python-dev,
r.david.murray, tarek, vstinner, yselivanov
___
Python tr
Giampaolo Rodola' added the comment:
Closing as duplicate of #33671.
--
stage: resolved -> patch review
___
Python tracker
<https://bugs.python.org
Giampaolo Rodola' added the comment:
Closing as duplicate of #33671.
--
resolution: -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder: -> Efficient zero-copy for shutil.copy* functions (Linux, OSX and
Win)
_
Giampaolo Rodola' added the comment:
PR: https://github.com/python/cpython/pull/7160
--
___
Python tracker
<https://bugs.python.org/issue33671>
___
___
Pytho
Change by Giampaolo Rodola' :
--
pull_requests: +6795
___
Python tracker
<https://bugs.python.org/issue33671>
___
___
Python-bugs-list mailing list
Unsubscr
Change by Giampaolo Rodola' :
--
title: Efficient efficient zero-copy syscalls for shutil.copy* functions
(Linux, OSX and Win) -> Efficient zero-copy for shutil.copy* functions (Linux,
OSX and Win)
___
Python tracker
<https://bugs
New submission from Giampaolo Rodola' :
Patch in attachment uses platform specific zero-copy syscalls on Linux and
Solaris (os.sendfile(2)), Windows (CopyFileW) and OSX (fcopyfile(2)) speeding
up shutil.copyfile() and other functions using it (copy(), copy2(), copytree(),
move()).
Av
Change by Giampaolo Rodola' :
--
nosy: -giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue32458>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
My bad, should have read the doc, sorry. =)
--
resolution: -> not a bug
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.pyth
Giampaolo Rodola' added the comment:
I updated the code to rely on sendfile(2) on Linux only, which apparently is
the only one supporting copy between regular files and added a check to fail
immediately in case the filesystem is full.
Can somebody review the
New submission from Giampaolo Rodola' :
>>> import errno
>>> OSError(errno.EBADF).errno
>>> OSError(errno.EBADF, "yo").errno
9
>>> IOError(errno.EBADF).errno
>>> IOError(errno.EBADF, "yo").errno
9
>>> EnvironmentErro
Giampaolo Rodola' added the comment:
Check for availability in configure.ac appears to be broken.
--
___
Python tracker
<https://bugs.python.org/is
Giampaolo Rodola' added the comment:
Oh! I got confused by the fact that #25063 was rejected due to concerns about
copyfileobj() otherwise I would have commented on your patch which I totally
missed. Yes, this overlaps with #25156 patch but it uses the same logic of
socket.sendfile()
Change by Giampaolo Rodola' :
--
keywords: +patch
pull_requests: +6737
___
Python tracker
<https://bugs.python.org/issue33639>
___
___
Python-bugs-list m
New submission from Giampaolo Rodola' :
This is a follow up of #25063 and similar to socket.sendfile() (#17552). It
provides a 20/25% speedup when copying files with shutil.copyfile(),
shutil.copy() and shutil.copy2(). Differently from #25063 this is used for
filesystem files onl
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue32622>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue32710>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
FYI, there's psutil.Process().num_handles() which you can use to count handles
before and after subprocess invocation.
--
___
Python tracker
<https://bugs.python.o
Giampaolo Rodola' added the comment:
This is a great addition. I have a working patch adding sendfile() support for
shutil.copyfileobj() which speeds it up by a factor of 1.3x on Linux. According
to this
https://lists.kernelnewbies.org/pipermail/kernelnewbies/2016-March/015999
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue25063>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue25156>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue26826>
___
___
Python-bugs-list mailing list
Change by Giampaolo Rodola' :
--
nosy: +giampaolo.rodola
___
Python tracker
<https://bugs.python.org/issue15443>
___
___
Python-bugs-list mailing list
Giampaolo Rodola' added the comment:
> For convenience, a handler that retries unlink() and rmdir() could be
> distributed with shutil. For ease of use, it could be enabled by default on
> Windows.
+1 on that. I bumped into this many times over the years as occasion
Giampaolo Rodola' added the comment:
That makes sense to me. Thanks for deciphering this.
--
___
Python tracker
<https://bugs.python.org/issue33166>
___
___
Giampaolo Rodola' added the comment:
I created a psutil branch using GetLogicalProcessorInformation() to determine
both logical and physical CPUs:
https://github.com/giampaolo/psutil/pull/1257
According to https://stackoverflow.com/questions/31209256 basically all Windows
APIs are unrel
Giampaolo Rodola' added the comment:
By re-reading
https://bugs.python.org/issue30581 and
https://github.com/giampaolo/psutil/issues/771#issuecomment-264457333 I now
remember why I haven't fixed this issue in psutil yet: because the whole thing
(MS APIs and doc basically) is
201 - 300 of 1749 matches
Mail list logo