[issue29659] Expose the `length` arg from shutil.copyfileobj for public use

[Inada Naoki]

Inada Naoki  added the comment:

bpo-33671 fixed this.

--
resolution:  -> fixed
stage: test needed -> resolved
status: open -> closed
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29659] Expose the `length` arg from shutil.copyfileobj for public use

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Could you please provide any benchmarks Tyler?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29659] Expose the `length` arg from shutil.copyfileobj for public use

[INADA Naoki]

INADA Naoki added the comment:

How about increasing default value to 32KiB or 64KiB too?

binutils's cp works well for most cases, while it doesn't have option to 
specify blocksize.
I want Python's copy functions works nice enough for common cases too.

binutils cp uses 128KiB block for normal file. see more information in 
https://github.com/coreutils/coreutils/blob/master/src/ioblksize.h

But copyfileobj can be used other file-likes.
On Linux, typical socket buffer size is 128KiB, and typical pipe buffer size is 
64KiB.  So block size larger than 64KiB may cause unneeded blocking.
That's why I suggested 32KiB or 64KiB.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29659] Expose the `length` arg from shutil.copyfileobj for public use

[Raymond Hettinger]

Raymond Hettinger added the comment:

This seems reasonable to me.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29659] Expose the `length` arg from shutil.copyfileobj for public use

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
nosy: +terry.reedy
stage:  -> test needed
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29659] Expose the `length` arg from shutil.copyfileobj for public use

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
components: +Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29659] Expose the `length` arg from shutil.copyfileobj for public use

[Tyler Goodlet]

Changes by Tyler Goodlet :


--
pull_requests: +288

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29659] Expose the `length` arg from shutil.copyfileobj for public use

[tyler]

New submission from tyler:

It would be handy to be able to pass the `length` default argument defined by 
`shutil.copyfileobj()` to other public functions througout the module. When 
copying very large files (1GB +), increasing the memory buffer can divide the 
copy time in half and currently the only way to accomplish this is by 
overriding the needed function(s) which call `shutil.copyfileobj()`.

I propose a simple non-invasive change where the following functions will also 
expose the `length` kwarg and pass it downwards to `copyfileobj`:
- copyfile
- copy
- copy2

--
messages: 288616
nosy: goodboy
priority: normal
severity: normal
status: open
title: Expose the `length` arg from shutil.copyfileobj for public use
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com