question about parallelism in cp command

2019-06-06 Thread Olga Kornievskaia
Hi folks, Is there something philosophically incorrect in making a “cp” multi-threaded and allow for parallel copies when “cp -r” is done? If it’s something that’s possible, are there any plans in making a multi-threaded cp? I’m not a member of the list so I kindly request you cc me on the

RE: question about parallelism in cp command

2019-06-06 Thread Marc Roos
Hmmm without being a maintainer. I would say cp -r is most used on single disk, so one thread is using the maximum disk iops taking y time to copy. What would solve using multiple threads each taking their share of the maximum disk iops, and because of the scheduling and other overhead

Re: question about parallelism in cp command

2019-06-06 Thread Olga Kornievskaia
The use case I'm consider are network file systems. So perhaps a default can be a single threaded system for the local filesystems but add an option to cp for the -r case that would enable network file system to copy files in parallel. On Thu, Jun 6, 2019 at 12:25 PM Marc Roos wrote: > > > Hmmm

Re: [PATCH V6 adjustments]

2019-06-06 Thread Jeff Layton
On Wed, 2019-06-05 at 23:46 +0100, Pádraig Brady wrote: > On 05/06/19 16:52, Jeff Layton wrote: > > On Mon, 2019-06-03 at 03:55 +0100, Pádraig Brady wrote: > > > The main change here is to refactor code duplication. > > > In particular this reuses the large print_stat() function > > > rather than

Re: question about parallelism in cp command

2019-06-06 Thread Assaf Gordon
> -Original Message- > From: Olga Kornievskaia [mailto:a...@umich.edu] > > Is there something philosophically incorrect in making a “cp” > multi-threaded and allow for parallel copies when “cp -r” is done? If > it’s something that’s possible, are there any plans in making a >

Re: question about parallelism in cp command

2019-06-06 Thread Olga Kornievskaia
On Thu, Jun 6, 2019 at 2:44 PM Assaf Gordon wrote: > > > -Original Message- > > From: Olga Kornievskaia [mailto:a...@umich.edu] > > > > Is there something philosophically incorrect in making a “cp” > > multi-threaded and allow for parallel copies when “cp -r” is done? If > > it’s

Re: patches for multi-threaded cp and md5sum (along with other features)

2019-06-06 Thread Paul Kolano (ARC-TN)[InuTeq, LLC]
On 6/6/19, 11:55 AM, "Assaf Gordon" wrote: Because the changes are massive, before we can start looking into their details and merits we'll need copyright assignment from the copyright holder of the code (you or NASA). ok, I will have to ask about this as I'm not sure I have the

extending cp to use copy_file_range system call

2019-06-06 Thread Olga Kornievskaia
Hi folks, In the spirit of my inquiries about cp functionality, I'd like to ask the community if there are any plans to make use of the copy_file_range() system call that's been available in the linux kernel for a number of years now. While it promises great improvement to the network file

Re: patches for multi-threaded cp and md5sum (along with other features)

2019-06-06 Thread Assaf Gordon
Hello Paul, On Mon, Jun 03, 2019 at 09:29:20PM +, Paul Kolano (ARC-TN)[InuTeq, LLC] wrote: > Many years ago, I developed a set of patches to add a number of > features to cp and md5sum [...] > https://pkolano.github.io/projects/mutil.html Thanks for sharing, looks very impressive.