I think this might be of some interest https://github.com/hpc/mpifileutils
On 1/23/20 4:33 PM, Bernd Melchers wrote: > Hi All, > we are copying large data sets within our lustre filesystem and between > lustre and an external nfs server. In both cases the performance is > unexpected low and the reason seems that rsync is reading and writing in > 32 kB Blocks, whereas our lustre would be more happy with 4 MB > Blocks. > rsync has an --block-size=SIZE Parameter but this adjusts only the > checksum block size (and the maximum is 131072), not the i/o block size. > Is there a solution to accelerate rsync? > > Mit freundlichen Grüßen > Bernd Melchers > -- Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden Internet: [email protected] Phone: +46 90 7866134 Fax: +46 90-580 14 Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
