Hi,
Thanks as well!
The source is spinning rust with SSD cache. The target is RAID-Z2 ZFS on
spinning rust.
And yes: way too much tiny files.
To be continued...
K.
On 21-09-2021 17:53, C. Chan wrote:
Is the file storage media hard disk, and are the files mostly
small and IOPS/latency bound? If so, you may have to live with sloooow
backups. We eventually added NVMe SSDs to our ZFS array as special
devices
and sent files smaller than 128KB to NVMe, which along with the metadata,
helped speed up backups quite a bit.
It may be worth trying star instead of tar, but I think it too is single
threaded. There's something called tarsplitter on github which is
multi-threaded if you don't mind experimenting:
https://github.com/AQUAOSOTech/tarsplitter
