On 2021-06-07, Michael Lowery Wilson <[email protected]> wrote: > Greetings, > > My attempts at creating a local mirror of Project Gutenberg's ebooks under > OpenBSD 6.9 using openrsync following official instructions: > https://www.gutenberg.org/help/mirroring.html have been unsuccessful. > > Specifically I am using: > > openrsync -av --del aleph.gutenberg.org::gutenberg-epub /disk5/gutenberg/ > > to sync 927606 files (approximately 440 GB), which then fails with the errors: > > openrsync: error: 39488: mkdirat: Too many links > openrsync: error: rsync_uploader > openrsync: error: rsync_receiver
I think this is because there are too many subdirectories in a single directory. Each subdirectory's ".." link is a hardlink with the parent directory and you run into LINK_MAX (32767). $ rsync aleph.gutenberg.org::gutenberg-epub|wc -l 65559 By the way, openrsync is a poor choice when dealing with many files/directories. It does not support the incremental file listing method that rsync added in 3.0.0 so will use more memory at your side and perhaps more importantly on the server you're fetching from as it has to build and transfer the entire file list in one go.
