Steve Bertrand wrote: > xufengnju wrote: >> Hi all, >> I have a storage server that holds one million of images with well >> structured directory structure such as >> >> /data/user1/2008/09/12/image1.jpg >> /data/user1/2008/09/12/image2.jpg >> ... >> /data/user2/2009/01/01/image1.jpg >> ... >> >> I want to copy them to the /data2 directory in the same server on another >> disk partion. >> I want to keep the directory structure and `chown` && `chmod` the >> directories and files,much like doing a `cp -rf /data /data2 && chown -R >> sysuser:sysuser /data2 && chmod -R 755 /data2`. >> >> File::Find maybe an option. >> Is there somebody who have some suggestions? >> >> If I do a `cp -rf /data /data2 && chown -R sysuser:sysuser /data2 && chmod >> -R 755 /data2`,how much time maybe taken to finish the job?(The images are >> about one million in count and 250GB in size totally). > > Perhaps I am missing something completely obvious. If not: > > Why-oh-why do you want to use Perl do perform such a task? > > Use dump/restore (which I can't recall a cli sequence for off the top of > my head), or rsync: > > # mount the new disk into /mnt/data, then: > > % rsync -arcvv /data /mnt/data & > > ...owns/perms will be copied. The '&' will ensure that the process will > continue if your term session breaks mid-stream.
fwiw:
Running a test with a single host, I can read -> write doing a 'backup'
with dump(8) ( using [AMANDA][1] ) at ~90 MBps (720 Mbps) over the
network (from relative commodity hardware to a FreeBSD system with ZFS):
%zpool iostat 1
capacity operations bandwidth
pool used avail read write read write
---------- ----- ----- ----- ----- ----- -----
storage 10.60T 226G 0 1.02K 126K 106M
storage 10.60T 226G 0 1000 0 94.3M
storage 10.60T 226G 1 947 127K 73.8M
storage 10.60T 226G 9 976 140K 105M
storage 10.60T 226G 19 933 35.4K 89.2M
storage 10.60T 226G 29 1.02K 174K 84.9M
At that rate, for 250GB, it would take approximately 12.85 minutes to
copy the data.
If both disks are within the same pc, you may be able to achieve higher
speeds. It depends highly on the hardware you are using.
rsync(1) performs just as well, but it's more suited for 'cloning' and
keeping current, as opposed to a simple copy.
Steve
[1]: http://www.amanda.org/
smime.p7s
Description: S/MIME Cryptographic Signature
