Am 10.05.2019 um 10:51 schrieb Lothar Schilling: > Am 10.05.2019 um 07:48 schrieb David Christensen: >> On 5/9/19 1:49 AM, Lothar Schilling wrote: >>> Hi everybody, >>> >>> for years I have used CentOS for our server landscape. Now I decided to >>> give Debian a try. I just set up a Stretch 9.8 system supposed to become >>> our main backup server. So I set up a backup job wih rsync. But the >>> going is really very very slooooow. Trying to figure out what's >>> happening: >>> >>> * iperf -c [host] => bandwith almost 1000 Mbit, that's fine. >>> * dd if=/dev/zero of=/daten/testfile bs=1G count=10 oflag=direct => >>> 10737418240 Bytes (11 GB, 10 GiB) kopiert, 40,4992 s, 265 MB/s, so >>> that's fine as well >>> >>> But whenever I try to rsync, cp or scp - whether copy files on the local >>> hard disk only or over the network - speed goes down to 500 kB/s. >>> >>> Filesystem is ext4. >>> >>> I don't have any clue about what's going on. Any kind of help would be >>> appreciated, thank you! >>> >>> Lothar >> >> On 5/9/19 2:46 AM, Lothar Schilling wrote:> Am 09.05.2019 um 11:14 >> schrieb Jonas Smedegaard: >>>> Is _only_ transfer speed affected? >>>> >>>> I am no expert in this, but imagine that if you rsync massive amounts >>>> involving hardlinks then memory becomes a problem too. >>>> >>>> Perhaps run atop to monitor bottlenecks live >>>> >>>> >>>> - Jonas >>>> >>> It is most definitely not a memory or cpu problem. It's a HP Proliant >>> with 32 GB RAM and a Intel Xeon CPU 2.40GHz with 4 cores. Also the >>> problem stays the same if I just copy one large file. >> >> On 5/9/19 3:26 AM, Lothar Schilling wrote:> Am 09.05.2019 um 11:51 >> schrieb Kevin DAGNEAUX: >>>> Check if it's related to the disk speed (hdparm and/or iotop). >>>> >>>> Kevin >>>> >>> hdparm -tT /dev/sda >>> /dev/sda: >>> Timing cached reads: 13348 MB in 2.00 seconds = 6683.42 MB/sec >>> Timing buffered disk reads: 1014 MB in 3.00 seconds = 337.72 MB/sec >>> >>> iotop -o (for rsync and cp) >>> Total DISK READ : 0.00 B/s | Total DISK WRITE : 476.15 K/s >>> Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 487.86 K/s >>> TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND >>> 19531 be/4 root 0.00 B/s 476.15 K/s 0.00 % 99.24 % rsync >>> --info=progress2 /daten/testfile /daten/testfile2 >>> >>> iotop -o (for dd) >>> Total DISK READ : 0.00 B/s | Total DISK WRITE : 297.68 M/s >>> Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 297.68 M/s >>> TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND >>> 19557 be/4 root 0.00 B/s 297.68 M/s 0.00 % 99.99 % dd >>> if=/dev/zero of=/daten/testfile bs=1G count=10 oflag=direct >> >> On 5/9/19 5:43 AM, Lothar Schilling wrote:> Am 09.05.2019 um 13:27 >> schrieb Martin: >>>> [..] >>>>> hdparm -tT /dev/sda >>>>> /dev/sda: >>>>> Timing cached reads: 13348 MB in 2.00 seconds = 6683.42 MB/sec >>>>> Timing buffered disk reads: 1014 MB in 3.00 seconds = 337.72 >> MB/sec >>>>> iotop -o (for rsync and cp) >>>>> Total DISK READ : 0.00 B/s | Total DISK WRITE : 476.15 K/s >>>>> Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 487.86 K/s >>>>> TID PRIO USER DISK READ DISK WRITE SWAPIN IO> >> COMMAND >>>>> 19531 be/4 root 0.00 B/s 476.15 K/s 0.00 % 99.24 % rsync >>>>> --info=progress2 /daten/testfile /daten/testfile2 >>>>> >>>>> iotop -o (for dd) >>>>> Total DISK READ : 0.00 B/s | Total DISK WRITE : 297.68 M/s >>>>> Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 297.68 M/s >>>>> TID PRIO USER DISK READ DISK WRITE SWAPIN IO> >> COMMAND >>>>> 19557 be/4 root 0.00 B/s 297.68 M/s 0.00 % 99.99 % dd >>>>> if=/dev/zero of=/daten/testfile bs=1G count=10 oflag=direct >>>> Show us the 'dd if=/daten/testfile bs=1G oflag=direct >> of=/dev/null', please. >>>> If this is as slow as this ~480k/s above, check your disk's health >> status. Like with smartmontools or some disk-utility software. >>>> Martin >>>> >>> Fast enough... >>> >>> dd if=/daten/testfile bs=1G oflag=direct of=/daten/testfile2 >>> 10+0 Datensätze ein >>> 10+0 Datensätze aus >>> 10737418240 Bytes (11 GB, 10 GiB) kopiert, 72,7297 s, 148 MB/s >>> >>> dd if=/daten/testfile of=/dev/null >>> 20971520+0 Datensätze ein >>> 20971520+0 Datensätze aus >>> 10737418240 Bytes (11 GB, 10 GiB) kopiert, 36,6887 s, 293 MB/s >> Please reply with your *complete* console session (you can remove all >> of the above in your reply). >> >> David >>
# cat /etc/debian_version 9.9 # uname -a Linux [my.server.com] 4.9.0-9-686-pae #1 SMP Debian 4.9.168-1 (2019-04-12) i686 GNU/Linux # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 3,7T 0 disk ├─sda1 8:1 0 476M 0 part /boot ├─sda2 8:2 0 93,1G 0 part / ├─sda3 8:3 0 18,6G 0 part [SWAP] └─sda4 8:4 0 931,3G 0 part /daten # mount | egrep 'sd[a-z]' /dev/sda2 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered) /dev/sda4 on /daten type ext4 (rw,relatime,data=ordered) /dev/sda1 on /boot type ext4 (rw,relatime,data=ordered) # df | egrep 'sd[a-z]' /dev/sda2 95596964 2241952 88455840 3% / /dev/sda4 960185376 41249236 870091648 5% /daten /dev/sda1 463826 57063 378296 14% /boot # time dd if=/dev/urandom of=test bs=1M count=100 conv=fsync 100+0 Datensätze ein 100+0 Datensätze aus 104857600 Bytes (105 MB, 100 MiB) kopiert, 192,778 s, 544 kB/s real 3m12,781s user 0m0,000s sys 0m1,480s # echo 3 > /proc/sys/vm/drop_caches && time dd if=test of=/dev/null 204800+0 Datensätze ein 204800+0 Datensätze aus 104857600 Bytes (105 MB, 100 MiB) kopiert, 0,600354 s, 175 MB/s real 0m0,675s user 0m0,048s sys 0m0,340s # echo 3 > /proc/sys/vm/drop_caches && f=test && uh=root@[my.other.server.com] && p=/tmp/test && ssh $uh rm -f $p/$g && rsync -a --stats $f $uh:$p Number of files: 1 (reg: 1) Number of created files: 1 (reg: 1) Number of regular files transferred: 1 Total file size: 104,857,600 bytes Total transferred file size: 104,857,600 bytes Literal data: 104,857,600 bytes Matched data: 0 bytes File list size: 0 File list generation time: 0.021 seconds File list transfer time: 0.000 seconds Total bytes sent: 104,883,277 Total bytes received: 34 sent 104,883,277 bytes received 34 bytes 13,984,441.47 bytes/sec total size is 104,857,600 speedup is 1.00 # echo 3 > /proc/sys/vm/drop_caches && f=test && p=/tmp && rsync -a --stats $f $p Number of files: 1 (reg: 1) Number of created files: 1 (reg: 1) Number of deleted files: 0 Number of regular files transferred: 1 Total file size: 104,857,600 bytes Total transferred file size: 104,857,600 bytes Literal data: 104,857,600 bytes Matched data: 0 bytes File list size: 0 File list generation time: 0.021 seconds File list transfer time: 0.000 seconds Total bytes sent: 104,883,283 Total bytes received: 35 sent 104,883,283 bytes received 35 bytes 629,929.84 bytes/sec total size is 104,857,600 speedup is 1.00