Re: Rsync is too slow
‐‐‐ Original Message ‐‐‐ On Thursday 30 July 2020 22:37, Chris Cappuccio wrote: > Rupert Gallagher [r...@protonmail.com] wrote: > > > No, I am not using USB. > > your dmesg didn't make it to the list because you are attaching a text file > and attachments are not allowed on misc. > > please put it inline with the message. OK OpenBSD 6.7 (GENERIC.MP) #2: Thu Jun 4 09:55:08 MDT 2020 r...@syspatch-67-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 17125511168 (16332MB) avail mem = 16593870848 (15825MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x7f0c3000 (34 entries) bios0: vendor American Megatrends Inc. version "1.2" date 11/05/2019 bios0: Supermicro A2SDi-4C-HLN4F acpi0 at bios0: ACPI 6.1 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP FPDT FIDT SPMI MCFG WDAT APIC BDAT HPET UEFI SSDT SSDT SSDT DMAR SPCR HEST BERT ERST EINJ WSMT acpi0: wakeup devices XHC1(S4) OBL1(S4) LAN1(S4) PEX0(S4) LAN2(S4) LAN3(S4) PEX1(S4) PEX6(S4) PEX7(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimcfg0 at acpi0 acpimcfg0: addr 0xe000, bus 0-255 acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 4 (boot processor) cpu0: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz, 1104.90 MHz, 06-5f-01 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu0: 2MB 64b/line 16-way L2 cache cpu0: cannot disable silicon debug cpu0: smt 0, core 2, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 25MHz cpu0: mwait min=64, max=64, C-substates=0.2.0.2, IBE cpu1 at mainbus0: apid 12 (application processor) cpu1: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz, 1100.10 MHz, 06-5f-01 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu1: 2MB 64b/line 16-way L2 cache cpu1: cannot disable silicon debug cpu1: smt 0, core 6, package 0 cpu2 at mainbus0: apid 16 (application processor) cpu2: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz, 1100.10 MHz, 06-5f-01 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu2: 2MB 64b/line 16-way L2 cache cpu2: cannot disable silicon debug cpu2: smt 0, core 8, package 0 cpu3 at mainbus0: apid 24 (application processor) cpu3: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz, 1100.10 MHz, 06-5f-01 cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu3: 2MB 64b/line 16-way L2 cache cpu3: cannot disable silicon debug cpu3: smt 0, core 12, package 0 ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins acpihpet0 at acpi0: 2399 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 6 (VRP0) acpiprt2 at acpi0: bus 2 (PEX0) acpiprt3 at acpi0: bus 1 (VRP2) acpiprt4 at acpi0: bus 7 (VRP1) acpiprt5 at acpi0: bus -1 (PEX1) acpiprt6 at acpi0: bus 3 (PEX6) acpiprt7 at acpi0: bus 4 (PEX7) acpiprt8 at acpi0: bus 5 (BR28) acpicpu0 at acpi0: C2(10@50 mwait.1@0x21), C1(1000@1 mwait.1@0x1), PSS acpicpu1 at acpi0: C2(10@50 mwait.1@0x21), C1(1000@1 mwait.1@0x1), PSS acpicpu2 at acpi0: C2(10@50 mwait.1@0x21), C1(1000@1 mwait.1@0x1), PSS acpicpu3 at acpi0: C2(10@50 mwait.1@0x21), C1(1000@1 mwait.1@0x1), PSS acpipci0 at acpi0 PCI0: 0x0010 0x0011 0x "PNP0003" at acpi0 not configured acpicmos0 at acpi0 "IPI0001" at acpi0 not configured "PNP0C33" at acpi0 not configured ipmi at mainbus0 not configured cpu0: Enhanced SpeedStep 1104 MHz: speeds: 2200, 2100, 2000, 1900, 1800, 1700, 1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz pci0 at
Re: Rsync is too slow
‐‐‐ Original Message ‐‐‐ On Thursday 30 July 2020 22:36, Chris Cappuccio wrote: > Rupert Gallagher [r...@protonmail.com] wrote: > > > No, I am not using USB. > > rsync between disks should be very fast. Right. > you are going from the sata to the nvme ? No. It is SATA to SATA, using a M14TQC with a Mini-SAS HD to 4 SATA cable: https://www.supermicro.com/en/products/accessories/mobilerack/CSE-M14TQC.php > it might be interesting to try using cp between filesystems, or tar > > such as: cp -r /usr/bin /mnt/usr/bin > or: tar cf - -C /usr/bin . | tar xpf - -C /mnt/usr/bin > > also what speeds are you getting on the destination filesystem? > > dd count=1 bs=1G if=/dev/zero of=/mnt/test conv=fsync > > might give you some rough idea of what 1G write costs. > > here's 1G write on my Samsung 845DC Pro which is one of my all-time favorite > SATA SSDs for reliability > > dd count=1 bs=1G if=/dev/zero of=test conv=fsync > > = > > 1+0 records in > 1+0 records out > 1073741824 bytes transferred in 2.906 secs (369450372 bytes/sec) > > here's the same for a Crucial M500 > > dd count=1 bs=1G if=/dev/zero of=test conv=fsync > > = > > 1+0 records in > 1+0 records out > 1073741824 bytes transferred in 4.356 secs (246484472 bytes/sec) > > it's not clear to me how much the buffer cache affects this but i'm hoping > here that conv=fsync helps. in a wierd twist, tests like this with conv=fsync > run consistently faster than without, so my understanding isn't that great. Yours are NVMe. I have an SSD on a SATA bus. This is my result: >doas dd count=1 bs=1G if=/dev/zero of=/archive2/test conv=fsync 1+0 records in 1+0 records out 1073741824 bytes transferred in 8.118 secs (132261121 bytes/sec) >grep archive2 /etc/fstab [label].a /archive2 ffs rw,nodev,nosuid,softdep,noatime 1 2 However, 1G is not enough to go past the cache in ram. This is what I do: write test doas /bin/dd if=/dev/zero of=$testfile bs=$(( $bs * 1024 )) count=$count conv=sync read test doas /bin/dd if=$testfile of=/dev/null bs=$(( $bs * 1024 )) conv=sync where $testfile = /archive2/test for example $bs=$(( stat -f "%k" /dev/sd1a )) where sd1a is the device of /archive2 count=$(( $ram / ( $bs * 1024 ) )); # ram expressed in blocks count=$(( $count + 1 )); # exceed ram by 1 block This is the speed test on WDS400T1R0A (WD RED SSD 4TB) Free disk space: 3151013175296 bytes RAM: 17125511168 bytes fs block size : 8192 bytes Size of test file : 17129537536 bytes = 2042 block(s) of 8192K Test file : /archive2/disk-speed-test.raw Writing speed : 182 MB/s Reading speed : 109 MB/s The product brief of WD RED SSD says "560MB/s read" and "530MB/s write". By comparison, this is the speed test on the ST2000NX0403 (Seagate Exos 2TB) Writing speed : 117 MB/s Reading speed : 99 MB/s The product brief of the Exos says "136MB/s" max transfer. Both the exos and the wd red have hardware bytes/sector of 512. This is how I prepared both, with details for the wd red ssd only: > fdisk -iy -g sd1 >echo "/ 1G-* 100%" >/tmp/my_disk_label >disklabel -w -A -T /tmp/my_disk_label sd1 > disklabel -hn sd1 # /dev/rsd1c: type: SCSI disk: SCSI disk label: WDC WDS400T1R0A duid: b8d30be7c118b250 flags: bytes/sector: 512 sectors/track: 255 tracks/cylinder: 511 sectors/cylinder: 130305 cylinders: 59967 total sectors: 7814037168 # total bytes: 3.6T boundstart: 64 boundend: 7814037105 drivedata: 0 16 partitions: #size offset fstype [fsize bsize cpg] a: 3.6T 64 4.2BSD 8192 65536 1 c: 3.6T0 unused > newfs -O2 sd1a /dev/rsd1a: 3815447.8MB in 7814036928 sectors of 512 bytes 1168 cylinder groups of 3266.88MB, 52270 blocks, 104704 inodes each super-block backups (for fsck -b #) at: [omissis] > dumpfs /dev/sd1a | head -19 magic 19540119 (FFS2) timeWed Jul 29 18:41:40 2020 superblock location 65536 id [ 5f21a6c4 bb9dec49 ] ncg 1168size488377308 blocks 484536905 bsize 65536 shift 16 mask0x fsize 8192shift 13 mask0xe000 frag8 shift 3 fsbtodb 4 minfree 5% optim timesymlinklen 120 maxbsize 0 maxbpg 8192maxcontig 1 contigsumsize 0 nbfree 60567111ndir1 nifree 122294269 nffree 16 bpg 52270 fpg 418160 ipg 104704 nindir 8192inopb 256 maxfilesize 36033195603132415 sbsize 8192cgsize 65536 csaddr 3304cssize 24576 sblkno 16 cblkno 24 iblkno 32 dblkno 3304 cgrotor 0 fmod0 ronly 0 clean 1 avgfpdir 64 avgfilesize 16384 flags none fsmnt volname swuid 0 Finally, this is how I use rsync: #!/bin/sh from="$1"; to="$2"; if [[ "$from" == "" || "$to" == "" ]]; then echo "usage: copy /from /to":
Re: Rsync is too slow
Attached. OpenBSD 6.7 (GENERIC.MP) #2: Thu Jun 4 09:55:08 MDT 2020 r...@syspatch-67-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 17125511168 (16332MB) avail mem = 16593870848 (15825MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x7f0c3000 (34 entries) bios0: vendor American Megatrends Inc. version "1.2" date 11/05/2019 bios0: Supermicro A2SDi-4C-HLN4F acpi0 at bios0: ACPI 6.1 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP FPDT FIDT SPMI MCFG WDAT APIC BDAT HPET UEFI SSDT SSDT SSDT DMAR SPCR HEST BERT ERST EINJ WSMT acpi0: wakeup devices XHC1(S4) OBL1(S4) LAN1(S4) PEX0(S4) LAN2(S4) LAN3(S4) PEX1(S4) PEX6(S4) PEX7(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimcfg0 at acpi0 acpimcfg0: addr 0xe000, bus 0-255 acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 4 (boot processor) cpu0: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz, 1104.90 MHz, 06-5f-01 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu0: 2MB 64b/line 16-way L2 cache cpu0: cannot disable silicon debug cpu0: smt 0, core 2, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 25MHz cpu0: mwait min=64, max=64, C-substates=0.2.0.2, IBE cpu1 at mainbus0: apid 12 (application processor) cpu1: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz, 1100.10 MHz, 06-5f-01 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu1: 2MB 64b/line 16-way L2 cache cpu1: cannot disable silicon debug cpu1: smt 0, core 6, package 0 cpu2 at mainbus0: apid 16 (application processor) cpu2: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz, 1100.10 MHz, 06-5f-01 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu2: 2MB 64b/line 16-way L2 cache cpu2: cannot disable silicon debug cpu2: smt 0, core 8, package 0 cpu3 at mainbus0: apid 24 (application processor) cpu3: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz, 1100.10 MHz, 06-5f-01 cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu3: 2MB 64b/line 16-way L2 cache cpu3: cannot disable silicon debug cpu3: smt 0, core 12, package 0 ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins acpihpet0 at acpi0: 2399 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 6 (VRP0) acpiprt2 at acpi0: bus 2 (PEX0) acpiprt3 at acpi0: bus 1 (VRP2) acpiprt4 at acpi0: bus 7 (VRP1) acpiprt5 at acpi0: bus -1 (PEX1) acpiprt6 at acpi0: bus 3 (PEX6) acpiprt7 at acpi0: bus 4 (PEX7) acpiprt8 at acpi0: bus 5 (BR28) acpicpu0 at acpi0: C2(10@50 mwait.1@0x21), C1(1000@1 mwait.1@0x1), PSS acpicpu1 at acpi0: C2(10@50 mwait.1@0x21), C1(1000@1 mwait.1@0x1), PSS acpicpu2 at acpi0: C2(10@50 mwait.1@0x21), C1(1000@1 mwait.1@0x1), PSS acpicpu3 at acpi0: C2(10@50 mwait.1@0x21), C1(1000@1 mwait.1@0x1), PSS acpipci0 at acpi0 PCI0: 0x0010 0x0011 0x "PNP0003" at acpi0 not configured acpicmos0 at acpi0 "IPI0001" at acpi0 not configured "PNP0C33" at acpi0 not configured ipmi at mainbus0 not configured cpu0: Enhanced SpeedStep 1104 MHz: speeds: 2200, 2100, 2000, 1900, 1800, 1700, 1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz pci0 at mainbus0 bus 0 0:31:5: mem address conflict 0xfe01/0x1000 pchb0 at pci0 dev 0 function 0 "Intel C3000 Host" rev 0x11 pchb1 at pci0 dev 4 function 0 "Intel C3000 GLREG" rev 0x11 "Intel C3000 RCEC" rev 0x11 at pci0 dev 5 function 0 not configured ppb0 at pci0 dev 6 function 0 "Intel C3000 PCIE" rev 0x11 pci1 at ppb0 bus 1 "Intel
Re: Rsync is too slow
No, I am not using USB.
Rsync is too slow
Latest obsd with new 4TB wd red ssd disk copying from 2TB seagate exos returns 80GB in 8 hours with zero activity by other tasks. The server has 12GB ecc ram cache. Copying 1.4 TB from a nas to the same exos took 2.5 hours shy. Is there a problem with how obsd handles internal storage? Or a problem with the default kernel sysconfig and staff defaults?
Re: Rsync is too slow
Chris Cappuccio [ch...@nmedia.net] wrote: > > such as: cp -r /usr/bin /mnt/usr/bin > or: tar cf - -C /usr/bin . | tar xpf - -C /mnt/usr/bin > also the destination filesystem should be mounted with async (dangerous on power loss) or softdep (not very dangerous on power loss) to avoid huge amounts of metadata updates slowing your action.
Re: Rsync is too slow
On Thu, 30 Jul 2020 13:37:39 -0700, Chris Cappuccio wrote: > Rupert Gallagher [r...@protonmail.com] wrote: > > No, I am not using USB. > > your dmesg didn't make it to the list because you are attaching a text file > and attachments are not allowed on misc. Actually, these days they are allowed. I didn't have any problem reading the attached dmesg. - todd
Re: Rsync is too slow
Rupert Gallagher [r...@protonmail.com] wrote: > No, I am not using USB. your dmesg didn't make it to the list because you are attaching a text file and attachments are not allowed on misc. please put it inline with the message.
Re: Rsync is too slow
Rupert Gallagher [r...@protonmail.com] wrote: > No, I am not using USB. rsync between disks should be very fast. you are going from the sata to the nvme ? NetBSD or FreeBSD or somebody made some speed improvements to nvme that we should review. i can't remember right now. anyways, 10GB/hour sounds extremely slow for an nvme SSD, way way way too slow for anything I have experienced in recent memory. it might be interesting to try using cp between filesystems, or tar such as: cp -r /usr/bin /mnt/usr/bin or: tar cf - -C /usr/bin . | tar xpf - -C /mnt/usr/bin also what speeds are you getting on the destination filesystem? dd count=1 bs=1G if=/dev/zero of=/mnt/test conv=fsync might give you some rough idea of what 1G write costs. here's 1G write on my Samsung 845DC Pro which is one of my all-time favorite SATA SSDs for reliability # dd count=1 bs=1G if=/dev/zero of=test conv=fsync 1+0 records in 1+0 records out 1073741824 bytes transferred in 2.906 secs (369450372 bytes/sec) here's the same for a Crucial M500 # dd count=1 bs=1G if=/dev/zero of=test conv=fsync 1+0 records in 1+0 records out 1073741824 bytes transferred in 4.356 secs (246484472 bytes/sec) it's not clear to me how much the buffer cache affects this but i'm hoping here that conv=fsync helps. in a wierd twist, tests like this with conv=fsync run consistently faster than without, so my understanding isn't that great.
Re: Rsync is too slow
Rupert Gallagher [r...@protonmail.com] wrote: > Latest obsd with new 4TB wd red ssd disk copying from 2TB seagate exos > returns 80GB in 8 hours with zero activity by other tasks. The server has > 12GB ecc ram cache. Copying 1.4 TB from a nas to the same exos took 2.5 hours > shy. Is there a problem with how obsd handles internal storage? Or a problem > with the default kernel sysconfig and staff defaults? sounds like you are copying from SATA to USB ? first, send a dmesg so people get a better idea. second, try -current. several improvements have been made for USB to be faster.
