Re: Problem with slow disk I/O
On Apr 23 18:09:55, Thomas Pfaff wrote: First on Ubuntu: /dev/sda2 on / type ext3 (rw,relatime,errors=remount-ro) ~$ time (tar -zxf ports.tar.gz sync) real0m47.784s 47.78 seconds wall clock time Then the same commands on OpenBSD: /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) $ time (tar -zxf ports.tar.gz sync) 1m2.62s real 0m1.15s user 0m7.15s system ~ 1 minute 2.5 seconds wall clock time So you have ~52 seconds on ext3 mounted 'realtime' (whatever that means), versus ~63 seconds on ffs mounted with 'softdep'. What was the problem again? That I cannot get the job done in less than a minute on OpenBSD while on Linux it takes only 18 seconds. Also, doesn't ext2/3 run with everything mount async? A quick test with ffs in async mode (instead of, or added to softdep) would also be worth running, in order to see how much grossly insecure I/O lessens the perceived time. I am one of those who like to keep my files, so I wont recommend USING async, but for the sake of argument here, such a test might be in order. Which reminds me to ask what the state of having a UBC in OpenBSD is, please? There is nothing close to it yet, to my knowledge, but I am hosting the 2009 filesystem hackathon this autumn in hopes of getting 'better' I/O out of OpenBSD, with the help of a nice grant to that goal. Perhaps magic will come out of that. History (and undeadly =) will tell. Mind you, I did run UBC on my obsd amiga back in the short while when art@ had UBC in, which did wonders when you have lots (128M) of ram and a PIO mode 0 harddisk to boot.
Re: Problem with slow disk I/O
First on Ubuntu:
/dev/sda2 on / type ext3 (rw,relatime,errors=remount-ro)
~$ time (tar -zxf ports.tar.gz sync)
real 0m47.784s
Then the same commands on OpenBSD:
/dev/wd0k on /home type ffs (local, nodev, nosuid, softdep)
$ time (tar -zxf ports.tar.gz sync)
1m2.62s real 0m1.15s user 0m7.15s system
So you have ~52 seconds on ext3 mounted 'realtime' (whatever that
means),
versus ~63 seconds on ffs mounted with 'softdep'.
Replying to myself,
'realtime' implies noatime, says http://lwn.net/Articles/244829/
(Isn't once upon atime an amusing title?)
And https://help.ubuntu.com/community/Fstab says that 'async'
is the default for Ubuntu ext3 mounts. Is your ext3 mounted
async? The mount line doesn't say so - but is that hidden
under 'realtime', too?
Also, doesn't ext2/3 run with everything mount async?
A quick test with ffs in async mode (instead of, or added to softdep)
would also be worth running, in order to see how much grossly insecure
I/O lessens the perceived time.
I am one of those who like to keep my
files, so I wont recommend USING async, but for the sake of argument
here, such a test might be in order.
softdep and async are mutually exclusive.
This is what happens with and without noatime (+ softdep, of course),
and with async replacing softdep, on my machine:
# uname -a
OpenBSD stary.dhcp.fjfi.cvut.cz 4.4 GENERIC.MP#2 i386
# mount
/dev/wd0a on / type ffs (local)
/dev/wd0d on /usr type ffs (local, nodev, softdep)
/dev/wd0e on /var type ffs (local, nodev, nosuid, softdep)
/dev/wd0f on /var/log type ffs (local, nodev, nosuid, softdep)
/dev/wd0g on /var/mail type ffs (local, nodev, nosuid, softdep)
/dev/wd0h on /tmp type ffs (local, nodev, nosuid, softdep)
/dev/wd0i on /home type ffs (local, nodev, nosuid, softdep)
/dev/wd0k on /dload type ffs (local, nodev, nosuid, softdep)
/dev/wd0j on /backup type ffs (local, nodev, nosuid, softdep)
# cd /backup
# ls -l ports.tar.gz
-rw-r--r-- 1 root wheel 14583699 Aug 9 2008 ports.tar.gz
# time { tar xzf ports.tar.gz ; sync ; }
1m5.51s real 0m0.00s user 0m0.00s system
# time rm -rf ports
0m13.88s real 0m0.20s user 0m1.56s system
# cd
# umount /backup
# mount -o nodev,nosuid,softdep,noatime /dev/wd0j /backup
# cd /backup
# time { tar xzf ports.tar.gz ; sync ; }
1m6.85s real 0m0.00s user 0m0.00s system
# time rm -rf ports
0m14.72s real 0m0.16s user 0m1.33s system
# cd
# umount /backup
# mount -o nodev,nosuid,async /dev/wd0j /backup
# cd /backup
# time { tar xzf ports.tar.gz ; sync ; }
0m39.44s real 0m0.00s user 0m0.01s system
# time rm -rf ports
0m6.80s real 0m0.19s user 0m1.45s system
Jan
Problem with slow disk I/O
I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real 0m18.440s user 0m1.212s sys 0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Thomas OpenBSD 4.5-current (GENERIC.MP) #13: Thu Apr 23 13:00:36 CEST 2009 tpf...@ws.tp76.info:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 3152609280 (3006MB) avail mem = 3045097472 (2904MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xf06b0 (76 entries) bios0: vendor American Megatrends Inc. version 1704 date 11/27/2007 bios0: ASUSTeK Computer INC. P5B-E acpi0 at bios0: rev 2 acpi0: tables DSDT FACP APIC MCFG OEMB HPET acpi0: wakeup devices P0P2(S4) P0P1(S4) UAR1(S4) PS2K(S4) PS2M(S4) EUSB(S4) USBE(S4) P0P4(S4) P0P5(S4) P0P6(S4) P0P7(S4) P0P8(S4) P0P9(S4) USB0(S4) USB1(S4) USB2(S4) USB3(S4) USB4(S4) USB5(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz, 2135.29 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG cpu0: 2MB 64b/line 8-way L2 cache cpu0: apic clock running at 266MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz, 2135.04 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG cpu1: 2MB 64b/line 8-way L2 cache ioapic0 at mainbus0 apid 2 pa 0xfec0, version 20, 24 pins acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (P0P2) acpiprt2 at acpi0: bus 5 (P0P1) acpiprt3 at acpi0: bus 4 (P0P4) acpiprt4 at acpi0: bus -1 (P0P5) acpiprt5 at acpi0: bus -1 (P0P6) acpiprt6 at acpi0: bus 3 (P0P7) acpiprt7 at acpi0: bus 2 (P0P8) acpicpu0 at acpi0 acpicpu1 at acpi0 acpibtn0 at acpi0: PWRB pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 Intel 82G965 Host rev 0x02 ppb0 at pci0 dev 1 function 0 Intel 82G965 PCIE rev 0x02: apic 2 int 16 (irq 11) pci1 at ppb0 bus 1 mem address conflict 0xc000/0x1000 vga1 at pci1 dev 0 function 0 NVIDIA GeForce 7600 GT rev 0xa1 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) uhci0 at pci0 dev 26 function 0 Intel 82801H USB rev 0x02: apic 2 int 16 (irq 11) uhci1 at pci0 dev 26 function 1 Intel 82801H USB rev 0x02: apic 2 int 17 (irq 5) ehci0 at pci0 dev 26 function 7 Intel 82801H USB rev 0x02: apic 2 int 18 (irq 15) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 azalia0 at pci0 dev 27 function 0 Intel 82801H HD Audio rev 0x02: apic 2 int 22 (irq 3) azalia0: codecs: Analog Devices AD1988A audio0 at azalia0 ppb1 at pci0 dev 28 function 0 Intel 82801H PCIE rev 0x02: apic 2 int 16 (irq 11) pci2 at ppb1 bus 4 ppb2 at pci0 dev 28 function 3 Intel 82801H PCIE rev 0x02: apic 2 int 19 (irq 10) pci3 at ppb2 bus 3 age0 at pci3 dev 0 function 0 Attansic Technology L1 rev 0xb0: apic 2 int 19 (irq 10), address 00:18:f3:9d:7d:04 atphy0 at age0 phy 0: F1 10/100/1000 PHY, rev. 5 ppb3 at pci0 dev 28 function 4 Intel 82801H PCIE rev 0x02: apic 2 int 16 (irq 11) pci4 at ppb3 bus 2 jmb0 at pci4 dev 0 function 0 JMicron JMB363 IDE/SATA rev 0x02 ahci0 at jmb0: apic 2 int 16 (irq 11), AHCI 1.0 scsibus0 at ahci0: 32 targets pciide0 at jmb0: DMA, channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide0: using apic 2 int 16 (irq 11) for native-PCI interrupt atapiscsi0 at pciide0 channel 0 drive 0 scsibus1 at atapiscsi0: 2 targets cd0 at scsibus1 targ 0 lun 0: PLEXTOR, DVDR PX-740A, 1.00 ATAPI 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 pciide0: channel 1 disabled (no drives) uhci2 at pci0 dev 29 function 0 Intel 82801H USB rev 0x02: apic 2 int 23 (irq 7) uhci3 at pci0 dev 29 function 1 Intel 82801H USB rev 0x02: apic 2 int 19 (irq 10) uhci4 at pci0 dev 29 function 2 Intel 82801H USB rev 0x02: apic 2 int 18 (irq 15) ehci1 at pci0 dev 29 function 7 Intel 82801H USB rev 0x02: apic 2 int 23 (irq 7) usb1 at ehci1: USB revision 2.0 uhub1 at usb1 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb4 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0xf2 pci5 at ppb4 bus 5 re0 at pci5 dev 2 function 0 D-Link Systems DGE-528T rev 0x10: RTL8169/8110SB
Re: Problem with slow disk I/O
On Thu, Apr 23, 2009 at 03:27:42PM +0200, Thomas Pfaff wrote: I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real0m18.440s user0m1.212s sys 0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Try: time tar -zxf ports.tar.gz sync
Re: Problem with slow disk I/O
On Thu, Apr 23, 2009 at 02:02:06PM +0200, Tobias Ulmer wrote: On Thu, Apr 23, 2009 at 03:27:42PM +0200, Thomas Pfaff wrote: I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real 0m18.440s user 0m1.212s sys 0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Try: time tar -zxf ports.tar.gz sync And include the output of mount and show the place where you are untarring. -Otto
Re: Problem with slow disk I/O
-Urspr|ngliche Nachricht- Von: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] Im Auftrag von Tobias Ulmer Gesendet: Donnerstag, 23. April 2009 14:02 An: Thomas Pfaff Cc: misc@openbsd.org Betreff: Re: Problem with slow disk I/O On Thu, Apr 23, 2009 at 03:27:42PM +0200, Thomas Pfaff wrote: I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real 0m18.440s user 0m1.212s sys 0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Try: time tar -zxf ports.tar.gz sync better use parenthesis: time ( tar -zxf ports.tar.gz sync ) Compare # time sleep 1 sleep 5 0m1.01s real 0m0.00s user 0m0.01s system to # time ( sleep 1 sleep 5 ) 0m6.01s real 0m0.00s user 0m0.03s system Christoph
Re: Problem with slow disk I/O
On Thu, 23 Apr 2009 14:13:18 +0200 Otto Moerbeek o...@drijf.net wrote: On Thu, Apr 23, 2009 at 02:02:06PM +0200, Tobias Ulmer wrote: On Thu, Apr 23, 2009 at 03:27:42PM +0200, Thomas Pfaff wrote: I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real0m18.440s user0m1.212s sys 0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Try: time tar -zxf ports.tar.gz sync And include the output of mount and show the place where you are untarring. $ mount /dev/wd0a on / type ffs (local, softdep) /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) fs:/pub on /pub type nfs (nodev, noexec, nosuid, v3, udp, timeo=100) $ pwd /home/tpfaff $ time tar -zxf ports.tar.gz sync # (... sync) ~same result 1m2.66s real 0m1.09s user 0m6.85s system $ time rm -rf ports 0m15.20s real 0m0.15s user 0m1.42s system
Re: Problem with slow disk I/O
On Thu, Apr 23, 2009 at 05:19:34PM +0200, Thomas Pfaff wrote: On Thu, 23 Apr 2009 14:13:18 +0200 Otto Moerbeek o...@drijf.net wrote: On Thu, Apr 23, 2009 at 02:02:06PM +0200, Tobias Ulmer wrote: On Thu, Apr 23, 2009 at 03:27:42PM +0200, Thomas Pfaff wrote: I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real 0m18.440s user 0m1.212s sys 0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Try: time tar -zxf ports.tar.gz sync And include the output of mount and show the place where you are untarring. $ mount /dev/wd0a on / type ffs (local, softdep) /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) fs:/pub on /pub type nfs (nodev, noexec, nosuid, v3, udp, timeo=100) $ pwd /home/tpfaff $ time tar -zxf ports.tar.gz sync # (... sync) ~same result 1m2.66s real 0m1.09s user 0m6.85s system $ time rm -rf ports 0m15.20s real 0m0.15s user 0m1.42s system and on linux? [ remembered the () thing the very second i hit 'y' ;). I figured you're clever enough... ]
Re: Problem with slow disk I/O
On Thu, 23 Apr 2009 15:27:42 +0200 Thomas Pfaff tpf...@tp76.info wrote: I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). This is on different hardware now, BTW (not the one that crashed).
Re: Problem with slow disk I/O
Thomas Pfaff wrote: On Thu, 23 Apr 2009 15:27:42 +0200 Thomas Pfaff tpf...@tp76.info wrote: I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). This is on different hardware now, BTW (not the one that crashed). Is your chipset revision recognized by OpenBSD? I had a similar problem with a new motherboard and upgrading to the latest snapshot resolved it. A dmesg would be helpful. Regards,
Re: Problem with slow disk I/O
On Thu, 23 Apr 2009 09:54:15 -0400 gjones gjones5...@netscape.net wrote: Thomas Pfaff wrote: On Thu, 23 Apr 2009 15:27:42 +0200 This is on different hardware now, BTW (not the one that crashed). Is your chipset revision recognized by OpenBSD? I had a similar problem with a new motherboard and upgrading to the latest snapshot resolved it. A dmesg would be helpful. I provided that in my first post ;-)
Re: Problem with slow disk I/O
On Thu, 23 Apr 2009 15:31:58 +0200 Tobias Ulmer tobi...@tmux.org wrote: [...] Try: time tar -zxf ports.tar.gz sync And include the output of mount and show the place where you are untarring. $ mount /dev/wd0a on / type ffs (local, softdep) /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) fs:/pub on /pub type nfs (nodev, noexec, nosuid, v3, udp, timeo=100) $ pwd /home/tpfaff $ time tar -zxf ports.tar.gz sync # (... sync) ~same result 1m2.66s real 0m1.09s user 0m6.85s system $ time rm -rf ports 0m15.20s real 0m0.15s user 0m1.42s system and on linux? First on Ubuntu: Script started on Thu 23 Apr 2009 03:50:27 PM CEST ~$ time (tar -zxf ports.tar.gz sync) real0m47.784s user0m1.576s sys 0m5.024s ~$ time (rm -rf ports sync) real0m1.883s user0m0.076s sys 0m1.664s time (tar -zxf ports.tar.gz sync) real0m20.652s user0m1.240s sys 0m2.592s ~$ time (rm -rf sync) real0m0.003s user0m0.004s sys 0m0.004s ~$ time tar -zxf ports.tar.gz real0m11.513s user0m1.268s sys 0m2.772s ~$ time rm -rf ports real0m1.752s user0m0.100s sys 0m1.648s ~$ time tar -zxf ports.tar.gz real0m14.400s user0m1.352s sys 0m2.560s ~$ time rm -rf ports real0m1.756s user0m0.076s sys 0m1.684s ~$ mount # watch your eyes! /dev/sda2 on / type ext3 (rw,relatime,errors=remount-ro) proc on /proc type proc (rw,noexec,nosuid,nodev) /sys on /sys type sysfs (rw,noexec,nosuid,nodev) varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755) varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777) udev on /dev type tmpfs (rw,mode=0755) devshm on /dev/shm type tmpfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) lrm on /lib/modules/2.6.24-19-generic/volatile type tmpfs (rw) securityfs on /sys/kernel/security type securityfs (rw) gvfs-fuse-daemon on /home/tpfaff/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=tpfaff) ~$ pwd /home/tpfaff ~$ exit Script done on Thu 23 Apr 2009 03:53:20 PM CEST Then the same commands on OpenBSD: Script started on Thu Apr 23 17:55:53 2009 $ time (tar -zxf ports.tar.gz sync) 1m2.62s real 0m1.15s user 0m7.15s system $ time (rm -rf ports sync) 0m14.24s real 0m0.14s user 0m1.53s system $ time (tar -zxf ports.tar.gz sync) 1m1.37s real 0m1.31s user 0m7.18s system $ time (rm -rf ports sync) 0m14.72s real 0m0.12s user 0m1.82s system $ time tar -zxf ports.tar.gz 1m3.39s real 0m1.08s user 0m6.69s system $ time rm -rf ports 0m15.41s real 0m0.12s user 0m1.38s system $ time tar -zxf ports.tar.gz 1m2.62s real 0m1.19s user 0m6.80s system $ time rm -rf ports 0m15.63s real 0m0.10s user 0m1.79s system $ mount /dev/wd0a on / type ffs (local, softdep) /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) fs:/pub on /pub type nfs (nodev, noexec, nosuid, v3, udp, timeo=100) $ pwd /home/tpfaff $ exit Script done on Thu Apr 23 18:02:13 2009
Re: Problem with slow disk I/O
On Thu, 23 Apr 2009 17:25:57 +0200 Jan Stary h...@stare.cz wrote: On Apr 23 18:09:55, Thomas Pfaff wrote: First on Ubuntu: /dev/sda2 on / type ext3 (rw,relatime,errors=remount-ro) ~$ time (tar -zxf ports.tar.gz sync) real0m47.784s user0m1.576s sys 0m5.024s Then the same commands on OpenBSD: /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) $ time (tar -zxf ports.tar.gz sync) 1m2.62s real 0m1.15s user 0m7.15s system So you have ~52 seconds on ext3 mounted 'realtime' (whatever that means), versus ~63 seconds on ffs mounted with 'softdep'. What was the problem again? That I cannot get the job done in less than a minute on OpenBSD while on Linux it takes only 18 seconds. What happens with 'noatime' on the ffs partition? Script started on Thu Apr 23 19:35:37 2009 $ mount /dev/wd0a on / type ffs (local, softdep) /dev/wd0k on /home type ffs (local, noatime, nodev, nosuid, softdep) fs:/pub on /pub type nfs (nodev, noexec, nosuid, v3, udp, timeo=100) $ pwd /home/tpfaff $ time tar -xzf ports.tar.gz 1m3.92s real 0m0.97s user 0m7.09s system $ time rm -rf ports 0m15.34s real 0m0.16s user 0m1.43s system $ exit Script done on Thu Apr 23 19:37:20 2009
Re: Problem with slow disk I/O
On Thu, 23.04.2009 at 19:40:34 +0200, Thomas Pfaff tpf...@tp76.info wrote: On Thu, 23 Apr 2009 17:25:57 +0200 Jan Stary h...@stare.cz wrote: On Apr 23 18:09:55, Thomas Pfaff wrote: First on Ubuntu: /dev/sda2 on / type ext3 (rw,relatime,errors=remount-ro) ~$ time (tar -zxf ports.tar.gz sync) real 0m47.784s user 0m1.576s sys 0m5.024s 47.78 seconds wall clock time Then the same commands on OpenBSD: /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) $ time (tar -zxf ports.tar.gz sync) 1m2.62s real 0m1.15s user 0m7.15s system ~ 1 minute 2.5 seconds wall clock time So you have ~52 seconds on ext3 mounted 'realtime' (whatever that means), versus ~63 seconds on ffs mounted with 'softdep'. What was the problem again? That I cannot get the job done in less than a minute on OpenBSD while on Linux it takes only 18 seconds. This is a misconception, imho. Your test above shows that the performance difference is about 15 seconds, or roughly 25%. I can't see the 18 seconds anywhere except in your first email about your perceived performance for the task. It is imho useful to remember that Linux caches disk access much more aggressively than OpenBSD. So, in reality, you don't write that much faster to disk, but to RAM, and the OS flushes the buffers at it's own leisure, while you are working on something else. Which reminds me to ask what the state of having a UBC in OpenBSD is, please? -- Kind regards, --Toni++
Re: Problem with slow disk I/O
On Thu, Apr 23, 2009 at 06:09:55PM +0200, Thomas Pfaff wrote: On Thu, 23 Apr 2009 15:31:58 +0200 Tobias Ulmer tobi...@tmux.org wrote: [...] Try: time tar -zxf ports.tar.gz sync And include the output of mount and show the place where you are untarring. $ mount /dev/wd0a on / type ffs (local, softdep) /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) fs:/pub on /pub type nfs (nodev, noexec, nosuid, v3, udp, timeo=100) $ pwd /home/tpfaff $ time tar -zxf ports.tar.gz sync # (... sync) ~same result 1m2.66s real 0m1.09s user 0m6.85s system $ time rm -rf ports 0m15.20s real 0m0.15s user 0m1.42s system and on linux? First on Ubuntu: Script started on Thu 23 Apr 2009 03:50:27 PM CEST ~$ time (tar -zxf ports.tar.gz sync) real 0m47.784s user 0m1.576s sys 0m5.024s ~$ time (rm -rf ports sync) real 0m1.883s user 0m0.076s sys 0m1.664s time (tar -zxf ports.tar.gz sync) real 0m20.652s user 0m1.240s sys 0m2.592s ~$ time (rm -rf sync) real 0m0.003s user 0m0.004s sys 0m0.004s ~$ time tar -zxf ports.tar.gz real 0m11.513s user 0m1.268s sys 0m2.772s ~$ time rm -rf ports real 0m1.752s user 0m0.100s sys 0m1.648s ~$ time tar -zxf ports.tar.gz real 0m14.400s user 0m1.352s sys 0m2.560s ~$ time rm -rf ports real 0m1.756s user 0m0.076s sys 0m1.684s ~$ mount # watch your eyes! /dev/sda2 on / type ext3 (rw,relatime,errors=remount-ro) proc on /proc type proc (rw,noexec,nosuid,nodev) /sys on /sys type sysfs (rw,noexec,nosuid,nodev) varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755) varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777) udev on /dev type tmpfs (rw,mode=0755) devshm on /dev/shm type tmpfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) lrm on /lib/modules/2.6.24-19-generic/volatile type tmpfs (rw) securityfs on /sys/kernel/security type securityfs (rw) gvfs-fuse-daemon on /home/tpfaff/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=tpfaff) ~$ pwd /home/tpfaff ~$ exit Script done on Thu 23 Apr 2009 03:53:20 PM CEST Then the same commands on OpenBSD: Script started on Thu Apr 23 17:55:53 2009 $ time (tar -zxf ports.tar.gz sync) 1m2.62s real 0m1.15s user 0m7.15s system $ time (rm -rf ports sync) 0m14.24s real 0m0.14s user 0m1.53s system $ time (tar -zxf ports.tar.gz sync) 1m1.37s real 0m1.31s user 0m7.18s system $ time (rm -rf ports sync) 0m14.72s real 0m0.12s user 0m1.82s system $ time tar -zxf ports.tar.gz 1m3.39s real 0m1.08s user 0m6.69s system $ time rm -rf ports 0m15.41s real 0m0.12s user 0m1.38s system $ time tar -zxf ports.tar.gz 1m2.62s real 0m1.19s user 0m6.80s system $ time rm -rf ports 0m15.63s real 0m0.10s user 0m1.79s system $ mount /dev/wd0a on / type ffs (local, softdep) /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) fs:/pub on /pub type nfs (nodev, noexec, nosuid, v3, udp, timeo=100) $ pwd /home/tpfaff $ exit Script done on Thu Apr 23 18:02:13 2009 ext3 and ffs are very different. So the same thing may take a different time to finish on either system because of different design decisions. From your benchmark it seems your server's only purpose is to untar and remove ports.tar.gz in a loop or what are you trying to show? I'm very happy that OpenBSD ffs is a bit slower then Linux ext3. -- :wq Claudio
Re: Problem with slow disk I/O
Thomas Pfaff wrote: On Thu, 23 Apr 2009 14:13:18 +0200 Otto Moerbeek o...@drijf.net wrote: On Thu, Apr 23, 2009 at 02:02:06PM +0200, Tobias Ulmer wrote: On Thu, Apr 23, 2009 at 03:27:42PM +0200, Thomas Pfaff wrote: I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real 0m18.440s user 0m1.212s sys 0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Try: time tar -zxf ports.tar.gz sync And include the output of mount and show the place where you are untarring. $ mount /dev/wd0a on / type ffs (local, softdep) /dev/wd0k on /home type ffs (local, nodev, nosuid, softdep) fs:/pub on /pub type nfs (nodev, noexec, nosuid, v3, udp, timeo=100) $ pwd /home/tpfaff $ time tar -zxf ports.tar.gz sync # (... sync) ~same result 1m2.66s real 0m1.09s user 0m6.85s system $ time rm -rf ports 0m15.20s real 0m0.15s user 0m1.42s system Sorry for the long message. I tried your test commands on three different servers -- two Dell servers and a generic box with Gigabyte motherboard. If there are any other tests I can run to help with this problem please let me know. Server1 has OpenBSD 4.4 -stable, GENERIC.MP, Dell 2950, PERC/6i, 15K SAS drives, two 146GB drives as RAID1 and another two 450GB drives as RAID1 On either RAID1 array, I get results similar to the following: server1$ time tar xzf ports.tar.gz 0m55.28s real 0m1.57s user 0m6.40s system server1$ time rm -rf ports 0m36.24s real 0m0.19s user 0m2.57s system Server2 has OpenBSD 4.5 GENERIC.MP#46 i386 (snapshot from April 12, 2009 or so), two 450GB drives as RAID1 and six 450GB drives as RAID6 On RAID1 array server2$ time tar xzf ports.tar.gz 0m56.95s real 0m1.47s user 0m6.08s system server2$ time rm -rf ports 0m32.44s real 0m0.19s user 0m2.50s system On RAID6 array server2$ time tar xzf ../ports.tar.gz 1m9.15s real 0m1.11s user 0m6.44s system server2$ time rm -rf ports 0m30.71s real 0m0.14s user 0m2.90s system On a new server (OpenBSD 4.4-stable, 320GB SATA drives), AHCI, test$ time tar xzf ports.tar.gz 5m12.78s real 0m1.39s user 0m5.04s system test# time rm -rf ports 2m34.91s real 0m0.15s user 0m2.59s system Because of the 103095897 bytes/sec transfer rate on the test box I thought that the test box would have faster writes but it seems to have taken 5 minutes compared to the Dell server which had a transfer rate of 75528478 bytes/sec. On Dell PE2950 servers with 15K RPM SAS drives I get $ dd if=/dev/arandom of=test.dat bs=1024 count=1 1+0 records in 1+0 records out 1024 bytes transferred in 0.135 secs (75528478 bytes/sec) Here is the output of mount on server1 and server2 server1# mount /dev/sd0a on / type ffs (local) /dev/sd0h on /home type ffs (local, nodev, nosuid) /dev/sd0g on /tmp type ffs (local, nodev, nosuid) /dev/sd0d on /usr type ffs (local, nodev) /dev/sd0e on /var type ffs (local, nodev, nosuid) /dev/sd0f on /var/www type ffs (local, nodev, nosuid) /dev/sd1a on /apps type ffs (local, nodev, nosuid) /dev/sd1b on /source type ffs (local, nodev, nosuid) /dev/sd1d on /products type ffs (local, nodev, nosuid) /dev/sd1e on /storage type ffs (local, nodev, nosuid) server2# mount /dev/sd0a on / type ffs (local) /dev/sd0k on /home type ffs (local, nodev, nosuid) /dev/sd0d on /tmp type ffs (local, nodev, nosuid) /dev/sd0f on /usr type ffs (local, nodev) /dev/sd0g on /usr/X11R6 type ffs (local, nodev) /dev/sd0h on /usr/local type ffs (local, nodev) /dev/sd0j on /usr/obj type ffs (local, nodev, nosuid) /dev/sd0i on /usr/src type ffs (local, nodev, nosuid) /dev/sd0e on /var type ffs (local, nodev, nosuid) /dev/sd1f on /mnt type ffs (local) On a generic box (gigabyte motherboard, 320GB 7K RPM SAS drives etc.) I get test$ dd if=/dev/arandom of=test.dat bs=1024 count=1 1+0 records in 1+0 records out 1024 bytes transferred in 0.099 secs (103095897 bytes/sec) Output of mount on test$ test$ mount /dev/sd0a on / type ffs (local) /dev/sd0h on /home type ffs (local, nodev, nosuid) /dev/sd0g on /tmp type ffs (local, nodev, nosuid) /dev/sd0d on /usr type ffs (local, nodev) /dev/sd0e on /var type ffs (local, nodev, nosuid) /dev/sd0f on /var/www type ffs (local, nodev, nosuid) ports.tar.gz was in /home/vsankar on each of the RAID1 arrays
Re: Problem with slow disk I/O
From your benchmark it seems your server's only purpose is to untar and remove ports.tar.gz in a loop or what are you trying to show? I'm very happy that OpenBSD ffs is a bit slower then Linux ext3. He's showing that for his test case, he should be running Linux. Enjoy.
Re: Problem with slow disk I/O
On Thu, 23 Apr 2009 18:40:53 +0200 Claudio Jeker cje...@diehard.n-r-g.com wrote: ext3 and ffs are very different. So the same thing may take a different time to finish on either system because of different design decisions. From your benchmark it seems your server's only purpose is to untar and remove ports.tar.gz in a loop or what are you trying to show? It's my workstation and I'm not trying to show anything. It was a simple observation I made and I was curious if there was something funny going on with my system, or if the performance difference in this particular case is considered normal.
Re: Problem with slow disk I/O
On Thu, Apr 23, 2009 at 12:35 PM, Vijay Sankar vsan...@foretell.ca wrote: Because of the 103095897 bytes/sec transfer rate on the test box I thought that the test box would have faster writes but it seems to have taken 5 minutes compared to the Dell server which had a transfer rate of 75528478 bytes/sec. Sequential write speed has practically nothing to do with the time it takes to create or delete a directory structure.
Re: Problem with slow disk I/O
Those are my numbers.It was running during my normal work : 10 x Xterm in one of them is ogg123 playing song 1 x FF3 with 7 tabs 1 x Pidgin with 5 tabs 2 x rdesktop to Win servers 1 x ssh connection to company server 1 x vpnc softdep is on on all partitions I don't think that those numbers are bad,because ports.tar.gz has a lot of small files.And I use OpenBSD for many good reasons like stabitilty,security,quality,community and documentation.So if sometimes it's slower then other OS's I don't care if it isn't a bug ;-) $ uname -a OpenBSD hexempo.eu.tieto.com 4.5 GENERIC.MP#78 i386 $ $ dd if=/dev/arandom of=test.dat bs=1024 count=1 1+0 records in 1+0 records out 1024 bytes transferred in 0.106 secs (96246029 bytes/sec) $ $ time tar xzf ports.tar.gz 2m19.16s real 0m1.41s user 0m5.45s system $ 2009/4/23 Thomas Pfaff tpf...@tp76.info: On Thu, 23 Apr 2009 18:40:53 +0200 Claudio Jeker cje...@diehard.n-r-g.com wrote: ext3 and ffs are very different. So the same thing may take a different time to finish on either system because of different design decisions. From your benchmark it seems your server's only purpose is to untar and remove ports.tar.gz in a loop or what are you trying to show? It's my workstation and I'm not trying to show anything. B It was a simple observation I made and I was curious if there was something funny going on with my system, or if the performance difference in this particular case is considered normal. -- http://www.openbsd.org/lyrics.html
Re: Problem with slow disk I/O
--- On Thu, 4/23/09, Thomas Pfaff tpf...@tp76.info wrote: From: Thomas Pfaff tpf...@tp76.info Subject: Problem with slow disk I/O To: misc@openbsd.org Received: Thursday, April 23, 2009, 9:27 AM I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real0m18.440s user0m1.212s sys0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Thomas OpenBSD 4.5-current (GENERIC.MP) #13: Thu Apr 23 13:00:36 CEST 2009 tpf...@ws.tp76.info:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 3152609280 (3006MB) avail mem = 3045097472 (2904MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xf06b0 (76 entries) bios0: vendor American Megatrends Inc. version 1704 date 11/27/2007 bios0: ASUSTeK Computer INC. P5B-E acpi0 at bios0: rev 2 acpi0: tables DSDT FACP APIC MCFG OEMB HPET acpi0: wakeup devices P0P2(S4) P0P1(S4) UAR1(S4) PS2K(S4) PS2M(S4) EUSB(S4) USBE(S4) P0P4(S4) P0P5(S4) P0P6(S4) P0P7(S4) P0P8(S4) P0P9(S4) USB0(S4) USB1(S4) USB2(S4) USB3(S4) USB4(S4) USB5(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz, 2135.29 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16, xTPR,NXE,LONG cpu0: 2MB 64b/line 8-way L2 cache cpu0: apic clock running at 266MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz, 2135.04 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16, xTPR,NXE,LONG cpu1: 2MB 64b/line 8-way L2 cache ioapic0 at mainbus0 apid 2 pa 0xfec0, version 20, 24 pins acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (P0P2) acpiprt2 at acpi0: bus 5 (P0P1) acpiprt3 at acpi0: bus 4 (P0P4) acpiprt4 at acpi0: bus -1 (P0P5) acpiprt5 at acpi0: bus -1 (P0P6) acpiprt6 at acpi0: bus 3 (P0P7) acpiprt7 at acpi0: bus 2 (P0P8) acpicpu0 at acpi0 acpicpu1 at acpi0 acpibtn0 at acpi0: PWRB pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 Intel 82G965 Host rev 0x02 ppb0 at pci0 dev 1 function 0 Intel 82G965 PCIE rev 0x02: apic 2 int 16 (irq 11) pci1 at ppb0 bus 1 mem address conflict 0xc000/0x1000 vga1 at pci1 dev 0 function 0 NVIDIA GeForce 7600 GT rev 0xa1 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) uhci0 at pci0 dev 26 function 0 Intel 82801H USB rev 0x02: apic 2 int 16 (irq 11) uhci1 at pci0 dev 26 function 1 Intel 82801H USB rev 0x02: apic 2 int 17 (irq 5) ehci0 at pci0 dev 26 function 7 Intel 82801H USB rev 0x02: apic 2 int 18 (irq 15) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 azalia0 at pci0 dev 27 function 0 Intel 82801H HD Audio rev 0x02: apic 2 int 22 (irq 3) azalia0: codecs: Analog Devices AD1988A audio0 at azalia0 ppb1 at pci0 dev 28 function 0 Intel 82801H PCIE rev 0x02: apic 2 int 16 (irq 11) pci2 at ppb1 bus 4 ppb2 at pci0 dev 28 function 3 Intel 82801H PCIE rev 0x02: apic 2 int 19 (irq 10) pci3 at ppb2 bus 3 age0 at pci3 dev 0 function 0 Attansic Technology L1 rev 0xb0: apic 2 int 19 (irq 10), address 00:18:f3:9d:7d:04 atphy0 at age0 phy 0: F1 10/100/1000 PHY, rev. 5 ppb3 at pci0 dev 28 function 4 Intel 82801H PCIE rev 0x02: apic 2 int 16 (irq 11) pci4 at ppb3 bus 2 jmb0 at pci4 dev 0 function 0 JMicron JMB363 IDE/SATA rev 0x02 ahci0 at jmb0: apic 2 int 16 (irq 11), AHCI 1.0 scsibus0 at ahci0: 32 targets pciide0 at jmb0: DMA, channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide0: using apic 2 int 16 (irq 11) for native-PCI interrupt atapiscsi0 at pciide0 channel 0 drive 0 scsibus1 at atapiscsi0: 2 targets cd0 at scsibus1 targ 0 lun 0: PLEXTOR, DVDR PX-740A, 1.00 ATAPI 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 pciide0: channel 1 disabled (no drives) uhci2 at pci0 dev 29 function 0 Intel 82801H USB rev 0x02: apic 2 int 23 (irq 7) uhci3 at pci0 dev 29 function 1 Intel 82801H USB rev 0x02: apic 2 int 19 (irq 10) uhci4 at pci0 dev 29 function 2 Intel 82801H USB rev 0x02: apic 2 int 18 (irq 15) ehci1 at pci0
