[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: linux-linaro Status: Fix Committed = Fix Released ** Changed in: linux-linaro Assignee: Mian Yousaf Kaukab (mian-yousaf-kaukab) = (unassigned) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
I have no USB analyzer in my office. The 'git bisect' was one of the few items remaining in my todo list. Thanks for your time. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 709245] Re: ARM SMP scheduler performance bug
Emeric, On Wed, Jul 11, 2012 at 1:20 AM, Emeric Vigier 709...@bugs.launchpad.net wrote: Ming Lei, Delay between URB submission and call-back for a given URB is huge indeed... My hardware (panda rev A3) is fine: linaro-12.05 (kernel-3.4) and linaro-11.09's ethernet works just fine. Yes, so it may be caused by ehci driver or some ehci-omap low level configuration. But I have no idea about the root cause since there are very few information about the problem. Maybe you need to use a USB analyzer to trace the bus traffic if you have it. Or one easier way is to run 'git bisect' on upstream kernel or stable-tree to look for the commit which fixes the problem. Thanks, -- Ming Lei -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 709245] Re: ARM SMP scheduler performance bug
Emeric， From your usbmon trace, I can find that your TX is extremely slow: It may take several seconds to complete one packet with 1514bytes. Could you run upstream kernel(3.4 or 3.5-rcX) first to see if the hardware of your smsc95xx NIC is OK? And you'd better to test a usb storage transfer on the omap4 ehci host. I guess the hardware of smsc95xx may be bad. Thanks, -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Ming Lei, Delay between URB submission and call-back for a given URB is huge indeed... My hardware (panda rev A3) is fine: linaro-12.05 (kernel-3.4) and linaro-11.09's ethernet works just fine. My aosp kernel is based on kernel-3.0.8 but has wide differences with linaro-11.09 (3.1) kernel, starting with PM_RUNTIME support. I ran the mass storage test on a Kingston DT mini-fun 4GB usb stick, featuring one ext3 partition. On my panda, I ran: # mount -o remount,rw / # mkdir /mnt/msd # mount -t ext3 /dev/block/sda1 /mnt/msd # sync echo 3 /proc/sys/vm/drop_caches # busybox dd bs=1M count=16 if=/dev/zero of=/mnt/msd/16MB-file conv=fsync conv=notrunc 16777216 bytes (16.0MB) copied, 144.580627 seconds, 113.3KB/s # sync echo 3 /proc/sys/vm/drop_caches # busybox dd bs=16M count=1 if=/dev/zero of=/mnt/msd/16MB-file conv=fsync conv=notrunc 16777216 bytes (16.0MB) copied, 137.893890 seconds, 118.8KB/s Then I assume issue is in ehci stack, not the net stack. Running top shows the cpus are mostly idle during the transfer. I ran usbmon before pluging the usb stick, mounting the FS, and starting the transfer. I interrupted it few seconds after transfer started. # cat /sys/kernel/debug/usb/usbmon/1u /sdcard/1.mon.out.20 http://paste.ubuntu.com/1084832/ dmesg: http://paste.ubuntu.com/1084838/ T: Bus=01 Lev=02 Prnt=02 Port=02 Cnt=03 Dev#= 5 Spd=480 MxCh= 0 D: Ver= 2.00 Cls=00(ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1 P: Vendor=0930 ProdID=6545 Rev= 1.00 S: Manufacturer=Kingston S: Product=DT Mini Fun G2 S: SerialNumber=0060E049DF6FEB908883 C:* #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=200mA I:* If#= 0 Alt= 0 #EPs= 2 Cls=08(stor.) Sub=06 Prot=50 Driver=usb-storage E: Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms E: Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=0m -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
1. Here I booted Android without RJ45 cable plugged. I issued: # cat /sys/kernel/debug/usb/usbmon/1u /sdcard/1.mon.out.11 2. Then I plugged in the cable, got an IP address through DHCP: (panda) # netcfg eth0 dhcp eth0 UP 192.168.50.107/24 0x1043 42:82:7e:fd:d9:71 3. Then I started a TX using nc command: (panda) # nc -l -p 1032 /dev/zero (desktop-pc) $ nc 192.168.50.107 1032 /dev/null (desktop-pc) $ sudo nethogs nc eth0 0.486 12.532 KB/sec Speed is very poor. 4. I wait for netdev watchdog to bark and stopped the nc on desktop-pc side. Then killed the nc on panda side. And eventually ctrl'ed-C the usbmon dump. On my dmesg, you can see the slow path occurring after TX struggled for a while: http://paste.ubuntu.com/1083276/ The usbmon trace on bus 1: http://paste.ubuntu.com/1083284/ My usb connected devices: http://paste.ubuntu.com/1083292/ Thanks for your kind support, Emeric -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
I ran iperf as well and got poor results with both 256KB and 16Bytes (2KB). (desktop-pc) $ iperf -s -w 256K Server listening on TCP port 5001 TCP window size: 256 KByte (WARNING: requested 256 KByte) [ 4] local 192.168.50.223 port 5001 connected with 192.168.50.107 port 39723 (panda) # iperf -c 192.168.50.223 -w 256K -t 10 Client connecting to 192.168.50.223, TCP port 5001 TCP window size: 256 KByte (WARNING: requested 256 KByte) [ 3] local 192.168.50.107 port 39723 connected with 192.168.50.223 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-13.3 sec200 KBytes123 Kbits/sec (desktop-pc) $ iperf -s -w 16 WARNING: TCP window size set to 16 bytes. A small window size will give poor performance. See the Iperf documentation. Server listening on TCP port 5001 TCP window size: 2.23 KByte (WARNING: requested 16.0 Byte) [ 4] local 192.168.50.223 port 5001 connected with 192.168.50.107 port 39724 [ ID] Interval Transfer Bandwidth [ 4] 0.0-18.3 sec 144 KBytes 64.5 Kbits/sec (panda) # iperf -c 192.168.50.223 -w 16 -t 10 WARNING: TCP window size set to 16 bytes. A small window size will give poor performance. See the Iperf documentation. Client connecting to 192.168.50.223, TCP port 5001 TCP window size: 2.00 KByte (WARNING: requested 16.0 Byte) [ 3] local 192.168.50.107 port 39724 connected with 192.168.50.223 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-18.3 sec144 KBytes 64.5 Kbits/sec -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 709245] Re: ARM SMP scheduler performance bug
On Fri, Jul 6, 2012 at 9:57 PM, Emeric Vigier 709...@bugs.launchpad.net wrote: I don't think it is the same problem. My ethernet TX gets stuck stuck to 10-20KBps (80-160Kbps). When I force some activity (USB mouse events) it can reach ~1MBps (8Mbps). Also the interface is supposed to (and says to) be 100Mbps... I opened a thread on various mailing-lists: http://comments.gmane.org/gmane.linux.network/234335 I saw your ehci patch on this thread: https://patchwork.kernel.org/patch/1113332/ The thing is, my kernel already includes your patch. I also included Will Deacon's patch in process.c (cpu_idle), the one you mentioned here. But I have some reasons to believe that none of the patches are suited in my case because: 1. Ethernet TX performance is still very bad with these patches and corresponding configs, 2. smsc95xx driver relies on usbnet. I think that usbnet uses interrupt for RX but not for TX. Then flushing the caches with wmb() just before entering idle (WFI in my case) won't be efficient for me as ethernet TX has no way to wake-up the cpu, it has to wait for an external wakeup source. USB Mouse event improves TX performance. 3. I cannot state whether your patch in ehci-q.c should work for me, because I don't know if (TX) URB submission in usbnet finally goes through this file's functions or another. Do you know if my statements above make sense? Do you know where I should put the memory barrier and test again in my particular case? OK, I don't suggest that you do some blind test further without knowing the reason. Could you provide the trace result according to the guide in Documentation/usb/usbmon.txt or the link[1]? [1], http://www.mjmwired.net/kernel/Documentation/usb/usbmon.txt thanks, -- Ming Lei -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
I don't think it is the same problem. My ethernet TX gets stuck stuck to 10-20KBps (80-160Kbps). When I force some activity (USB mouse events) it can reach ~1MBps (8Mbps). Also the interface is supposed to (and says to) be 100Mbps... I opened a thread on various mailing-lists: http://comments.gmane.org/gmane.linux.network/234335 I saw your ehci patch on this thread: https://patchwork.kernel.org/patch/1113332/ The thing is, my kernel already includes your patch. I also included Will Deacon's patch in process.c (cpu_idle), the one you mentioned here. But I have some reasons to believe that none of the patches are suited in my case because: 1. Ethernet TX performance is still very bad with these patches and corresponding configs, 2. smsc95xx driver relies on usbnet. I think that usbnet uses interrupt for RX but not for TX. Then flushing the caches with wmb() just before entering idle (WFI in my case) won't be efficient for me as ethernet TX has no way to wake-up the cpu, it has to wait for an external wakeup source. USB Mouse event improves TX performance. 3. I cannot state whether your patch in ehci-q.c should work for me, because I don't know if (TX) URB submission in usbnet finally goes through this file's functions or another. Do you know if my statements above make sense? Do you know where I should put the memory barrier and test again in my particular case? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Hi, I am having ethernet performance issue on panda rev A3. TX gets stuck while RX seems fine. At least pinging is ok... I started a thread on various mailing-list (http://comments.gmane.org/gmane.linux.network/234335). I made a lot of trials but no luck so far... My problem may be related to the one described here for usb storage. My kernel already embeds Ming Lei's patch (https://patchwork.kernel.org/patch/1113332/), and CONFIG_ARM_DMA_MEM_BUFFERABLE and CONFIG_ARCH_HAS_BARRIERS are enabled. But still ethernet performance is really poor. Smsc9514 device driver relies on usbnet. I lack usb knowledge, thus I cannot state whether Ming Lei's patch in ehci-q.c should fix my issue if they were related. Otherwise, did anyone find out that CONFIG_PL310_ERRATA_769419 patch above effectively fixes the usb storage issue? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 709245] Re: ARM SMP scheduler performance bug
On Fri, Jul 6, 2012 at 6:57 AM, Emeric Vigier 709...@bugs.launchpad.net wrote: Hi, I am having ethernet performance issue on panda rev A3. TX gets stuck while RX seems fine. At least pinging is ok... We are discussing a similar problem now, see below link: http://marc.info/?t=13414531181r=1w=2 I found that enabling DEBUG_SLAB may degrade TX performance a lot( ~35Mbps). But if the option is disabled, the TX performance should be OK( 90Mbps). I am not sure if it is same with your problem. I started a thread on various mailing-list (http://comments.gmane.org/gmane.linux.network/234335). I made a lot of trials but no luck so far... My problem may be related to the one described here for usb storage. My kernel already embeds Ming Lei's patch (https://patchwork.kernel.org/patch/1113332/), and CONFIG_ARM_DMA_MEM_BUFFERABLE and CONFIG_ARCH_HAS_BARRIERS are enabled. But still ethernet performance is really poor. Smsc9514 device driver relies on usbnet. I lack usb knowledge, thus I cannot state whether Ming Lei's patch in ehci-q.c should fix my issue if they were related. Otherwise, did anyone find out that CONFIG_PL310_ERRATA_769419 patch above effectively fixes the usb storage issue? Yes, it is fine for the problem, so the ehci patch isn't needed to upstream. Thanks, -- Ming Lei -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Branch linked: lp:ubuntu/linux-ti-omap4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Finally, Will Deacon figured out a workaround for the issue on arm code, see
below upstream commit:
(CONFIG_PL310_ERRATA_769419 is needed for the fix)
commit 11ed0ba1754841316d4095478944300acf19acc3
Author: Will Deacon will.dea...@arm.com
Date: Mon Nov 14 17:24:58 2011 +0100
ARM: 7161/1: errata: no automatic store buffer drain
This patch implements a workaround for PL310 erratum 769419. On
revisions of the PL310 prior to r3p2, the Store Buffer does not
automatically drain. This can cause normal, non-cacheable writes to be
retained when the memory system is idle, leading to suboptimal I/O
performance for drivers using coherent DMA.
This patch adds an optional wmb() call to the cpu_idle loop. On systems
with an outer cache, this causes an explicit flush of the store buffer.
Cc: sta...@vger.kernel.org
Acked-by: Catalin Marinas catalin.mari...@arm.com
Tested-by: Marc Zyngier marc.zyng...@arm.com
Signed-off-by: Will Deacon will.dea...@arm.com
Signed-off-by: Russell King rmk+ker...@arm.linux.org.uk
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 44789ef..83aa746 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1352,6 +1352,18 @@ config ARM_ERRATA_764369
relevant cache maintenance functions and sets a specific bit
in the diagnostic control register of the SCU.
+config PL310_ERRATA_769419
+ bool PL310 errata: no automatic Store Buffer drain
+ depends on CACHE_L2X0
+ help
+ On revisions of the PL310 prior to r3p2, the Store Buffer does
+ not automatically drain. This can cause normal, non-cacheable
+ writes to be retained when the memory system is idle, leading
+ to suboptimal I/O performance for drivers using coherent DMA.
+ This option adds a write barrier to the cpu_idle loop so that,
+ on systems with an outer cache, the store buffer is drained
+ explicitly.
+
endmenu
source arch/arm/common/Kconfig
diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c
index 75316f0..3d0c6fb 100644
--- a/arch/arm/kernel/process.c
+++ b/arch/arm/kernel/process.c
@@ -192,6 +192,9 @@ void cpu_idle(void)
#endif
local_irq_disable();
+#ifdef CONFIG_PL310_ERRATA_769419
+ wmb();
+#endif
if (hlt_counter) {
local_irq_enable();
cpu_relax();
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/709245
Title:
ARM SMP scheduler performance bug
To manage notifications about this bug go to:
https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: oem-priority/natty Status: Fix Committed = Fix Released ** Changed in: oem-priority/maverick Status: Fix Committed = Fix Released ** Changed in: oem-priority Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
This bug was fixed in the package linux-ti-omap4 - 2.6.38-1209.16 --- linux-ti-omap4 (2.6.38-1209.16) natty-proposed; urgency=low * Release tracking bug - LP: #862554 [ Ming Lei ] * SAUCE: usb: ehci: make HC see up-to-date qh/qtd descriptor ASAP - LP: #709245 [ Upstream Kernel Changes ] * cifs: fix possible memory corruption in CIFSFindNext, CVE-2011-3191 - LP: #834135 - CVE-2011-3191 * befs: Validate length of long symbolic links, CVE-2011-2928 - LP: #834124 - CVE-2011-2928 * gro: Only reset frag0 when skb can be pulled, CVE-2011-2723 - LP: #844371 - CVE-2011-2723 * Validate size of EFI GUID partition entries, CVE-2011-1776 - LP: #844365 - CVE-2011-1776 * inet_diag: fix inet_diag_bc_audit(), CVE-2011-2213 - LP: #838421 - CVE-2011-2213 * si4713-i2c: avoid potential buffer overflow on si4713, CVE-2011-2700 - LP: #844370 - CVE-2011-2700 * Bluetooth: Prevent buffer overflow in l2cap config request, CVE-2011-2497 - LP: #838423 - CVE-2011-2497 * crypto: Move md5_transform to lib/md5.c, CVE-2011-3188 - LP: #834129 - CVE-2011-3188 * net: Compute protocol sequence numbers and fragment IDs using MD5, CVE-2011-3188 - LP: #834129 - CVE-2011-3188 * ext4: Fix max file size and logical block counting of extent format file, CVE-2011-2695 - LP: #819574 - CVE-2011-2695 -- Paolo Pisati paolo.pis...@canonical.com Fri, 30 Sep 2011 12:12:00 +0200 ** Changed in: linux-ti-omap4 (Ubuntu Natty) Status: Fix Committed = Fix Released ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-2695 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Tested kernel 2.6.38-1209.16. Much improvement to write speeds. Test used: dd bs=4M count=100 if=/dev/zero of=test.img Before: 419430400 bytes (419 MB) copied, 50.2334 s, 8.3 MB/s After: 419430400 bytes (419 MB) copied, 18.7676 s, 22.3 MB/s ** Tags removed: verification-needed-natty ** Tags added: verification-done-natty -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Natty 2.6.38-1209.16 release is now available in -proposed. This bug is awaiting verification that this kernel for Natty in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-natty' to 'verification-done-natty'. If verification is not done by one week from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-natty -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Branch linked: lp:ubuntu/natty-proposed/linux-ti-omap4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
This bug was fixed in the package linux-ti-omap4 - 2.6.35-903.25 --- linux-ti-omap4 (2.6.35-903.25) maverick-proposed; urgency=low [ Ming Lei ] * SAUCE: usb: ehci: make HC see up-to-date qh/qtd descriptor ASAP - LP: #709245 [ Upstream Kernel Changes ] * cifs: fix possible memory corruption in CIFSFindNext, CVE-2011-3191 - LP: #834135 - CVE-2011-3191 * befs: Validate length of long symbolic links, CVE-2011-2928 - LP: #834124 - CVE-2011-2928 * gro: Only reset frag0 when skb can be pulled, CVE-2011-2723 - LP: #844371 - CVE-2011-2723 * Validate size of EFI GUID partition entries, CVE-2011-1776 - LP: #844365 - CVE-2011-1776 * inet_diag: fix inet_diag_bc_audit(), CVE-2011-2213 - LP: #838421 - CVE-2011-2213 * si4713-i2c: avoid potential buffer overflow on si4713, CVE-2011-2700 - LP: #844370 - CVE-2011-2700 * Bluetooth: Prevent buffer overflow in l2cap config request, CVE-2011-2497 - LP: #838423 - CVE-2011-2497 * core: Fix memory leak/corruption on VLAN GRO_DROP, CVE-2011-1576 - LP: #844361 - CVE-2011-1576 * crypto: Move md5_transform to lib/md5.c, CVE-2011-3188 - LP: #834129 - CVE-2011-3188 * net: Compute protocol sequence numbers and fragment IDs using MD5, CVE-2011-3188 - LP: #834129 - CVE-2011-3188 -- Paolo Pisati paolo.pis...@canonical.com Tue, 20 Sep 2011 19:12:28 +0200 ** Changed in: linux-ti-omap4 (Ubuntu Maverick) Status: Fix Committed = Fix Released ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-1576 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-1776 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-2213 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-2497 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-2700 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-2723 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-2928 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-3188 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2011-3191 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: linaro-ubuntu Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: linaro-ubuntu Status: Confirmed = Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Tested the Maverick kernel in proposed. While I am not seeing much improvement, the kernel at least doesn't break other things. The following is the results of testing on a PATA drive in an external enclosure: -- BEFORE -- Linux panda5 2.6.35-903-omap4 #24-Ubuntu SMP PREEMPT Thu Sep 1 20:37:40 UTC 2011 armv7l GNU/Linux /dev/sda: Timing cached reads: 388 MB in 2.00 seconds = 193.66 MB/sec Timing buffered disk reads: 54 MB in 3.10 seconds = 17.45 MB/sec -- AFTER -- Linux panda5 2.6.35-903-omap4 #25-Ubuntu SMP PREEMPT Tue Sep 20 21:02:11 UTC 2011 armv7l GNU/Linux /dev/sda: Timing cached reads: 402 MB in 2.00 seconds = 200.71 MB/sec Timing buffered disk reads: 70 MB in 3.07 seconds = 22.83 MB/sec ** Tags removed: verification-needed-maverick ** Tags added: verification-done-maverick -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: linux-linaro Status: Confirmed = Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: oem-priority Status: New = Fix Committed ** Also affects: oem-priority/maverick Importance: Undecided Status: New ** Also affects: oem-priority/natty Importance: Undecided Status: New ** Also affects: oem-priority/oneiric Importance: Undecided Status: New ** Changed in: oem-priority/maverick Status: New = Fix Committed ** Changed in: oem-priority/natty Status: New = Fix Committed ** Changed in: oem-priority/oneiric Status: New = Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Branch linked: lp:ubuntu/maverick-proposed/linux-ti-omap4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
This bug is awaiting verification that the kernel for Maverick in -proposed (2.6.35-903.25) solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-maverick' to 'verification-done-maverick'. If verification is not done by one week from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-maverick -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: linux-ti-omap4 (Ubuntu Maverick) Status: Confirmed = Fix Committed ** Changed in: linux-ti-omap4 (Ubuntu Natty) Status: Confirmed = Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
This bug was fixed in the package linux-ti-omap4 - 3.0.0-1204.9 --- linux-ti-omap4 (3.0.0-1204.9) oneiric; urgency=low [ Leann Ogasawara ] * [Config] Update CONFIG_EFI_VARS enforcer check [ Paolo Pisati ] * Revert [Config] Temporarily turn off audio support on omap4. * [Config] update configs after BSP update * Rebased against Ubuntu-3.0.0-9.15 * TI BSP update: kernel-tilt/tilt-3.0-nodspvideo1 @ a6871e8f1f3c906b37884fb35837546b7c1dc15e [ Ricardo Salveti de Araujo ] * [Config] Enabling DRM_OMAP as built-in [ Upstream Kernel Changes ] * Revert usb: ehci: fix update qtd-token in qh_append_tds * usb: ehci: make HC see up-to-date qh/qtd descriptor ASAP - LP: #709245 -- Paolo Pisati paolo.pis...@canonical.com Thu, 01 Sep 2011 11:56:08 +0200 ** Changed in: linux-ti-omap4 (Ubuntu Oneiric) Status: Confirmed = Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Branch linked: lp:ubuntu/linux-ti-omap4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Tags added: rls-mgr-o-tracking -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: linaro-ubuntu Assignee: (unassigned) = John Rigby (jcrigby) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
new discussion: https://patchwork.kernel.org/patch/1113332/ ** Changed in: linux-ti-omap4 (Ubuntu Oneiric) Assignee: (unassigned) = Ming Lei (tom-leiming) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
discussion ongoing: https://patchwork.kernel.org/patch/1103812/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Ming-Lei it sounds like a really good explanation. But when I test it, I don't see any real difference on USB memory stick before and after. Maybe my test is broken somehow? 8GB no-name USB stick Before the patch -- root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.9735 s, 1.9 MB/s root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.3798 s, 1.9 MB/s root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.3694 s, 1.9 MB/s After the patch -- root@linaro:~# mount /dev/sda1 /media/ root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.6143 s, 1.9 MB/s root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.6175 s, 1.9 MB/s root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.5983 s, 1.9 MB/s I like the idea the delay is coming from L2 cache not being evicted until something else forces it a lot though. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
works here on an external usb hdd: oneiric/ti-omap4: flag@omap:~$ sudo hdparm -t /dev/sda1 /dev/sda1: Timing buffered disk reads: 6 MB in 3.53 seconds = 1.70 MB/sec oneiric/ti-omap4 + Ming's patch: flag@omap:~$ sudo hdparm -t /dev/sda1 /dev/sda1: Timing buffered disk reads: 60 MB in 3.09 seconds = 19.40 MB/sec -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 709245] Re: ARM SMP scheduler performance bug
Hi warmcat, On Sun, Aug 28, 2011 at 4:32 PM, warmcat 709...@bugs.launchpad.net wrote: Ming-Lei it sounds like a really good explanation. But when I test it, I don't see any real difference on USB memory stick before and after. Maybe my test is broken somehow? Could you check if you have applied patch, built kernel and update correctly? What is your board and version? and could you post out your kernel config? 8GB no-name USB stick Before the patch -- root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.9735 s, 1.9 MB/s root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.3798 s, 1.9 MB/s root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.3694 s, 1.9 MB/s After the patch -- root@linaro:~# mount /dev/sda1 /media/ root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.6143 s, 1.9 MB/s root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.6175 s, 1.9 MB/s root@linaro:~# dd if=/dev/urandom of=/media/dump bs=40960 count=1000 1000+0 records in 1000+0 records out 4096 bytes (41 MB) copied, 21.5983 s, 1.9 MB/s I think the patch should have some changes on the test result. I like the idea the delay is coming from L2 cache not being evicted until something else forces it a lot though. The delay is coming from the delayed ehci irq or lost ehci irq, and which is coming from the not up-to-date qtd descriptor. thanks, -- Ming Lei -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Hi Ming-Lei - What I like about your explanation is it allows cache status to be the communication path for network traffic to impact the issue, ie network activity finally spills the cache that does update the qtd descriptor and then we get less deadtime. That sounds very good to me. I'm using this tree http://git.linaro.org/gitweb?p=people/andygreen/kernel- tilt.git;a=patch;h=7021a2ea35d0b0f96e6c5a753312c26466919efd Your patch http://git.linaro.org/gitweb?p=people/andygreen/kernel- tilt.git;a=shortlog;h=refs/heads/tilt-linux-linaro-3.0 I realized I do have a USB HDD here. I will try that as well in case the USB flash drive is the limiting factor. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
Ah! /dev/urandom is the limiting factor ^^ USB HDD is also stuck at 1.9Mbps with the same test. Same test with of=/dev/null is only getting 2.0Mbps. With the USB HDD and if=/dev/zero instead, and 10x count root@linaro:~# dd if=/dev/zero of=/media/dump bs=40960 count=1 1+0 records in 1+0 records out 40960 bytes (410 MB) copied, 16.5168 s, 24.8 MB/s root@linaro:~# dd if=/dev/zero of=/media/dump bs=40960 count=1 1+0 records in 1+0 records out 40960 bytes (410 MB) copied, 18.4294 s, 22.2 MB/s With the patch out, this takes an insanely long time. Against the USB stick, it rises to 7.1MBps with the patch in vs 3.5MBps with it out. So nice job! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Tags added: patch ** Changed in: linaro-ubuntu Milestone: 11.08 = 11.09 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Description changed: Original Bug name: panda: USB disk IO slow + + This bug effects ARM Cortex A9 cores, snowball, nvidia, OMAP 4, and + other Cortex A9 processors. Problem is in Fedora ARM builds also so not + limited to Ubuntu. My Panda's USB seems to be significantly slower than a Beagle C4. hdparm shows buffered reads as ~12MB/s on the Panda, and about ~20-25MB/s on a Beagle C4 from the same external Lacie USB disk. Kernel is 2.6.37-1002-linaro-omap Disk shows as: [5.170440] scsi 0:0:0:0: Direct-Access LaCied2 quadra PQ: 0 ANSI: 4 [5.172546] sd 0:0:0:0: Attached scsi generic sg0 type 0 [5.175415] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB) The board is otherwise idle during the test. Doing perf_2.6.37-12 record -a dd if=/dev/sda of=/dev/null bs=4096 count=10 shows : 81.41% swapper [kernel.kallsyms] [k] default_idle 6.33% dd [kernel.kallsyms] [k] __copy_to_user 0.94% swapper [kernel.kallsyms] [k] cpu_idle 0.51% dd [kernel.kallsyms] [k] __make_request 0.51% perf_2.6.37-12 [kernel.kallsyms] [k] __copy_from_user which suggests it's not CPU constrained. Dave -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
I filed a bug for adding dbgsyms to the kernel so we can use systemtap easily in similar debugging scenarios https://bugs.launchpad.net/ubuntu/+source/linux-ti-omap4/+bug/832606 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Also affects: oem-priority Importance: Undecided Status: New ** Changed in: oem-priority Importance: Undecided = High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
comparing the boot logs of an smp and a nosmp kernel (see attached
file), what struck me was an error message about the smp_twd clock:
CPU: Testing write buffer coherency: ok
-twd_timer_setup: no clock found
-Calibrating local timer... 492.00MHz.
L310 cache controller enabled
l2x0: 16 ways, CACHE_ID 0x41c4, AUX_CTRL 0x5e47, Cache size: 1048576 B
looking closer at the code, i found that:
smp_twd.c::twd_timer_setup():
...
if (twd_clk == NULL) {
twd_clk = clk_get_sys(smp_twd, NULL);
if (IS_ERR_OR_NULL(twd_clk))
pr_warn(%s: no clock found\n, __func__);
}
...
and clk_get_sys() prototype is:
struct clk *clk_get_sys(const char *dev_id, const char *con_id)
while clocks are defined as:
clock44xx_data.c in struct omap_clk omap44xx_clks[]:
...
CLK(NULL, smp_twd, smp_twd, CK_44XX)
...
#define CLK(dev, con, ck, cp) \
{ \
.cpu = cp, \
.lk = { \
.dev_id = dev, \
.con_id = con, \
.clk = ck, \
}, \
}
so the parameters in clk_get_sys() are inverted:
--- a/arch/arm/kernel/smp_twd.c
+++ b/arch/arm/kernel/smp_twd.c
@@ -175,7 +175,8 @@ static void __cpuinit twd_calibrate_rate(void)
*/
void __cpuinit twd_timer_setup(struct clock_event_device *clk)
{
if (twd_clk == NULL) {
- twd_clk = clk_get_sys(smp_twd, NULL);
+ twd_clk = clk_get_sys(NULL, smp_twd);
if (IS_ERR_OR_NULL(twd_clk))
pr_warn(%s: no clock found\n, __func__);
this will make the error go away, but unfortunately won't fix the i/o
problem.
hdparm -t /dev/sda1
smp:
/dev/sda1:
Timing buffered disk reads: 6 MB in 4.73 seconds = 1.27 MB/sec
nosmp:
/dev/sda1:
Timing buffered disk reads: 44 MB in 3.08 seconds = 14.27 MB/sec
** Attachment added: dmesg comparison: smp vs nosmp
https://bugs.launchpad.net/ubuntu/+source/linux-ti-omap4/+bug/709245/+attachment/2286211/+files/3.0.0-1202-omap4.smpvsnosmp
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/709245
Title:
ARM SMP scheduler performance bug
To manage notifications about this bug go to:
https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
I tried to gather more data on this and made several runs of hdparm -Tt for the good and bad case: - For the good case (nosmp) - cached reads avg. 184.61MB/sec (MIN=181.02 MAX=186.30 STDDEV=1.62) - buffered reads avg. 3.64MB/sec (MIN=3.64 MAX=3.65 STDDEV=0) - For the bad case - cached reads avg. 62.00MB/sec (MIN=21.98 MAX=111.55 STDDEV=27.97) - buffered reads avg. 1.35MB/sec (MIN=1.09 MAX=1.86 STDDEV=0.20) So not only the SMP case is slower, there is also a high variation in the numbers. Whatever happens, it is not a linear slowdown. When trying various options maxcpus=1 has the same effect as nosmp (disabling a core later does not improve things), there was one option (which right now slips my memory) causing a higher rate of timer interrupts which lead to a even worse buffered reads performance (as was disabling the irqbalance daemon). Using nohlt seemed to have no effect. When looking at the interrupts, the main difference seemed to be the use of gp_timer when booted with nosmp or maxcpus=1 and LOCal timers in the other case. Booting with 2 CPUs and disabling one did not change that. I cannot recall whether the IPIs stopped being incremented completely or only on the other CPU. But LOC definitely was still used for CPU#0. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
the thing about the GPTimer is interesting... i believe the kernel has CONFIG_NOHZ defined, that might be interesting to try without that (e.g. with HZ). in fact it would be nice to make a custom kernel that uses GPTimer instead of local timer, but that might be a little more tricky (even though we had that for very early OMAP chips since local timer weren't functional). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: linux-ti-omap4 (Ubuntu Natty) Status: New = Confirmed ** Changed in: linux-ti-omap4 (Ubuntu Maverick) Status: New = Confirmed ** Changed in: linux-ti-omap4 (Ubuntu Maverick) Importance: Undecided = High ** Changed in: linux-ti-omap4 (Ubuntu Natty) Importance: Undecided = High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 709245] Re: ARM SMP scheduler performance bug
** Changed in: linaro-ubuntu Milestone: 11.07 = 11.08 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/709245 Title: ARM SMP scheduler performance bug To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-ubuntu/+bug/709245/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
