On Thu, Sep 15, 2011 at 03:17:48PM -0500, Mike Christie wrote: > On 09/15/2011 05:26 AM, Heinrich Langos wrote: > > Hi Mike, > > > > On Thu, Sep 08, 2011 at 09:25:29PM -0500, Mike Christie wrote: > >> On 09/08/2011 09:23 PM, Mike Christie wrote: > >>> On 09/08/2011 04:36 PM, Mike Christie wrote: > >>>> On 09/08/2011 02:06 AM, Heinrich Langos wrote: > >>>>> > >>>>> This is raw "dd" throughput for reading ~30GB from an iSCSI storage > >>>>> via a dedicated 1GB Ethernet link. > >>>>> > >>>>> 2.6.32 : 102 MB/s > >>>>> 2.6.38 : 100 MB/s > >>>>> 2.6.39 : 44 MB/s > >>>>> 3.0.1 : 43 MB/s > >>>>> > >>>> > >>>> I can replicate this now. For some reason I only see it with 1 gig. I > >>>> think my 10 gig setups that I have been testing with are limited by > >>>> something else. > >>>> > >>>> Doing git bisect now to track down the change that caused the problem. > >>>> > >>> > >>> I did not find anything really major in the iscsi code. But I did notice > >>> that if I just disable iptables throughput goes from about 5 MB/s back > >>> up to 85 MB/s. > >>> > >>> If you disable iptables do you see something similar. > >>> > >> > >> I also noticed that in 2.6.38 throughput would almost immediately start > >> at 80-90 MB/s, but with 2.6.39 it takes a while (maybe 10 seconds > >> sometimes) to ramp up. > > > > Did you find the reason of the performance drop? Is there anything I can do > > to help? I'll have to get that new box into production soon and since I am > > Did you confirm that stopping iptables also solves the problem for you? > Was waiting to hear back from you to make sure we are working on the > same bug.
Hello Mike, iptables doesn't seem to be involved... The effect I observed (going from 43MB/s to 63MB/s was only a short term thing. The full test still shows the same low performance as before. 3.0.1: ... 29843+0 records in 29842+0 records out 31291604992 bytes (31 GB) copied, 728.783 s, 42.9 MB/s 29917+0 records in 29916+0 records out 31369199616 bytes (31 GB) copied, 730.761 s, 42.9 MB/s 29993+0 records in 29992+0 records out 31448891392 bytes (31 GB) copied, 732.776 s, 42.9 MB/s 30000+0 records in 30000+0 records out 31457280000 bytes (31 GB) copied, 737.593 s, 42.6 MB/s 30000+0 records in 30000+0 records out 31457280000 bytes (31 GB) copied, 737.593 s, 42.6 MB/s [1]+ Done dd if=/dev/disk/by-path/ip-172.26.0.100\:3260-iscsi-iqn.2001-05.com.equallogic\:0-8a0906-f11e2e306-4c60013c7064e675-testme-lun-0 of=/dev/null bs=1024k count=30000 -bash: kill: (2159) - No such process root@janus01:~# lsmod | grep tab root@janus01:~# Then I ran "iptables -L -n" to make sure those modules are loaded root@janus01:~# lsmod | grep tab iptable_filter 12536 0 ip_tables 21818 1 iptable_filter x_tables 18886 2 iptable_filter,ip_tables I repeated the test and got pretty much the same throughput. Sometimes the beginning seems faster, sometimes it seems slower. Sometimes the leveling off happens after a couple of seconds, sometimes it takes up to 10 GB to of data to to reach a "stable" throughput level. (Though when measuring 2.6.38 I had a slow slope going down to 99 MB/s average for the full 30GB even when after 10GB the average still was around 106MB/s.) What I guess is that caching has to be observed with care when running multiple tests with the same kernel. After all the machine I am testing with has 48GB RAM and I am not running anything else on the machine. What it boils down to is no significant effect of iptables ! cheers -henrik -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
