Re: Over one million IOPS using software iSCSI and 10 Gbit Ethernet, 1.25 million IOPS update
On Fri, Feb 05, 2010 at 02:10:32PM +0300, Vladislav Bolkhovitin wrote: Pasi Kärkkäinen, on 01/28/2010 03:36 PM wrote: Hello list, Please check these news items: http://blog.fosketts.net/2010/01/14/microsoft-intel-push-million-iscsi-iops/ http://communities.intel.com/community/openportit/server/blog/2010/01/19/100-iops-with-iscsi--thats-not-a-typo http://www.infostor.com/index/blogs_new/dave_simpson_storage/blogs/infostor/dave_simpon_storage/post987_37501094375591341.html 1,030,000 IOPS over a single 10 Gb Ethernet link Specifically, Intel and Microsoft clocked 1,030,000 IOPS (with 512-byte blocks), and more than 2,250MBps with large block sizes (16KB to 256KB) using the Iometer benchmark So.. who wants to beat that using Linux + open-iscsi? :) I personally, don't like such tests and don't trust them at all. They are pure marketing. The only goal of them is to create impression that X (Microsoft and Windows in this case) is a super-puper ahead of the world. I've seen on the Web a good article about usual tricks used by vendors to cheat benchmarks to get good marketing material, but, unfortunately, can't find link on it at the moment. The problem is that you can't say from such tests if X will also ahead of the world on real life usages, because such tests always heavily optimized for particular used benchmarks and such optimizations almost always hurt real life cases. And you hardly find descriptions of those optimizations as well as a scientific description of the tests themself. The results published practically only in marketing documents. Anyway, as far as I can see Linux supports all the used hardware as well as all advance performance modes of it, so if one repeats this test in the same setup, he/she should get not worse results. For me personally it was funny to see how MS presents in the WinHEC presentation (http://download.microsoft.com/download/5/E/6/5E66B27B-988B-4F50-AF3A-C2FF1E62180F/COR-T586_WH08.pptx) that they have 1.1GB/s from 4 connections. In the beginning of 2008 I saw a *single* dd pushing data on that rate over a *single* connection from Linux initiator to iSCSI-SCST target using regular Myricom hardware without any special acceleration. I didn't know how proud I must have been for Linux :). It seems they've described the setup here: http://communities.intel.com/community/wired/blog/2010/04/20/1-million-iop-article-explained And today they seem to have a demo which produces 1.3 million IOPS! 1 Million IOPS? How about 1.25 Million!: http://communities.intel.com/community/wired/blog/2010/04/22/1-million-iops-how-about-125-million -- Pasi -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: Centos 5.4 bnx2i iscsi woes
Ben, Mike, Is there a RedHat bug covered by this? I've searched and the closest I see is https://bugzilla.redhat.com/show_bug.cgi?id=578005 but that seems to indicate that a service network restart would be a workaround. I'm not seeing that work for me. Thank you, Tarun On Mon, May 24, 2010 at 3:16 PM, Benjamin Li be...@broadcom.com wrote: Hi Mike, I don't have a version of brcm_iscsiuio which I have readily avaliable for 5.5.z which has the fix you are looking for without adding new features. I will need to create a version of brcm_iscsiuio with the iscsid/brcm_iscsiuio backported for 5.5.z. When does this need to be accomplished? Thanks again. -Ben On Mon, 2010-05-24 at 14:05 -0700, Mike Christie wrote: On 05/21/2010 07:35 PM, Tarun Reddy wrote: Mike, So is there any update to this? It seems to fix the problem in RHEL 6 testing. http://people.redhat.com/mchristi/iscsi/rhel6.0/iscsi-initiator-utils/ (I think the rhel6 init scripts will not work with RHEL5 perfectly). Ben, did you have a brcm release you wanted me to use for a 5.5.z release (the 5.5.z should not have new features as you know so I could not use the brcm release I used in the RHEL 6 realease). -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
Re: mc/s - not yet in open-iscsi?
On Fri, 2010-06-11 at 23:17 +, Raj wrote: Nicholas A. Bellinger n...@... writes: Btw, just for those following along, here is what MC/S and ERL=2 when used in combination (yes, they are complementary) really do: http://linux-iscsi.org/builds/user/nab/Inter.vs.OuterNexus.Multiplexing.pdf Also, I should mention in all fairness that my team was the first to implement both a Target and Initiator capable of MC/S and ErrorRecoveryLevel=2 running on Linux, and the first target capable of running MC/S from multiple initiator implementations. But the end result is what? open-iSCSI still doesn't have the MC/S even though it is useful? So without going into a multi-year history lesson as to why MC/S is not currently supported in Open-iSCSI, what it boils down to is this: MC/S (or InterNexus multiplexing) is very useful for scaling iSCSI sessions across multiple cores and network ports with minimal overhead compared with traditional link layer bonding, not to mention that individual iSCSI TCP connections can be independently going across multiple different subnets and backbone providers. It also provides less overhead and fewer kernel threads context switches than host OS dependent MPIO and allows for the ability to scale the number of LUNs and bandwith of individual LUNs across groups of TCP TX/RX kernel threads (for a single iSCSI session (I_T Nexus). MC/S is complementary to legacy symmetric and new SPC-4 ALUA MPIO, and can be used together following with RFC-3720. MC/S functions independently of the negotiated ErrorRecoveryLevel, but if one connection fails in ERL=0 then all of the connections must be restarted with session reinstatement (restart I_T Nexus). The Core-iSCSI MC/S Initiator has existed out of tree for Linux v2.6 for about 5 years now, and has been ported to a number of different embedded devices and server class architectures.MC/S support is included in every install of MSFT including their Hyper-V offering (and is used for the 1 million IOPs windows iSCSI initiator). MC/S is also now becoming a hard requirement for hypervisor level iSCSI initiators (either KVM or based on Linux) in order to scale bandwith across a number of CPUs cores on small pipes (4x 1 Gb/sec) and on bigger pipes (10 Gb/sec) with traditional iSCSI. MC/S is available from select iSCSI target implementations, including all versions of the open source LIO-Target stack and TCM fabric module (v2.x, v3.x and v4.x-rc) and from certain Netapp hardware and probably other big array vendors who compete with Netapp at this point. In terms of implementing MC/S support into Open-iSCSI, the initiator side fast path for MC/S is straight forward enough, eg: once the CmdSN window has opened, assign the incoming struct scsi_cmnd alligence to one active iSCSI connection, and queue up to that connections TX thread. I will say the user - kernel netlink design for the control path does make this a little more interesting, becase also with MC/S iSCSI connections can be brought up and down on the fly at any time, which also adds a certain level of complexity in the shutdown path for an iSCSI Initiator. Anyways, this is something that I have been considering for a while, and I think it will eventually happen (either by myself or someone else) who is doing bringup against existing stable LIO-Target MC/S logic. Best, --nab -- You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
