And finally the webrev (no review board — haven’t set up rbt yet) — http://cr.illumos.org/~webrev/gdamore/sfxge/
Thanks in advance for review feedback! - Garrett On Tue, Mar 22, 2016 at 11:20 AM, Garrett D'Amore <garr...@damore.org> wrote: > I’m now posting the sfxge driver for code review. I don’t have > webrev/review board (yet), but here’s a pull-request that you can comment > on, or merge into a private tree (yes I know illumos doesn’t merge PRs.): > > https://github.com/illumos/illumos-gate/pull/16 > > So this is the sfxge driver which supports modern Solarflare NICs, > including 10 and 40 GbE. I’ve only tested 10GbE using the > Solarflare Flareon Ultra 7000 Series. > > My testing is that this driver, when tuned properly, gives significantly > lower latency than the ixgbe driver. > > I’ve not benchmarked for throughput, or CPU utilization. In my world > latency is king. I have however put rather severe load in terms of packets > per second on the NIC… I’ve run stress runs with TCP where I push around 1M > pkts/sec (TCP messages at 200 bytes per message with Nagle disabled) across > this NIC; at that rate the TCP stack and the soft rings from GLDv3 become > concerns, although I ran out of test client capacity before I saturated the > capabilities of this NIC. (My testing involved an in-kernel TCP proxy that > I developed for our SDN at Lucera.) Round trip latencies with this driver > and combination are typically several usec (or more!) better than I could > achieve with ixgbe. > > The driver lacks support for Crossbow virtualization, which means it uses > soft rings. Even with this deficiency, it out performs the ixgbe NIC in my > configuration. (I will work on addressing Crossbow support with Solarflare > in a future update.) > > SFC9300 NICs are not enabled, but this can be done with a compile time > option; if anyone has such NICs I’d be grateful for help testing. SFC 9200 > work fine. (I’m not sure SFC 9300 are actually available for purchase yet.) > > The older SFC 4000 family is *not* supported — the support for this was > present in earlier versions of the code, but Solarflare have removed the > code from their common code base, and so such NICs should not be used. > > This driver was contributed directly for illumos integration by > Solarflare, and their engineering staff have been enormously helpful in > this effort. Please join me in thanking Solarflare. > > The driver has some illumos-specific improvements (mostly cleaning up > unused Solaris 10 support for legacy NDD, some other minor fixes and > improvements, man page, packaging, etc.) that were performed by me, as well > as lint cleanups. Note that the driver was already in very good shape, and > is filled with rich dtrace probes, etc. > > If you want to see the gory details of the changes I made, have a look at > my github branch here: > > https://github.com/gdamore/illumos-core/tree/sfxge > > We are using this driver and NIC at Lucera. That said, there may be sharp > edges still, and I’ve not yet run the entire thing through NICDRV. (Mostly > because setting up NICDRV is a major PITA.) > > Again, if you can help with testing (especially with 40GbE!), I’d be > grateful. > > And review feedback is most welcome as well! > > - Garrett > ------------------------------------------- illumos-discuss Archives: https://www.listbox.com/member/archive/182180/=now RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be Modify Your Subscription: https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4 Powered by Listbox: http://www.listbox.com
