----- Forwarded message from Phil Schwan <[EMAIL PROTECTED]> ----- From: Phil Schwan <[EMAIL PROTECTED]> Date: Thu, 03 Feb 2005 19:11:28 -0500 To: Christopher Alexander Stein <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Subject: Re: [Lustre-discuss] portals User-Agent: Microsoft-Entourage/11.1.0.040913
On 2/3/2005 18:07, Christopher Alexander Stein wrote: > >> From some lustre documentation: "Lustre uses the Portals software > originally developed at Sandia Labs to provide a network > abstraction layer that simplifies using Lustre across multiple > types of networks." > > Isn't this network abstraction exactly why IP, TCP, and UDP > exist? Why portals? Yes and no. TCP is a useful abstraction for a lot of purposes, but it's far from perfect for a high-performance cluster file system. For organizations that invest in the high-end cluster networking gear -- Quadrics Elan, InfiniBand, SCI, Myrinet etc. -- we can do a _lot_ better than TCP. We can provide 300% or 400% better performance over native Infiniband than we can over TCP over IB. The same for Quadrics. And because these interconnects support RDMA for zero-copy transmit and receive, we do it with a fraction of the CPU overhead that TCP imposes on us. The next question is often "but what about TCP offload cards?" We experimented with a few cards and found them severely lacking. They didn't go far enough -- or change enough of the kernel APIs -- to eliminate the really costly parts of the protocol, and the memory copies. In short: to reach our performance targets, we could not always accept the costs and limitations of TCP. We needed a different abstraction layer that could be efficient on a variety of specialized hardware. Because we also want to support commodity interconnects like Ethernet, we of course implement TCP/IP as one such Portals backend. -Phil _______________________________________________ Lustre-discuss mailing list [EMAIL PROTECTED] https://lists.clusterfs.com/mailman/listinfo/lustre-discuss ----- End forwarded message ----- -- Eugen* Leitl <a href="http://leitl.org";>leitl</a> ______________________________________________________________ ICBM: 48.07078, 11.61144 http://www.leitl.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE http://moleculardevices.org http://nanomachines.net ------- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
pgpO8uXuRrWO1.pgp
Description: PGP signature
