Hi Shirley - I had a chance to try with the tiny blocksizes but I'm afraid the results aren't useful to estimate max throughput. The server I am using runs out of CPU at about 33,600 IOPS for small I/Os (<=4KB), so with 2000 byte reads, all I can get is about 65MB/sec. (I get 33MB/s with 1KB, 120MB/s with 4KB, etc). And recall with NFS-default 32KB reads I get 450MB/s. All these limits are due to this server's CPU at 100%. Time to find a bigger server!
The good news is, performance is nice and flat right up until the server hits the CPU wall. In fact, the more directio threads I run in parallel, the lower the client overhead. With 50 threads issuing reads, I see as little as 0.5 interrupts per I/O! Sorry I couldn't push more throughput using only small reads. I could trunk the I/O to multiple servers, but I assume you're only interested in single- stream results. Tom. At 11:11 PM 5/10/2006, Shirley Ma wrote: >"Talpey, Thomas" <[EMAIL PROTECTED]> wrote on 05/10/2006 03:10:57 PM: >> Sure, but I wonder why it's interesting. Nobody ever uses NFS in such >> small blocksizes, and 2044 bytes would mean, say, 1800 bytes of payload. >> What data are you looking for, throughput and overhead? Direct RDMA, >> or inline? >> >> Tom. > >Throughput. I am wondering how much room IPoIB performance (throughput) can >go. > >Thanks >Shirley Ma >IBM Linux Technology Center >15300 SW Koll Parkway >Beaverton, OR 97006-6063 >Phone(Fax): (503) 578-7638 _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
