Hi Brock, > On Wed, 10 Sep 2008, Erwin, Brock A wrote: > > However, I am trying to understand why I am not getting 100% network > > saturation on a single client for writing to one very large file (i.e. > > one single stream of data). Instead, I find that I am getting less than > > 25% utilization of the network interface that the client is connected > > to. After noticing this, I proceeded to locate the source of the > > bottleneck. I discovered that processor, disk, and network utilization > > on the OSDs and the MDS as well all remained below 25%. > > > > Then I decided to do parallel writes to different files on the client (I > > wrote 20 different large files at the same time). In this case, the > > network utilization on the client jumped to 100%. > > > > Thus, I am wondering why a single stream of data cannot utilize the > > hardware fully. I am thinking this maybe has something to do with the > > way the kernel buffers data before it is sent over a socket. Maybe > > larger amounts of data need to be sent at any given time, or maybe the > > ceph client needs to buffer its data until it gets to a particular size > > before it writes to the open socket. However, I am just thinking aloud > > at this point given the fact that I do not completely understand how > > client communicates its data to the OSDs.
I've pushed out a new asynchronous writepages() implementation that should fix any problems pushing out dirty data on a single file. The overall performance is now primarily determined by the writeback throttling done by the kernel's backing_dev_info. This fix also allowed me to adjust the writeback so that pages aren't unlocked until data is fully committed to disk on the OSDs (it was cheating a bit before to get decent writeback performance). In my (limited) testing, things are going much faster. I'd be curious how things perform on your system, when you get the chance. Thanks- sage ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Ceph-devel mailing list Ceph-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ceph-devel
