On Wed, Jun 22, 2005 at 11:07:51AM +0200, Eli Billauer wrote: > Muli Ben-Yehuda wrote: > > >>mknod mypipe p > >>mbuffer -i mypipe -o /fatfs/output-file & > >>./writefat mypipe > listfile
> >Until 2.6.mumble, pipes only used a single page in memory. Since > >2.6.mumble we're using up to 16 pages and flipping between consumer > >and producer, which should give much better pipe utilization for large > >writes. > Note that mbuffer is the RAM FIFO, and it was empty all the time (as one > could expect). Since mbuffer never blocked, I don't think it matters how > good the pipe between them is. This is why I found it weird that I got > delays at all, using a RAM FIFO. Where can I find the sourcve for mbuffer? > >- use write, not fwrite!!! > >- use DIRECT_IO to bypass kernel caching > >- use the appropriate IO elevator > > > Are these general guidelines for writing fast I/O, or are there good > reasons to suspect that one of these cause occasional long blocks? All of these are candidates. > It would be nice if some of you tried the program I sent, and let's see > if you get the same results. Note that the real action begins when the > partition you write to is getting full. I ran four tests, 2 on 2.4.30 and 2 on 2.6.11-rc2 on my slow SCSI test machine. Each was run against an ext3 fs (.orig) and against my swap fs (.raw) I'm attaching the tail end of the distribution for each case. What this tells me is: - that the file system definitely gets in the way - that 2.6 is much better (no suprises there) - that adding both DIRECT_IO and (especially) kernel AIO will do wonder to improve the results. I might take a stab at adding both later and measuring. Which kernel are you using? listfile.2.4.orig: 1033934 1039532 1050982 1062859 1072628 1074470 1095608 1117049 1117059 1130159 ------------------- listfile.2.4.raw: 982159 989523 992505 1001441 1002068 1017735 1021283 1023135 1064356 1089229 ------------------- listfile.2.6.orig: 56340 70775 118317 149468 154185 172567 174960 271780 359151 1113761 ------------------- listfile.2.6.raw: 94354 94550 94693 98262 99790 112159 124361 147720 186502 192103 ------------------- -- Muli Ben-Yehuda http://www.mulix.org | http://mulix.livejournal.com/ -------------------------------------------------------------------------- Haifa Linux Club Mailing List (http://www.haifux.org) To unsub send an empty message to [EMAIL PROTECTED]
