General insight. Try XFS, it tries to avoid disk operations as much as
possible.
It should outperform ext3 on large files.
Regards,
tzahi.
> -----Original Message-----
> From: Haifux - Haifa Linux Club
> [mailto:[EMAIL PROTECTED] On Behalf Of Eli Billauer
> Sent: Wednesday, June 22, 2005 2:42 AM
> To: Haifa Linux Club Mailing list
> Subject: [Haifux] Real-time write on *ANY* filesystem
>
>
> Hello again,
>
> It turns out that it's not a FAT issue, but that the same
> problem occurs
> on ext3 systems as well. I've written a small program to test
> the delays
> between writes, and the results are not very encouraging.
> Specially when
> the disk gets full (it always does, doesn't it?).
>
> In my opinion, this should concert anyone who want to use a Linux box
> for storing a data stream (audio, video, whatever).
>
> I've attached the source of the program I used. Basically, it
> loops on
> writing 32kB chunks of data to a file, creating a list of
> number telling
> how much time (in microseconds) elapsed since the last loop
> (to stdout).
> There are two modes of testing: One is to let it write as fast as
> possible, and the second is to put delays between writes, which
> simulates waiting for incoming data. If there is enough room, the
> program will write slightly less than 2 GB (guess why?).
>
> Since Linux is a multitasking system, the results are not exactly
> repeatable. But the general impression is that writing to FAT or on
> ext3, on my laptop or on my desktop, they all behave more or
> less the same.
>
> First test regards full-speed write. Data was simply written
> as fast as
> possible. For a non-full partition, the write operation dwelled
> typically 5.5 ms, with occasional bursts of 0.7-0.9 *seconds*
> delay on
> the write operation. When the partition gets full, things get even
> nastier. Several seconds of blocking was observed. 5 seconds,
> and up to
> 14 seconds delay typically appeared a few times for a 2 GB
> writing session.
>
> Then I added a short sleep in the loop, in order to simulate data
> written at ~ 3MB/sec (which is reasonable for video
> capturing). This is
> far below the disk's physical capicity. The disk LED showed
> occasional
> flushes.
>
> Results: For a non-full partition, occasional peaks of up to
> 60ms were
> observed, which is something one can live with, probably. At
> 3 MB/s this
> means 180 kB stuck in the buffer. But when the partition
> started to get
> full, peaks of 0.2-0.3 seconds started to appear. The latter
> means 900
> kB waiting to go out, and this maybe explains why I
> originally had problems.
>
> If you want to see how your system behaves, just compile the attached
> code and go:
> ./writefat output-junk-data-file > listfile
>
> The list of loop timings will be in listfile. Use your
> favourite number
> cruncher to view graphs. (The program's name is due to "historic
> reasons"...) If you want to test the slower writing speeds, check the
> typical delay in the listfile, or see how fast the output file grows.
> The sleep period defined in the program itself is not reliable, since
> the operating system may not be able to sleep for too short periods.
>
> And finally: Does an RAM FIFO help? Surprisingly, the answer is no. I
> did the following:
>
> mknod mypipe p
> mbuffer -i mypipe -o /fatfs/output-file &
> ./writefat mypipe > listfile
>
> and was quite surprised to find delays of 0.2 sec. BTW,
> mbuffer seems to
> force the data to be flushed to disk much more often. The disk LED
> showed that writes occured all the time, unlike the direct write to
> file, in which flushes occured occasionally. And "mypipe" and
> "logfile"
> are on an ext3, while outfile-file is on FAT.
>
> Insights, anybody?
>
> Eli
>
> --
> Web: http://www.billauer.co.il
>
>
--------------------------------------------------------------------------
Haifa Linux Club Mailing List (http://www.haifux.org)
To unsub send an empty message to [EMAIL PROTECTED]
