Hi, > You are still thinking about benefit to the image generation application > rather than the overall system.
I have both in mind. My app shall not run poorly and the system shall stay usable. > The reads are small and scattered, but they > are single use i/o, meaning that it is highly unlikely that the data will be > used before it passes out of the cache. I thought rather of something like read-ahead caching. My impression when backing up small files is that first the disk rattles like crazy but soon it becomes calm while the burner is still getting data at full speed. I blame that effect on the disk cache which can become several 100 MB, i confess. > use readahead buffering and > therefore will have the data to burn before it's needed. We use fifos to smoothen the output stream. That does not help with small files from disk because the ISO formatter first has to collect them and transform them into a stream. The "read-ahead" i think of is assumed in the file system driver and i believe it reads more than one disk block with a single read transaction. O_DIRECT is supposed only to read what the app demands and will not cache any surplus which it got due to block aligning. Ok. We have two read processes: - The (ISO) formatter where i doubt O_DIRECT is beneficial. - The burn program where i mostly wonder whether it is worth to introduce a non-usual i/o gesture. I assume it will not be of harm in most cases although this discussion started because David Weisgerber found one case where it was suboptimal. > Burning is a slow process relative to creating the image, Not necessarily. The said small files can bring disk performance under 8x DVD speed. (If i burn the image on the fly.) > That depends on how large your machine is. ;-) If you have 64GB a Blu-Ray > will fit. It is not about fitting all the media into RAM. It is about the time span which other processes have to renew the claim of their own RAM cache. Any process which is agile enough can keep it. I believe this effect eats up most of the theoretical benefit of O_DIRECT streaming on my systems. (I see no hampering with non-O_DIRECT so i can hardly expect big relief if i enable it.) David and Andy, i believe that there are reasons to use O_DIRECT on the input side of the burner fifo. But i never stumbled over them. Not on normal desktop Linux systems. My typical i/o problems - in burn app as in other concurrent processes - is when the disk goes wild during system backups in /usr. Then my X Window curser begins to move in hops and my system monitors cannot get their samples every second. As long as larger files (>= 100 kB) are to be backuped everyting runs smooth: One burner burning, one drive checkreading (via cached i/o), one or two dowloads streaming on disk, ... no problems. No kernel compiling here, i have to confess. Maybe that's the reason for our differences in perception: my use cases for large data are mostly sequential. Only the small file collector is truely random access. > You could try: > 1 - drop cache (echo 1 >/proc/sys/vm/drop_caches) > 2 - start a kernel compile > 3 - start building a 4GB or larger ISO image. > Then repeat with O_DIRECT. Use of iostat may give you additional data. Step 1 is obscure to me. I assume it gives up all cached data but allows to build up new caches during the test. Right ? Is step 3 intended as disk to disk operation ? (or as disk to burner on the fly ?) I have O_DIRECT tests on my todo list now. I'll see. Have a nice day :) Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
