Andreas Dilger wrote:
> Marty Fouts, you write:
> > [snip] ... I want to:
> > Inform the file system that the file open on fd but not yet written to
> > will be length bytes in size. This is an advisory call to the file system
> > and may be ignored.
>
> There was some discussion a few months ago of implementing file
> pre-allocation for the VFS in general in the 2.5 kernel. This can
> happen in the kernel at large because many files are held entirely
> in cache before they start writing to disk at the next sync call, so
> the size is also known in advance. This is what the XFS filesystem
> does under IRIX, although I doubt it currently does this under Linux.
The Linux version of XFS does allocate on flush, or delayed allocation.
We also have a preallocate call implemented, although not as a
generic VFS op, currently it is an ioctl call. These two are really
different ways of getting similar results - obviously preallocation
to a known size gets better results.
I suspect the other part of the picture for an application like this
might be direct I/O for putting the data into the filesystem in the
first place.
Steve Lord
>
> This is a more generic mechanism than what you propose, but it may be
> that at least the ext2 block selection algorithms could be enhanced to
> do decent contiguous pre-allocation for files if told to do so via ioctl,
> in preparation of the VFS doing this internally.
>
> I don't recall the details, but Al Viro, Stephen Tweedie and maybe Rik
> van Riel? would be the people to talk to about maximizing the chance
> that your code is what is wanted for the 2.5 kernel. I believe they
> already have solid ideas on how they think this should be implemented.
>
> Because ext2 doesn't have any knowledge of contiguous extents itself, it
> may be that you want to generate a sorted (partial) list of contiguous
> extents (start block,size) at mount time (maybe based on some knowledge
> of your expected file sizes), and keep enough of these extents cached
> so that you don't need to search the bitmaps for them all the time.
> As you do garbage collection you can re-populate the extent lists, if
> you are sure that all of the files you delete are contiguous (maybe via
> a flag stored in the inode), or if you check in truncate that they are
> contiguous.
>
> Obviously, since you are concerned with fragmentation and the I/O bandwidth
> is an issue, you must be using mostly large files, so you obviously want
> to tune your ext2 filesystem for files of that size (i.e. 4k block size,
> number of inodes, etc).
>
> Another thing that just came to mind is some work that was mentioned
> on the list a couple of months ago (in a long thread about kernel
> latency issues) about a "multi-media filesystem" that was being used
> for music recording. It was a much simpler filesystem than ext2, and
> was more geared towards real-time recording many large audio tracks of
> a fixed size. You may want to look at it, either to use it directly,
> or possibly implement a custom solution for your appliance that does
> more of what you want, rather than ext2 which may be a more generic,
> full-featured filesystem than you want for your data. You may want to
> keep ext2 for the root filesystem, however.
>
> Cheers, Andreas
> -
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to [EMAIL PROTECTED]
>
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [EMAIL PROTECTED]
