[EMAIL PROTECTED] wrote on Wed, 01 Nov 2006 18:43 -0800:
> Oh ok.. I guess if we do whole page granularity we might be ok.. not
> sure. think of the following scenario,
> I am mostly worried about correctness issues with caching even in the
> presence of non-overlapping data.
> Initial file size is 0.
> Client 1 on node 1 writes to file offsets 0 to 3000 and client 2 on
> node 2 from 3001 to 6000. With a page size of 4096, they both share
> file block 0.. What ends up on disk for file block 0 with write
> buffering? Sadly, we cannot punt and say user error since the program
> was operating with nonoverlapping offsets.
But close-to-open consistency semantics allows that, right? On
close, local modifications are flushed to the server. On open, a
client checks to see if its local cache is still valid (timestamp
based or whatever). If two processes open the file, make
modifications, then one closes the file, the second process will
not try to merge the changes from the first---it will just overwrite
(parts of) the entire file. I don't think "nonoverlapping offsets"
guarantees that contents will be merged.
> Perhaps, we could ask such users/programs from not using the write
> buffering algorithm..
> Will MPI-IO programs be able to prevent this kind of thing from
> happening magically?
Oh, for MPI, it will certainly cause problems. ROMIO on NFS relies
on a working fcntl, doesn't it? They go beyond CTO to get
correctness.
I'm kind of not seeing the point of doing any optional write
buffering with CTO semantics in the first place. If we really want
a consistent, write-cached distributed file system we should just
grab an existing DLM and use it and client callbacks to manage
cached ranges. But that's something that interests me not at all.
(The read buffering you're doing is different. That makes some
sense for files that are marked as "known readonly", as you have
implemented it.)
-- Pete
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
