Hi, the code assumption is that the underlying device has no volatile write cache, i was absolute sure we have that somewhere in the FAQ, but i couldn't find it, so i will talk to somebody to correct this. if i understand https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt correct one could enforce this by setting REQ_FUA, but thats not explicitly set today, at least i can't see it. i will discuss this with one of our devs who owns this code and come back.
sven On Thu, Sep 7, 2017 at 8:05 PM Aaron Knister <[email protected]> wrote: > Thanks Sven. I didn't think GPFS itself was caching anything on that > layer, but it's my understanding that O_DIRECT isn't sufficient to force > I/O to be flushed (e.g. the device itself might have a volatile caching > layer). Take someone using ZFS zvol's as NSDs. I can write() all day log > to that zvol (even with O_DIRECT) but there is absolutely no guarantee > those writes have been committed to stable storage and aren't just > sitting in RAM until an fsync() occurs (or some other bio function that > causes a flush). I also don't believe writing to a SATA drive with > O_DIRECT will force cache flushes of the drive's writeback cache.. > although I just tested that one and it seems to actually trigger a scsi > cache sync. Interesting. > > -Aaron > > On 9/7/17 10:55 PM, Sven Oehme wrote: > > I am not sure what exactly you are looking for but all blockdevices are > > opened with O_DIRECT , we never cache anything on this layer . > > > > > > On Thu, Sep 7, 2017, 7:11 PM Aaron Knister <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi Everyone, > > > > This is something that's come up in the past and has recently > resurfaced > > with a project I've been working on, and that is-- it seems to me as > > though mmfsd never attempts to flush the cache of the block devices > its > > writing to (looking at blktrace output seems to confirm this). Is > this > > actually the case? I've looked at the gpl headers for linux and I > don't > > see any sign of blkdev_fsync, blkdev_issue_flush, WRITE_FLUSH, or > > REQ_FLUSH. I'm sure there's other ways to trigger this behavior that > > GPFS may very well be using that I've missed. That's why I'm asking > :) > > > > I figure with FPO being pushed as an HDFS replacement using commodity > > drives this feature has *got* to be in the code somewhere. > > > > -Aaron > > > > -- > > Aaron Knister > > NASA Center for Climate Simulation (Code 606.2) > > Goddard Space Flight Center > > (301) 286-2776 > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at spectrumscale.org <http://spectrumscale.org> > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > > > > _______________________________________________ > > gpfsug-discuss mailing list > > gpfsug-discuss at spectrumscale.org > > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > -- > Aaron Knister > NASA Center for Climate Simulation (Code 606.2) > Goddard Space Flight Center > (301) 286-2776 > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
