Re: Page sharing between a block device and a mounted file system?
Hello, Raymond Jennings, le jeu. 16 mai 2019 10:35:07 -0700, a ecrit: > The question is, if i have a task serving a block device, which has a task > serving a filesystem as a client, how do I keep the same page of data for a > file block from being stored redundantly as two separate pages, perhaps in two > separate memory objects? Actually IIRC what really happens in a hurd system is that the filesystem asks the storeio translator where the data is (file_get_storage_info), and tries to open that directly (see libstore's store_create used by ext2fs), so there is no duplication in the end. Samuel
Re: Page sharing between a block device and a mounted file system?
On Thu, May 16, 2019 at 08:58:07AM -0700, Raymond Jennings wrote: > So um...I may be totally missing the mark on this but if I have one task > running a file system and another task managing a block device mounted as > that file system, how does one manage a file in such a way that it won't > store data in memory redundantly that is also stored in the block device? > > Criteria that I'm guessing at: > > * The file needs to be mmap'able as well as accepting read/write requests > * The file is stored on the block device at locations determined by the > file's inode/extent tree > * The block device itself presumably can also be mmap'ed and accept > read/write requests, including to the same area of the "disk" that the file > data occupies > * The same data, and presumably, the same pages in memory, are being used > to cache both > * A read or write in one will reflect immediately in the other > > I must admit that I have no idea beyond the theoretical stuff gleanable > from the mach reference guide how this would work. > > As a thought exercise I'm trying to design my own microkernel inspired by > mach so I'm curious how one would prevent redundancy or inconsistency in > this sort of scenario. > > By contrast, I'm assuming that the file's metadata itself (extent trees, > directory listings, and so on) would just be read/write direct to the block > device without worrying about being "aliased" from the file itself. Ditto > if the file is accessed/mmap'ed uncompressed, but stored in a compressed > form on the block device. You could take a look at the ext2fs translator. Although I don't see the difficulty here. The file system should be the only server to cache file data in the page cache, and should merely use direct access to the block device. -- Richard Braun
Re: Page sharing between a block device and a mounted file system?
Already did, that's why I'm asking. The question is, if i have a task serving a block device, which has a task serving a filesystem as a client, how do I keep the same page of data for a file block from being stored redundantly as two separate pages, perhaps in two separate memory objects? On Thu, May 16, 2019 at 9:42 AM Samuel Thibault wrote: > Hello, > > Just a quick answer: data caching is typically managed by the mach > memory objects, so you'll want to read about them. > > Samuel > > Raymond Jennings, le jeu. 16 mai 2019 08:58:07 -0700, a ecrit: > > (referred here by Richard Braun <[1]rbr...@sceen.net>) > > > > So um...I may be totally missing the mark on this but if I have one task > > running a file system and another task managing a block device mounted > as that > > file system, how does one manage a file in such a way that it won't > store data > > in memory redundantly that is also stored in the block device? > > > > Criteria that I'm guessing at: > > > > * The file needs to be mmap'able as well as accepting read/write requests > > * The file is stored on the block device at locations determined by the > file's > > inode/extent tree > > * The block device itself presumably can also be mmap'ed and accept > read/write > > requests, including to the same area of the "disk" that the file data > occupies > > * The same data, and presumably, the same pages in memory, are being > used to > > cache both > > * A read or write in one will reflect immediately in the other > > > > I must admit that I have no idea beyond the theoretical stuff gleanable > from > > the mach reference guide how this would work. > > > > As a thought exercise I'm trying to design my own microkernel inspired > by mach > > so I'm curious how one would prevent redundancy or inconsistency in this > sort > > of scenario. > > > > By contrast, I'm assuming that the file's metadata itself (extent trees, > > directory listings, and so on) would just be read/write direct to the > block > > device without worrying about being "aliased" from the file itself. > Ditto if > > the file is accessed/mmap'ed uncompressed, but stored in a compressed > form on > > the block device. > > > > References: > > > > [1] mailto:rbr...@sceen.net >
Page sharing between a block device and a mounted file system?
(referred here by Richard Braun ) So um...I may be totally missing the mark on this but if I have one task running a file system and another task managing a block device mounted as that file system, how does one manage a file in such a way that it won't store data in memory redundantly that is also stored in the block device? Criteria that I'm guessing at: * The file needs to be mmap'able as well as accepting read/write requests * The file is stored on the block device at locations determined by the file's inode/extent tree * The block device itself presumably can also be mmap'ed and accept read/write requests, including to the same area of the "disk" that the file data occupies * The same data, and presumably, the same pages in memory, are being used to cache both * A read or write in one will reflect immediately in the other I must admit that I have no idea beyond the theoretical stuff gleanable from the mach reference guide how this would work. As a thought exercise I'm trying to design my own microkernel inspired by mach so I'm curious how one would prevent redundancy or inconsistency in this sort of scenario. By contrast, I'm assuming that the file's metadata itself (extent trees, directory listings, and so on) would just be read/write direct to the block device without worrying about being "aliased" from the file itself. Ditto if the file is accessed/mmap'ed uncompressed, but stored in a compressed form on the block device.
Re: Page sharing between a block device and a mounted file system?
Hello, Just a quick answer: data caching is typically managed by the mach memory objects, so you'll want to read about them. Samuel Raymond Jennings, le jeu. 16 mai 2019 08:58:07 -0700, a ecrit: > (referred here by Richard Braun <[1]rbr...@sceen.net>) > > So um...I may be totally missing the mark on this but if I have one task > running a file system and another task managing a block device mounted as that > file system, how does one manage a file in such a way that it won't store data > in memory redundantly that is also stored in the block device? > > Criteria that I'm guessing at: > > * The file needs to be mmap'able as well as accepting read/write requests > * The file is stored on the block device at locations determined by the file's > inode/extent tree > * The block device itself presumably can also be mmap'ed and accept read/write > requests, including to the same area of the "disk" that the file data occupies > * The same data, and presumably, the same pages in memory, are being used to > cache both > * A read or write in one will reflect immediately in the other > > I must admit that I have no idea beyond the theoretical stuff gleanable from > the mach reference guide how this would work. > > As a thought exercise I'm trying to design my own microkernel inspired by mach > so I'm curious how one would prevent redundancy or inconsistency in this sort > of scenario. > > By contrast, I'm assuming that the file's metadata itself (extent trees, > directory listings, and so on) would just be read/write direct to the block > device without worrying about being "aliased" from the file itself. Ditto if > the file is accessed/mmap'ed uncompressed, but stored in a compressed form on > the block device. > > References: > > [1] mailto:rbr...@sceen.net
