Hi Eric, Eric Biggers wrote:
> >> So I finally ended up with cacheing the inodes so that >> reopening an inode which has been opened recently is >> fast. The stat() data are cached, but the runlists are >> not. > > Hmm... I wasn't paying attention to the fact that inodes are cached. This is in ntfs_inode_open() as opposed to ntfs_inode_real_open() > Are you saying the issue was that entries in long runlists prior to the read > or > write took longer to skip over when the runlist was cached as opposed to > uncached? Maybe the runlist just needs to be held in a different data > structure > in memory. Compressed and video file can have their runlist split over tens of extents. Currently only the needed fragment is decompressed and the search for an lcn is only done in one (sometimes two) extents. My tries showed that if you want to cache, you have to know how to get rid of the parts of the runlist you do not need any more, or to improve the logic to find an lcn. So you need to improve how parts of runlists are controlled in memory. I only tested adding the number of runs so that I could make dichotomical searches, and that had a big impact on code (I gave up, but I have the patches still available). I did not investigate getting rid of unneeded parts. >> About integrating system compression into ntfs-3g, I >> want to try making it a plugin loaded on demand. this >> will improve modularity and facilitate maintainance. >> For instance an upgrade such as the one you are >> announcing could be released independently of ntfs-3g. >> In the fuse interface (ntfs-3g.c or lownetfs-3g.c) the >> inode would be opened, and if it has a reparse mark, >> control would be passed to a handler depending of the >> reparse tag, either internal (symlinks and junctions) >> or external (system compression). The interface would >> be like the fuse one, with the path replaced by a pointer >> to inode descriptor. More thoughts needed... > > Interesting idea. I agree that it would be nice to support new types of > reparse > points without having them directly tied to the core library/driver. There is > quite a bit of code needed to support the system-compressed files and I expect > that the majority of users don't need it. > > I suppose that NTFS-3g would have to provide a plugins directory, a plugin > package would drop a .so file into it, and the FUSE driver would dynamically > load the available plugins when starting up? My initial view (subject to change) is to have the reparse tag in the name of the .so (such as ntfs_plugin_80000017.so). When some application wants to access a file with reparse tag 80000017 for the first time, the plugin is loaded and its init() function returns an array of functions. Defining a plugin directory would require some customization by distributions. Roughly speaking, the hooks you have inserted have to be replaced by two function calls, one to locate the plugin, then a call to the needed plugin entry. > I also suppose it would have to be a plugin at the FUSE driver level and not > at > the library level, since the library interface is more low-level and deals > directly with inodes and attributes. Yes. > I'm wondering about where and what the "hooks" would need to be. In my latest > patch I have three "hooks" for system compressed files: one in getattr(), one > in > open(), and one in read(). But different reparse point handlers will likely > need other hooks. I think it's also expected that reparse point handlers will > need to override only specific behavior, even within a single FUSE operation, > and leave the rest as the default. For example, with system compressed files > there is no special behavior needed for alternate data streams and inode > metadata such as timestamps, but that may not be true of all reparse points. Well if an entry is not defined in a plugin, the standard behavior would apply. For instance there is probably no need for a specific utime() or setxattr() on system compressed files. > By the way, Microsoft's software which handles system-compressed files is more > than just a reparse point handler. It's actually a filter driver that hides > the > fact that the feature is implemented using reparse points at all. The reparse > point attribute is hidden, and as far as I know there is no way to even tell > that a given file is a system-compressed file without using a special ioctl > (FSCTL_GET_EXTERNAL_BACKING). But I think that's a complete hack and NTFS-3g > doesn't necessarily need to do things the same way... I am not expecting to support all Microsoft features. Even trying to support junctions implies overcoming pitfalls such as interpreting drive letters and GUIDs designating volumes. I need a few days for a proposal on which we could debate. Regards Jean-Pierre ------------------------------------------------------------------------------ _______________________________________________ ntfs-3g-devel mailing list ntfs-3g-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel
