Re: Mis-Design of Btrfs?
Am Freitag, den 15.07.2011, 10:24 -0400 schrieb Chris Mason: > Excerpts from Hugo Mills's message of 2011-07-15 10:07:24 -0400: > > On Fri, Jul 15, 2011 at 10:00:35AM -0400, Chris Mason wrote: > > > Excerpts from Ric Wheeler's message of 2011-07-15 09:31:37 -0400: > > > > On 07/15/2011 02:20 PM, Chris Mason wrote: > > > > > Excerpts from Ric Wheeler's message of 2011-07-15 08:58:04 -0400: > > > > >> On 07/15/2011 12:34 PM, Chris Mason wrote: > > > > > [ triggering IO retries on failed crc or other checks ] > > > > > > > > > >>> But, maybe the whole btrfs model is backwards for a generic layer. > > > > >>> Instead of sending down ios and testing when they come back, we > > > > >>> could > > > > >>> just set a verification function (or stack of them?). > > > > >>> > > > > >>> For metadata, btrfs compares the crc and a few other fields of the > > > > >>> metadata block, so we can easily add a compare function pointer and > > > > >>> a > > > > >>> void * to pass in. > > > > >>> > > > > >>> The problem is the crc can take a lot of CPU, so btrfs kicks it off > > > > >>> to > > > > >>> threading pools so saturate all the cpus on the box. But there's no > > > > >>> reason we can't make that available lower down. > > > > >>> > > > > >>> If we pushed the verification down, the retries could bubble up the > > > > >>> stack instead of the other way around. > > > > >>> > > > > >>> -chris > > > > >> I do like the idea of having the ability to do the verification and > > > > >> retries down > > > > >> the stack where you actually have the most context to figure out > > > > >> what is possible... > > > > >> > > > > >> Why would you need to bubble back up anything other than an error > > > > >> when all > > > > >> retries have failed? > > > > > By bubble up I mean that if you have multiple layers capable of doing > > > > > retries, the lowest levels would retry first. Basically by the time > > > > > we > > > > > get an -EIO_ALREADY_RETRIED we know there's nothing that lower level > > > > > can > > > > > do to help. > > > > > > > > > > -chris > > > > > > > > Absolutely sounds like the most sane way to go to me, thanks! > > > > > > > > > > It really seemed like a good idea, but I just realized it doesn't work > > > well when parts of the stack transform the data. > > > > > > Picture dm-crypt on top of raid1. If raid1 is responsible for the > > > crc retries, there's no way to crc the data because it needs to be > > > decrypted first. > > > > > > I think the raided dm-crypt config is much more common (and interesting) > > > than multiple layers that can retry for other reasons (raid1 on top of > > > raid10?) > > > >Isn't this a case where the transformative mid-layer would replace > > the validation function before passing it down the stack? So btrfs > > hands dm-crypt a checksum function; dm-crypt then stores that function > > for its own purposes and hands off a new function to the DM layer > > below that which decrypts the data and calls the btrfs checksum > > function it stored earlier. > > Then we're requiring each transformation layer to have their own crcs, > and if the higher layers have a stronger crc (or other checks), there's > no path to ask the lower layers for other copies. > > Here's a concrete example. In each metadata block, btrfs stores the > fsid and the transid of the transaction that created it. In the case of > a missed write, we'll read a perfect block from the lower layers. Any > crcs will be correct and it'll pass through dm-crypt with flying colors. > > But, it won't be the right block. Btrfs will notice this and EIO. In > the current ask-for-another-mirror config we'll go down and grab the > other copy. > > In the stacked validation function model, dm-crypt replaces our > verification functions with something that operates on the encrypted > data, and it won't be able to detect the error or kick down to the > underlying raid1 for another copy. > > -chris > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html I think the point is not to replace the crc function in the dm_crypt case, but to wrap it with an decrypt function which then calls the crc function. So even if a lower mirror uses the new dm-crypt crc function, the btrfs crc function still gets called - at the end of the chain. Regards, Christian Aßfalg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Required Kernel version to compile git btrfs with DKMS? Problems with Ubuntu 10.10
Hi all, I'd like to look into btrfs code and try some things, so I followed the instructions on the wiki (link below) to setup git and get a dkms build against my kernel. I'm using Ubuntu 10.10 with a 2.6.35 Kernel, which I assume is too old. What version do I need? https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories I've followed the guide (skipped initrd) until the dkms build command, which fails with the following error: dkms build -m btrfs -v git root@chris-desktop:/home/chris/workspace/btrfs/git-repo/btrfs-unstable# cat /var/lib/dkms/btrfs/git/build/make.log DKMS make.log for btrfs-git for kernel 2.6.35-28-generic (i686) So 10. Apr 22:05:11 CEST 2011 make: Gehe in Verzeichnis '/usr/src/linux-headers-2.6.35-28-generic' LD /var/lib/dkms/btrfs/git/build/built-in.o CC [M] /var/lib/dkms/btrfs/git/build/super.o In file included from /var/lib/dkms/btrfs/git/build/super.c:43: /var/lib/dkms/btrfs/git/build/ctree.h:31: fatal error: trace/events/btrfs.h: Datei oder Verzeichnis nicht gefunden compilation terminated. make[1]: *** [/var/lib/dkms/btrfs/git/build/super.o] Fehler 1 make: *** [_module_/var/lib/dkms/btrfs/git/build] Fehler 2 make: Verlasse Verzeichnis '/usr/src/linux-headers-2.6.35-28-generic' This seems to be fairly new (related to initial tracepoint support), from the two Mailing list threads "[PATCH] Btrfs: add initial tracepoint support for btrfs" and "[GIT PULL] Btrfs updates for 2.6.39". The attached make.log is when I simply copy the missing file over from the btrfs git tree (fails with among others "/var/lib/dkms/btrfs/git/build/super.c:578: error: ‘struct super_block’ has no member named ‘s_d_op’"). So I am guessing that my Ubuntu 10.10 2.6.35 Kernel is too old? Will the 2.6.38 from the next Ubuntu release work? Regards, Christian Aßfalg DKMS make.log for btrfs-git for kernel 2.6.35-28-generic (i686) So 10. Apr 22:15:02 CEST 2011 make: Gehe in Verzeichnis '/usr/src/linux-headers-2.6.35-28-generic' LD /var/lib/dkms/btrfs/git/build/built-in.o CC [M] /var/lib/dkms/btrfs/git/build/super.o In file included from /var/lib/dkms/btrfs/git/build/super.c:43: /var/lib/dkms/btrfs/git/build/ctree.h:2237: warning: ‘struct fstrim_range’ declared inside parameter list /var/lib/dkms/btrfs/git/build/ctree.h:2237: warning: its scope is only this definition or declaration, which is probably not what you want In file included from include/trace/ftrace.h:285, from include/trace/define_trace.h:73, from include/trace/events/btrfs.h:667, from /var/lib/dkms/btrfs/git/build/super.c:56: include/trace/events/btrfs.h: In function ‘ftrace_raw_output_btrfs_transaction_commit’: include/trace/events/btrfs.h:47: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:47: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:47: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h: In function ‘ftrace_raw_output_btrfs__inode’: include/trace/events/btrfs.h:68: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:68: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:68: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h: In function ‘ftrace_raw_output_btrfs_get_extent’: include/trace/events/btrfs.h:144: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:144: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:144: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:144: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:144: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:144: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:144: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h: In function ‘ftrace_raw_output_btrfs__ordered_extent’: include/trace/events/btrfs.h:197: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:197: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:197: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h: In function ‘ftrace_raw_output_btrfs__writepage’: include/trace/events/btrfs.h:273: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:273: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h:273: warning: large integer implicitly truncated to unsigned type include/trace/events/btrfs.h: In function ‘ftrace_raw_output_btrfs_writepage_end_io_hook’: include/trace/
