Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk
On 04/12/2015 01:47, Qu Wenruo wrote: > [run btrfsck] I did that, too with an old btrfsck version (4.0) and it found the following errors. Then I did a btrfsck --repair, and I have been able to complete my "du -s" test. The next step will we to run a "btrfs scrub" to check if data loss did happen... # btrfsck /dev/sdb1 Checking filesystem on /dev/sdb1 UUID: f6d4db2e-962b-42db-87b1-35064a4d38e0 checking extents checking free space cache block group 314635714560 has wrong amount of free spacefailed to load free space cache for block group 314635714560 There is no free space entry for 353290420224-353290764288 There is no free space entry for 353290420224-353827291136 cache appears valid but isnt 353290420224 There is no free space entry for 541732175872-541732208640 There is no free space entry for 541732175872-542268981248 cache appears valid but isnt 541732110336 Wanted bytes 32768, found 262144 for off 1008273178624 Wanted bytes 536625152, found 262144 for off 1008273178624 cache appears valid but isnt 1008272932864 block group 1475887497216 has wrong amount of free spacefailed to load free space cache for block group 1475887497216 block group 1823242977280 has wrong amount of free spacefailed to load free space cache for block group 1823242977280 There is no free space entry for 1827001073664-1827002810368 There is no free space entry for 1827001073664-1827537944576 cache appears valid but isnt 1827001073664 There is no free space entry for 1969305501696-1969305518080 There is no free space entry for 1969305501696-1969842290688 cache appears valid but isnt 1969305419776 There is no free space entry for 2021381947392-2021381963776 There is no free space entry for 2021381947392-2021918769152 cache appears valid but isnt 2021381898240 There is no free space entry for 2027287478272-2027287724032 There is no free space entry for 2027287478272-2027824349184 cache appears valid but isnt 2027287478272 There is no free space entry for 2143889227776-2143889244160 There is no free space entry for 2143889227776-2144426000384 cache appears valid but isnt 2143889129472 found 1977224107644 bytes used err is -22 total csum bytes: 1925245108 total tree bytes: 5773115392 total fs tree bytes: 3504685056 total extent tree bytes: 156975104 btree space waste bytes: 780048699 file data blocks allocated: 1971884707840 referenced 1971875930112 btrfs-progs v4.0 -- Laurent. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Subvolume UUID, data corruption?
On 2015-12-05 07:01, Hugo Mills wrote: On Sat, Dec 05, 2015 at 04:28:24AM +0100, Christoph Anton Mitterer wrote: On Fri, 2015-12-04 at 13:07 +, Hugo Mills wrote: I don't think it'll cause problems. Is there any guaranteed behaviour when btrfs encounters two filesystems (i.e. not talking about the subvols now) with the same UUID? Nothing guaranteed, but the likelihood is that things will go badly wrong, in the sense of corrupt filesystems. Given that it's long standing behaviour that people could clone filesystems (dd, etc.) and this just worked™, btrfs should at least handle such case gracefully. For example, when already more than one block device with a btrfs of the same UUID are known, then it should refuse to mount any of them. And if one is already known and another device pops up it should refuse to mount that and continue to normally use the already mounted one. Except that that's exactly the mechanism that btrfs uses to handle multi-device filesystems, so you've just broken anything with more than one device in the FS. If you inspect the devid on each device as well, and refuse duplicates of those, you've just broken any multipathing configurations. This already potentially breaks multipath configurations, as well as dm-cache, some soft raid configurations, and probably other things as well. Even if you can handle that, if you have two copies of dev1, and two copies of dev2, how do you guarantee that the "right" pair of dev1 and dev2 is selected? (e.g. if you have them as network devices, and the device enumeration order is unstable on each boot). In some cases it can be done without much effort. Take dm-cache for example. The hierarchy of devices in a dm-cache device looks like this: cached-device + backing-device + cache-pool + pool-storage + pool-metadata At a minimum, the cached device and the backing device contain identical data (the cached-device just has a writeback or writethrough cache on it), and the pool storage device may under some circumstances look like a BTRFS filesystem as well. In this case, it's pretty obvious that the only device that BTRFS should be accessing is the cached device, not the backing device or the pool storage device. For this, if we simply blacklist all devices that are themselves components in device-mapper tables, then we avoid the issue here, and possibly in some other as of yet undiscovered cases. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk
On 04/12/2015 01:47, Qu Wenruo wrote: > The chunk mismatch problem should be resolved already, as the patch is merged > in david's devel branch. Great ! I am looking forward to a new release with this bug fix... > But for the kernel abort transaction, I'm sorry there is no good clue yet. > Only generic advice like update kernel to recent rc, as it has some new fixes > which may help your case. I tested again my "du -s" test with kernel 4.4-rc4 and got the following backtrace. Is this a normal error message or an anomaly in the kernel ? [ 194.509475] BTRFS: device label sauvegarde-IUT2 devid 1 transid 9057 /dev/sdb1 [ 194.599397] BTRFS info (device sdb1): disk space caching is enabled [ 227.764561] BTRFS warning (device sdb1): block group 314635714560 has wrong amount of free space [ 227.764564] BTRFS warning (device sdb1): failed to load free space cache for block group 314635714560, rebuild it now [ 228.108072] [ cut here ] [ 228.108089] WARNING: CPU: 0 PID: 3303 at /home/kernel/COD/linux/fs/btrfs/extent-tree.c:2927 btrfs_run_delayed_refs+0x26b/0x2a0 [btrfs]() [ 228.108090] BTRFS: Transaction aborted (error -17) [ 228.108091] Modules linked in: ses enclosure uas usb_storage xt_CHECKSUM iptable_mangle xt_tcpudp rfcomm xt_addrtype xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 iptable_filter ip_tables x_tables nf_nat nf_conntrack bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c binfmt_misc bnep drbg ansi_cprng dm_crypt dell_wmi sparse_keymap intel_rapl dell_rbtn snd_hda_codec_hdmi iosf_mbi dell_laptop x86_pkg_temp_thermal intel_powerclamp dcdbas snd_hda_codec_idt coretemp dell_smm_hwmon snd_hda_codec_generic crct10dif_pclmul crc32_pclmul snd_hda_intel snd_hda_codec aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd snd_hda_core snd_hwdep arc4 btusb joydev input_leds btrtl btbcm btintel bluetooth iwlmvm [ 228.108120] serio_raw mac80211 snd_pcm iwlwifi snd_seq_midi snd_seq_midi_event snd_rawmidi cfg80211 snd_seq snd_seq_device snd_timer lpc_ich mei_me snd mei soundcore kvm_intel shpchp kvm irqbypass dell_smo8800 8250_fintek mac_hid tpm_rng parport_pc ppdev lp parport autofs4 btrfs xor raid6_pq hid_generic usbhid hid i915 psmouse firewire_ohci ahci libahci i2c_algo_bit firewire_core drm_kms_helper sdhci_pci e1000e sdhci crc_itu_t syscopyarea sysfillrect sysimgblt ptp fb_sys_fops pps_core drm wmi video fjes [ 228.108144] CPU: 0 PID: 3303 Comm: btrfs-transacti Not tainted 4.4.0-040400rc4-generic #201512061930 [ 228.108145] Hardware name: Dell Inc. Latitude E6520/0NVF5K, BIOS A19 11/14/2013 [ 228.108146] 6496ca82 88007c257d08 813c8ab4 [ 228.108148] 88007c257d50 88007c257d40 8107d772 8801b3b9b000 [ 228.108149] 880221f92000 8801c4034170 880203a2d280 [ 228.108151] Call Trace: [ 228.108155] [] dump_stack+0x44/0x60 [ 228.108158] [] warn_slowpath_common+0x82/0xc0 [ 228.108159] [] warn_slowpath_fmt+0x5c/0x80 [ 228.108166] [] ? __btrfs_run_delayed_refs+0xc40/0x1150 [btrfs] [ 228.108173] [] btrfs_run_delayed_refs+0x26b/0x2a0 [btrfs] [ 228.108180] [] ? btrfs_wait_pending_ordered+0x22/0x90 [btrfs] [ 228.108188] [] btrfs_commit_transaction+0x4d2/0xa70 [btrfs] [ 228.108195] [] transaction_kthread+0x229/0x240 [btrfs] [ 228.108201] [] ? btrfs_cleanup_transaction+0x550/0x550 [btrfs] [ 228.108204] [] kthread+0xd8/0xf0 [ 228.108206] [] ? kthread_create_on_node+0x1a0/0x1a0 [ 228.108210] [] ret_from_fork+0x3f/0x70 [ 228.108211] [] ? kthread_create_on_node+0x1a0/0x1a0 [ 228.108213] ---[ end trace 39e2e9062b36c29e ]--- [ 228.108215] BTRFS: error (device sdb1) in btrfs_run_delayed_refs:2927: errno=-17 Object already exists [ 228.108217] BTRFS info (device sdb1): forced readonly [ 228.108219] BTRFS warning (device sdb1): Skipping commit of aborted transaction. [ 228.108220] BTRFS: error (device sdb1) in cleanup_transaction:1747: errno=-17 Object already exists -- Laurent. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: new oops in 4.4.0-rc4
On Thu, Dec 10, 2015 at 10:36:17AM -0600, Jon Christopherson wrote: > Hello, > > I noticed this new oops since running 4.4.0-rc4. Happens shortly after boot > and pretty much kills the system: > > > [ 177.774250] [ cut here ] > >[ 177.774256] kernel BUG at /data0/Source/mainline/mm/page-writeback.c:2654! > >[ 177.774258] invalid opcode: [#1] SMP > >[ 177.774261] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE > >nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 > >nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 xt_tcpudp > >bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables > >iptable_filter ip_tables x_tables ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad > >ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi rfcomm > >bnep nfsd auth_rpcgss nfs_acl binfmt_misc nfs lockd grace sunrpc fscache xfs > >libcrc32c snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi > >nvidia_modeset(POE) eeepc_wmi mxm_wmi asus_wmi sparse_keymap intel_rapl > >iosf_mbi x86_pkg_temp_thermal intel_powerclamp dm_multipath nvidia(POE) > >btusb kvm_intel btrtl nls_iso8859_1 kvm btbcm irqbypass snd_hd > a_intel wl(POE) btintel hid_logitech_hidpp joydev bluetooth serio_raw > snd_hda_codec snd_hda_core snd_seq_midi cfg80211 snd_seq_midi_event snd_hwdep > snd_rawmidi lpc_ich snd_pcm drm snd_seq snd_seq_dev > ice snd_timer 8250_fintek snd mei_me mei soundcore wmi mac_hid parport_pc > shpchp ppdev msr nct6775 hwmon_vid coretemp lp parport btrfs xor raid6_pq > drbg ansi_cprng dm_crypt dm_mirror dm_region_hash dm_log hid_generic > hid_logitech_dj usbhid hid crct10dif_pclmul crc32_pclmul ahci aesni_intel > aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse libahci video > >[ 177.774357] CPU: 5 PID: 5158 Comm: thunderbird Tainted: PW OE > >4.4.0-121-generic #201512100930 > >[ 177.774360] Hardware name: System manufacturer System Product Name/P8P67 > >DELUXE, BIOS 3602 10/31/2012 > >[ 177.774362] task: 88040b6d ti: 8803af864000 task.ti: > >8803af864000 > >[ 177.774364] RIP: 0010:[] [] > >clear_page_dirty_for_io+0xe1/0x1a0 Dave Jones sent in a report about this with trinity too, I'm digging in today. Since you can trigger this reliably, what was the last known-good kernel for you? -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: attacking btrfs filesystems via UUID collisions?
Sorry, I'm just about to change my mail system, and used a bogus test From: address in the previous mail (please replace fo@fo with cales...@scientia.net). Apologies for any inconveniences and this noise here. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: attacking btrfs filesystems via UUID collisions?
On Thu, 2015-12-10 at 12:42 -0700, Chris Murphy wrote: > That isn't what I'm suggesting. In the multiple device volume case > where there are two exact (same UUID, same devid, same generation) > instances of one of the block devices, Btrfs could randomly choose > either one if it's an RO mount. No, for the same reasons as just stated in my mail few minutes ago. An attacker could probably find out the UUID/devid/generation... it would probably possible for him to craft a device with exactly those and try to use it. If then btrfs would select any of these, it may also select the wrong one - ro or rw, this may likely lead to problems. > > About 1 and 2 ... if 3 gets fulfilled, why? > > DD itself is not a problem "if" the UUID is changed after it > > (which is a command as simple as dd), and if someone doesn't > > know that, he/she will notice when mount refuses to work > > because UUID duplicate. > > dd is not a copy operation. It's creating a 2nd original. You don't > end up with an original and a copy (or clone). A copy or clone has > some distinguishing difference. Volume UUID is used throughout Btrfs > metadata, it's not just in the superblocks. Changing volume UUID > requires a rewrite of all metadata. This is inefficient for two > reasons: one dd copies unused sectors; two it copies metadata that > will have to be completely rewritten by btrfstune to change volume > UUID; and also the subvolume UUIDs aren't changed, so it's an > incomplete solution that has problems (see other threads). Well dd is surely not the only thing that can be used to create a clone (i.e. a bitwise identical copy - I guess we don't really care which is the "original" and which are the "clones", or whether these are "2nd originals). We always just use it here as an example for scenarios in which bitwise identical copies are created. And even if internally it's a big thing, from the user's PoV, changing the UUID is pretty simple (I guess that's what S.J. meant). > If your workflow requires making an exact copy (for the shelf or for > an emergency) then dd might be OK. But most often it's used because > it's been easy, not because it's a good practice. Ufff.. I wouldn't got that far to call something here bad or good practice. At least, I do not see any reason to call it a bad practice, except that systems got over time much more complex and haven't dealt properly with the problems that can occur by using dd. Again, I don't demand magical "solutions" (i.e. the btrfs or LVM people getting code into all dd like tools, so that these auto-detect when the duplicate such data and auto-change the UUIDs)... they just should handle the situations gracefully. > Note that Btrfs is > not unique, XFS v5 does a very similar thing with volume UUID as > well, > and resulted in this change: > http://oss.sgi.com/pipermail/xfs/2015-April/041267.html Do you mean that xfs may suffer from the same issues that we're talking about here? If so, one should probably give them a notice. > Using dd also means the volume is offline. Not really, you could do it on a snapshotted LV, while the "original" is still running. Or in emergency cases one could do it on a ro-remounted... probably not guaranteed to work, but may do so in practise. Cheers, Chris. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: attacking btrfs filesystems via UUID collisions?
On Fri, Dec 11, 2015 at 3:21 PM, Christoph Anton Mittererwrote: > On Thu, 2015-12-10 at 12:42 -0700, Chris Murphy wrote: >> That isn't what I'm suggesting. In the multiple device volume case >> where there are two exact (same UUID, same devid, same generation) >> instances of one of the block devices, Btrfs could randomly choose >> either one if it's an RO mount. > No, for the same reasons as just stated in my mail few minutes ago. > An attacker could probably find out the UUID/devid/generation... it > would probably possible for him to craft a device with exactly those > and try to use it. For anything but a new and empty Btrfs volume, this hypothetical attack would be a ton easier to do on LVM and mdadm raid because they have a tiny amount of metadata to spoof compared to a Btrfs volume with even a little bit of data on it. I think this concern is overblown. > If then btrfs would select any of these, it may also select the wrong > one - ro or rw, this may likely lead to problems. >> dd is not a copy operation. It's creating a 2nd original. You don't >> end up with an original and a copy (or clone). A copy or clone has >> some distinguishing difference. Volume UUID is used throughout Btrfs >> metadata, it's not just in the superblocks. Changing volume UUID >> requires a rewrite of all metadata. This is inefficient for two >> reasons: one dd copies unused sectors; two it copies metadata that >> will have to be completely rewritten by btrfstune to change volume >> UUID; and also the subvolume UUIDs aren't changed, so it's an >> incomplete solution that has problems (see other threads). > Well dd is surely not the only thing that can be used to create a clone > (i.e. a bitwise identical copy - I guess we don't really care which is > the "original" and which are the "clones", or whether these are "2nd > originals). > We always just use it here as an example for scenarios in which bitwise > identical copies are created. I'm suggesting bitwise identical copies being created is not what is wanted most of the time, except in edge cases. > > And even if internally it's a big thing, from the user's PoV, changing > the UUID is pretty simple (I guess that's what S.J. meant). > > >> If your workflow requires making an exact copy (for the shelf or for >> an emergency) then dd might be OK. But most often it's used because >> it's been easy, not because it's a good practice. > Ufff.. I wouldn't got that far to call something here bad or good > practice. It's not just bad practice, it's sufficiently sloppy that it's very nearly user sabotage. That this is due to innocent ignorance, and a long standing practice that's bad advice being handed down from previous generations doesn't absolve the practice and mean we should invent esoteric work arounds for what is not a good practice. We have all sorts of exhibits why it's not a good idea. > At least, I do not see any reason to call it a bad practice, except > that systems got over time much more complex and haven't dealt properly > with the problems that can occur by using dd. The lack of maturity in tools to make it just as easy, or easier, and faster, to make a *data* bitwise identical copy, that preserves the intent and integrity of UUID by ensuring there aren't duplicates of them floating around, as well as profile reshaping on the fly, as well as a means to account for format changes, etc is a completely reasonable excuse for continuing to use dd - but it's still suboptimal which is what I mean by bad idea. > Again, I don't demand magical "solutions" (i.e. the btrfs or LVM people > getting code into all dd like tools, so that these auto-detect when the > duplicate such data and auto-change the UUIDs)... they just should > handle the situations gracefully. I disagree. It was due to the rudimentary nature of earlier filesystems' metadata paradigm that it worked. That's no longer the case. Sure, the kernel code should get smarter about refusing to mount in ambiguous cases, so that a file system isn't nerfed. That shouldn't happen. But we also need to get away from this idea that dd is actually an appropriate tool for making a file system copy. > > >> Note that Btrfs is >> not unique, XFS v5 does a very similar thing with volume UUID as >> well, >> and resulted in this change: >> http://oss.sgi.com/pipermail/xfs/2015-April/041267.html > Do you mean that xfs may suffer from the same issues that we're talking > about here? If so, one should probably give them a notice. They're aware, that's why xfs_db had the option to change the UUID in the first place. And the XFS kernel code knows not to mount a 2nd instance of a volume UUID. But it doesn't support multiple devices, so it's no where near as prone to problems in this area. If you're using LVM snapshots, the duplicate UUID problem certainly comes up. While there is a 'nouuid' mount option for XFS, I have no idea what problems this might cause for V5 filesystems. -- Chris Murphy -- To
Re: attacking btrfs filesystems via UUID collisions?
On Wed, 2015-12-09 at 22:48 +0100, S.J. wrote: > > 3. Some way to fail gracefully, when there's ambiguity that cannot > > be > > resolved. Once there are duplicate devs (dd or lvm snapshots, etc) > > then there's simply no way to resolve the ambiguity automatically, > > and > > the volume should just refuse to rw mount until the user resolves > > the > > ambiguity. I think it's OK to fallback to ro mount (maybe) by > > default > > in such a case rather than totally fail to mount. > About 3: > RO fallback for the second device/partitions is not good. > It won't stop confusing the two partitions, and even if both are RO, > thinking it's ok to read and then reading the wrong data is bad. Adding my two cents about that, just to emphasise it, even though S.J. already covered it: Even romounts, if anything is ambiguous, are evil: Even if the filesystem itself wouldn't be destroyed by that, it could mean that bogus data (or even evil data by an attacker) shows up in the system that is then used and causes damage by being used. In the "accidental" scenario, data from the wrong device could e.g. contain outdated binaries, that still have security holes, or they could contain lists of datasets to be deleted by some software, but since being outdated or simply garbage, the wrong data could be deleted. In the "attacker" scenario,... well again as above, old binaries could get used, or garbage data injected into the system (even if ro) could make it compromised or be used for DoS. In general, the longer I think about it, the more I come to the conclusion that any form of auto activation (mounting, assembling, rebuilding, etc.) is kind of dangerous... (see below) And this applies in general, not just when using UUIDs,... but since in btrfs UUIDs are the main criterion for selecting/auto-assembling these devices, it's what applies for us here. We have several stages, where wrong devices could be picked up and lead to damage (either accidentally or as part of a tricky attack): 1) When the system boots, i.e. replacing parts of the system (e.g. root fs) itself. There's little we can do here in general (regardless of UUID, labels or device=/dev/sda,/dev/sdb). If an attacker can exchange one of the devices, he may do evil things. That's bad of course, but I think "fixing" it, is beyond the scope of btrfs. - If e.g. the ATM has an unsecured BIOS/UEFI/bootloader and allows the attacker easily to access these and select which device to boot from,... well than I feel no sorry for the owner (their fault). - If they configure their grub/initrd/etc. to boot LABEL/UUID... well that's certainly handy, but it's also stupid if these boots happen unattended, and there is an way around it (specify the device paths or e.g. /dev/sda)... if the HDDs are properly secured by steel, and attacker cannot use the possibly more easily accessible USB bus. - Another way to partially help here is: use disk dm-crypt and boot/assemble your system based on the dm-crypt devices. E.g. boot from the multi-device-btrfs device=/dev/mapper/crypt1,/dev/mapper/crypt2 and so on. As long as the kernel and initrd (which does all that) are secure (which is assumed here), then even when the attacker manages to replace one of the devices, it wouldn't help him, as the he couldn't present a device for which a dm-crypt mapping can be set up (unless he has the keys, but then game's over anyway) => Long story short, if the system boots unattended, then people should not use UUID/LABEL to select the device, if they do, their fault, not btrfs scope. If boots are attended, there's anyway not problem. => IHMO, this conceptually "fixes" (in the sense, that there's nothing to do specifically from the btrfs side) the possible problems of such a system being booted, with an attacker having replaced or added some devices to it (especially when unattended). And also the situation, that such system was left back, in an incomplete multi-device state (i.e. left back unattended with a degraded RAID) In other words, I think any problems, resulting of auto- assembly/activation/mounting, based on UUIDs/device-scanning/etc. that affect the valid system becoming running (i.e. booting) are beyond our scope here. Yes there are problems, but one can at least try to avoid them, by using dm-crypt or device paths instead of LABELS/UUIDs, and properly securing (i.e. steel and so on) the system disks, mainboard, bios, etc. So the remaining issues are those we discussed already before: The system runs already. 1) Further devices show up with colliding UUIDs /device IDs. a) Either none of them are used (mounted, fsck, etc.) already. b) Or some of them are used (mounted, fsck, etc.) already. 2) Further devices show up, that have no UUID / device ID collisions, but that may fit to an already used multi-device btrfs. E.g. in
Re: attacking btrfs filesystems via UUID collisions?
On 12/11/15 4:21 PM, Christoph Anton Mitterer wrote: >> Note that Btrfs is >> > not unique, XFS v5 does a very similar thing with volume UUID as >> > well, >> > and resulted in this change: >> > http://oss.sgi.com/pipermail/xfs/2015-April/041267.html > Do you mean that xfs may suffer from the same issues that we're talking > about here? If so, one should probably give them a notice. That was disabled temporarily because changing the fs UUID meant that every piece of checksummed metadata with an embedded UUID would then mismatch. It was fixed (re-allowed) with ce748ea xfs: create new metadata UUID field and incompat flag in the kernel and 9c4e12f xfsprogs: Add new sb_meta_uuid field, update userspace tools to manipulate it in xfsprogs. -Eric -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: attacking btrfs filesystems via UUID collisions?
A bit more about the dd-is-bad-topic: IMHO it doesn't matter at all. a) For this specific problem here, fixing a security problem automatically fixes the risk of data corruption because careless cloning+mounting (without UUID adjustments) too. So, if the user likes to use dd with its disadvantages, like waiting hours to copy lots of free space, and bad practice, etc.etc., why should it concern the Btrfs developers and/or us here? b) At wider scope; while Btrfs is more complex than Xfs etc., currently there is no other reason why things could go bad when dd'ing something. As long as this holds, is there really a place in the official Btrfs documentation for telling the users "dd is bad [practice]"? ... -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Will "btrfs check --repair" fix the mounting problem?
Btrfs crashes in few seconds after mounting RW. If it's important: the volume was converted from ext4. "ext2_saved" subvolume still presents. dmesg: [ 625.998387] BTRFS info (device sda1): disk space caching is enabled [ 625.998392] BTRFS: has skinny extents [ 627.727708] BTRFS: checking UUID tree [ 708.514128] [ cut here ] [ 708.514161] WARNING: CPU: 1 PID: 2263 at fs/btrfs/extent-tree.c:6255 __btrfs_free_extent.isra.68+0x8c8/0xd70 [btrfs]() [ 708.514164] Modules linked in: bnep bluetooth rfkill ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_broute bridge ebtable_filter ebtable_nat ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_raw ip6table_security ip6table_mangle ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_raw iptable_security iptable_mangle gpio_ich coretemp kvm_intel kvm iTCO_wdt iTCO_vendor_support snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device lpc_ich snd_pcm snd_timer ppdev snd i2c_i801 mei_me mei soundcore parport_pc parport shpchp tpm_infineon tpm_tis tpm acpi_cpufreq nfsd auth_rpcgss nfs_acl lockd grace isofs squashfs btrfs xor raid6_pq i915 hid_logitech_hidpp [ 708.514277] 8021q garp stp video llc mrp i2c_algo_bit drm_kms_helper r8169 uas crc32c_intel drm serio_raw mii hid_logitech_dj usb_storage scsi_dh_rdac scsi_dh_emc scsi_dh_alua sunrpc loop [ 708.514311] CPU: 1 PID: 2263 Comm: btrfs-transacti Not tainted 4.2.3-300.fc23.x86_64 #1 [ 708.514315] Hardware name: MSI MS-7636/H55M-P31(MS-7636) , BIOS V1.9 09/14/2010 [ 708.514319] f50458a6 880066b03ad8 81771fca [ 708.514326] 880066b03b18 8109e4a6 [ 708.514332] 0002 00252f595000 fffe [ 708.514338] Call Trace: [ 708.514349] [] dump_stack+0x45/0x57 [ 708.514359] [] warn_slowpath_common+0x86/0xc0 [ 708.514365] [] warn_slowpath_null+0x1a/0x20 [ 708.514391] [] __btrfs_free_extent.isra.68+0x8c8/0xd70 [btrfs] [ 708.514429] [] ? find_ref_head+0x5a/0x80 [btrfs] [ 708.514456] [] __btrfs_run_delayed_refs+0x998/0x1080 [btrfs] [ 708.514477] [] btrfs_run_delayed_refs.part.73+0x74/0x270 [btrfs] [ 708.514496] [] btrfs_run_delayed_refs+0x15/0x20 [btrfs] [ 708.514518] [] btrfs_commit_transaction+0x56/0xad0 [btrfs] [ 708.514541] [] transaction_kthread+0x214/0x230 [btrfs] [ 708.514564] [] ? btrfs_cleanup_transaction+0x500/0x500 [btrfs] [ 708.514569] [] kthread+0xd8/0xf0 [ 708.514574] [] ? kthread_worker_fn+0x160/0x160 [ 708.514581] [] ret_from_fork+0x3f/0x70 [ 708.514585] [] ? kthread_worker_fn+0x160/0x160 [ 708.514588] ---[ end trace 673f3bf2295a ]--- [ 708.514594] BTRFS info (device sda1): leaf 535035904 total ptrs 204 free space 4451 [ 708.514598] item 0 key (159696797696 169 0) itemoff 16250 itemsize 33 [ 708.514601] extent refs 1 gen 21134 flags 2 [ 708.514604] tree block backref root 2 [ 708.514609] item 1 key (159696830464 169 1) itemoff 16217 itemsize 33 [ 708.514612] extent refs 1 gen 21134 flags 2 [ 708.514615] tree block backref root 2 [ 708.514619] item 2 key (159696846848 169 0) itemoff 16184 itemsize 33 *** a lot of similar messages *** [ 708.516923] item 203 key (159711268864 169 0) itemoff 9551 itemsize 33 [ 708.516927] extent refs 1 gen 21082 flags 2 [ 708.516930] tree block backref root 384 [ 708.516937] BTRFS error (device sda1): unable to find ref byte nr 159708172288 parent 0 root 385 owner 2 offset 0 [ 708.516944] [ cut here ] [ 708.516975] WARNING: CPU: 1 PID: 2263 at fs/btrfs/extent-tree.c:6261 __btrfs_free_extent.isra.68+0x92f/0xd70 [btrfs]() [ 708.516979] BTRFS: Transaction aborted (error -2) [ 708.516982] Modules linked in: bnep bluetooth rfkill ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_broute bridge ebtable_filter ebtable_nat ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_raw ip6table_security ip6table_mangle ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_raw iptable_security iptable_mangle gpio_ich coretemp kvm_intel kvm iTCO_wdt iTCO_vendor_support snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device lpc_ich snd_pcm snd_timer ppdev snd i2c_i801 mei_me mei soundcore parport_pc parport shpchp tpm_infineon tpm_tis tpm acpi_cpufreq nfsd auth_rpcgss nfs_acl lockd grace isofs squashfs btrfs xor raid6_pq i915 hid_logitech_hidpp [ 708.517075] 8021q garp stp video llc mrp i2c_algo_bit drm_kms_helper r8169 uas crc32c_intel drm serio_raw mii hid_logitech_dj usb_storage scsi_dh_rdac scsi_dh_emc scsi_dh_alua sunrpc loop [ 708.517108] CPU:
Transaction aborted (error -17) during balance
Hi, during a balance on my main notebook, I have received the following call trace: [ 1545.229672] [ cut here ] [ 1545.229688] WARNING: CPU: 4 PID: 5545 at /build/linux-eGTGmU/linux-4.3/fs/btrfs/extent-tree.c:2093 __btrfs_inc_extent_ref.isra.52+0x20e/0x280 [btrfs]() [ 1545.229689] BTRFS: Transaction aborted (error -17) [ 1545.229690] Modules linked in: ctr ccm tun rfcomm cpufreq_userspace binfmt_misc cpufreq_stats cpufreq_powersave cpufreq_conservative nf_conntrack_netlink nfnetlink bnep ip6table_filter ip6_tables xt_TCPMSS xt_tcpudp iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter ip_tables x_tables bridge stp llc joydev arc4 iTCO_wdt iwldvm iTCO_vendor_support mac80211 snd_hda_codec_conexant intel_rapl snd_hda_codec_generic iosf_mbi x86_pkg_temp_thermal btusb intel_powerclamp btrtl snd_hda_intel iwlwifi btbcm kvm_intel snd_hda_codec btintel kvm snd_hda_core psmouse bluetooth snd_hwdep snd_pcm_oss pcspkr serio_raw i2c_i801 sg cfg80211 snd_mixer_oss lpc_ich snd_pcm mfd_core snd_timer mei_me shpchp mei thinkpad_acpi nvram [ 1545.229718] tpm_tis snd tpm soundcore rfkill evdev battery ac processor coretemp loop drbd lru_cache libcrc32c parport_pc ppdev lp parport autofs4 btrfs xor raid6_pq ext4 crc16 mbcache jbd2 algif_skcipher af_alg dm_crypt dm_mod md_mod hid_generic hid_logitech_hidpp hid_logitech_dj usbhid hid sd_mod uas usb_storage crct10dif_pclmul crc32_pclmul crc32c_intel jitterentropy_rng sha256_ssse3 sha256_generic hmac drbg ansi_cprng aesni_intel aes_x86_64 lrw gf128mul glue_helper i915 ahci ablk_helper cryptd libahci sdhci_pci i2c_algo_bit libata ehci_pci drm_kms_helper sdhci ehci_hcd scsi_mod mmc_core e1000e usbcore ptp usb_common drm pps_core thermal wmi video button [ 1545.229747] CPU: 4 PID: 5545 Comm: kworker/u16:1 Not tainted 4.3.0-trunk-amd64 #1 Debian 4.3-1~exp2 [ 1545.229747] Hardware name: LENOVO 4240CTO/4240CTO, BIOS 8AET63WW (1.43 ) 05/08/2013 [ 1545.229758] Workqueue: btrfs-extent-refs btrfs_extent_refs_helper [btrfs] [ 1545.229760] a0627250 812c5319 88020dc23ba0 8106ebcd [ 1545.229761] 880406146000 88020dc23bf0 8803c90b9410 [ 1545.229762] 0106 8106ec4c a0627420 0020 [ 1545.229764] Call Trace: [ 1545.229768] [] ? dump_stack+0x40/0x57 [ 1545.229771] [] ? warn_slowpath_common+0x7d/0xb0 [ 1545.229772] [] ? warn_slowpath_fmt+0x4c/0x50 [ 1545.229778] [] ? insert_tree_block_ref+0x49/0x60 [btrfs] [ 1545.229783] [] ? __btrfs_inc_extent_ref.isra.52+0x20e/0x280 [btrfs] [ 1545.229789] [] ? __btrfs_run_delayed_refs+0xc47/0x1050 [btrfs] [ 1545.229792] [] ? sched_clock+0x5/0x10 [ 1545.229795] [] ? check_preempt_curr+0x50/0x90 [ 1545.229797] [] ? ttwu_do_wakeup+0x14/0xc0 [ 1545.229803] [] ? btrfs_run_delayed_refs+0x78/0x2a0 [btrfs] [ 1545.229808] [] ? delayed_ref_async_start+0x32/0x80 [btrfs] [ 1545.229816] [] ? btrfs_scrubparity_helper+0xc8/0x260 [btrfs] [ 1545.229818] [] ? process_one_work+0x19f/0x3d0 [ 1545.229819] [] ? worker_thread+0x4d/0x450 [ 1545.229821] [] ? process_one_work+0x3d0/0x3d0 [ 1545.229822] [] ? kthread+0xbd/0xe0 [ 1545.229824] [] ? kthread_create_on_node+0x170/0x170 [ 1545.229827] [] ? ret_from_fork+0x3f/0x70 [ 1545.229829] [] ? kthread_create_on_node+0x170/0x170 [ 1545.229830] ---[ end trace 6671e30ac2882b40 ]--- [ 1545.229832] BTRFS: error (device dm-11) in __btrfs_inc_extent_ref:2093: errno=-17 Object already exists [ 1545.229834] BTRFS info (device dm-11): forced readonly [ 1545.229836] BTRFS: error (device dm-11) in btrfs_run_delayed_refs:2851: errno=-17 Object already exists I have been trying to balance this filesystem for the better part of the afternoon, with numerous freezes of my notebook. I was able to finish the balance by not doing anything on the notebook while the balance was running. I then proceeded to initiate a second rebalance of the same filesystem "just to be sure", which led to a read-only btrfs and me at least being able to obtain this trace. This is a distribution kernel, I have debug symbols installed after this log extrct was obtained. Is there a tool which can help to make this trace useable? Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: new oops in 4.4.0-rc4
On 12/11/2015 09:35 AM, Chris Mason wrote: On Thu, Dec 10, 2015 at 10:36:17AM -0600, Jon Christopherson wrote: Dave Jones sent in a report about this with trinity too, I'm digging in today. Since you can trigger this reliably, what was the last known-good kernel for you? -chris Hello, My last known good kernel was from Dec 5 04:00 CST: Linux version 4.4.0-120-generic (r...@enigma.jons.org) (gcc version 5.2.1 20151124 (Ubuntu 5.2.1-1ubuntu1~14.04) ) #201512050400 SMP Sat Dec 5 03:54:25 CST 2015 I will check over the commits from then till now as well. Regards, Jon Christopherson -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Will "btrfs check --repair" fix the mounting problem?
On Fri, Dec 11, 2015 at 10:50 AM, Ivan Sizovwrote: > Btrfs crashes in few seconds after mounting RW. > If it's important: the volume was converted from ext4. "ext2_saved" > subvolume still presents. > > dmesg: > [ 625.998387] BTRFS info (device sda1): disk space caching is enabled > [ 625.998392] BTRFS: has skinny extents > [ 627.727708] BTRFS: checking UUID tree > [ 708.514128] [ cut here ] > [ 708.514161] WARNING: CPU: 1 PID: 2263 at fs/btrfs/extent-tree.c:6255 > __btrfs_free_extent.isra.68+0x8c8/0xd70 [btrfs]() > [ 708.514164] Modules linked in: bnep bluetooth rfkill ip6t_rpfilter > ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_broute bridge ebtable_filter > ebtable_nat ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 > nf_nat_ipv6 ip6table_raw ip6table_security ip6table_mangle ip6table_filter > ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat > nf_conntrack iptable_raw iptable_security iptable_mangle gpio_ich coretemp > kvm_intel kvm iTCO_wdt iTCO_vendor_support snd_hda_codec_realtek > snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep > snd_seq snd_seq_device lpc_ich snd_pcm snd_timer ppdev snd i2c_i801 mei_me > mei soundcore parport_pc parport shpchp tpm_infineon tpm_tis tpm acpi_cpufreq > nfsd auth_rpcgss nfs_acl lockd grace isofs squashfs btrfs xor raid6_pq i915 > hid_logitech_hidpp > [ 708.514277] 8021q garp stp video llc mrp i2c_algo_bit drm_kms_helper > r8169 uas crc32c_intel drm serio_raw mii hid_logitech_dj usb_storage > scsi_dh_rdac scsi_dh_emc scsi_dh_alua sunrpc loop > [ 708.514311] CPU: 1 PID: 2263 Comm: btrfs-transacti Not tainted > 4.2.3-300.fc23.x86_64 #1 > [ 708.514315] Hardware name: MSI MS-7636/H55M-P31(MS-7636) , BIOS V1.9 > 09/14/2010 > [ 708.514319] f50458a6 880066b03ad8 > 81771fca > [ 708.514326] 880066b03b18 > 8109e4a6 > [ 708.514332] 0002 00252f595000 fffe > > [ 708.514338] Call Trace: > [ 708.514349] [] dump_stack+0x45/0x57 > [ 708.514359] [] warn_slowpath_common+0x86/0xc0 > [ 708.514365] [] warn_slowpath_null+0x1a/0x20 > [ 708.514391] [] __btrfs_free_extent.isra.68+0x8c8/0xd70 > [btrfs] > [ 708.514429] [] ? find_ref_head+0x5a/0x80 [btrfs] > [ 708.514456] [] __btrfs_run_delayed_refs+0x998/0x1080 > [btrfs] > [ 708.514477] [] > btrfs_run_delayed_refs.part.73+0x74/0x270 [btrfs] > [ 708.514496] [] btrfs_run_delayed_refs+0x15/0x20 [btrfs] > [ 708.514518] [] btrfs_commit_transaction+0x56/0xad0 > [btrfs] > [ 708.514541] [] transaction_kthread+0x214/0x230 [btrfs] > [ 708.514564] [] ? btrfs_cleanup_transaction+0x500/0x500 > [btrfs] > [ 708.514569] [] kthread+0xd8/0xf0 > [ 708.514574] [] ? kthread_worker_fn+0x160/0x160 > [ 708.514581] [] ret_from_fork+0x3f/0x70 > [ 708.514585] [] ? kthread_worker_fn+0x160/0x160 > [ 708.514588] ---[ end trace 673f3bf2295a ]--- > [ 708.514594] BTRFS info (device sda1): leaf 535035904 total ptrs 204 free > space 4451 > [ 708.514598] item 0 key (159696797696 169 0) itemoff 16250 itemsize 33 > [ 708.514601] extent refs 1 gen 21134 flags 2 > [ 708.514604] tree block backref root 2 > [ 708.514609] item 1 key (159696830464 169 1) itemoff 16217 itemsize 33 > [ 708.514612] extent refs 1 gen 21134 flags 2 > [ 708.514615] tree block backref root 2 > [ 708.514619] item 2 key (159696846848 169 0) itemoff 16184 itemsize 33 > > *** a lot of similar messages *** > > [ 708.516923] item 203 key (159711268864 169 0) itemoff 9551 itemsize 33 > [ 708.516927] extent refs 1 gen 21082 flags 2 > [ 708.516930] tree block backref root 384 > [ 708.516937] BTRFS error (device sda1): unable to find ref byte nr > 159708172288 parent 0 root 385 owner 2 offset 0 > [ 708.516944] [ cut here ] > [ 708.516975] WARNING: CPU: 1 PID: 2263 at fs/btrfs/extent-tree.c:6261 > __btrfs_free_extent.isra.68+0x92f/0xd70 [btrfs]() > [ 708.516979] BTRFS: Transaction aborted (error -2) > [ 708.516982] Modules linked in: bnep bluetooth rfkill ip6t_rpfilter > ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_broute bridge ebtable_filter > ebtable_nat ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 > nf_nat_ipv6 ip6table_raw ip6table_security ip6table_mangle ip6table_filter > ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat > nf_conntrack iptable_raw iptable_security iptable_mangle gpio_ich coretemp > kvm_intel kvm iTCO_wdt iTCO_vendor_support snd_hda_codec_realtek > snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep > snd_seq snd_seq_device lpc_ich snd_pcm snd_timer ppdev snd i2c_i801 mei_me > mei soundcore parport_pc parport shpchp tpm_infineon tpm_tis tpm acpi_cpufreq > nfsd auth_rpcgss nfs_acl lockd grace isofs squashfs
Re: [PATCH] Btrfs: fix transaction handle leak in balance
On Thu, Dec 10, 2015 at 11:16:27AM +, fdman...@kernel.org wrote: > From: Filipe Manana> > If we fail to allocate a new data chunk, we were jumping to the error path > without release the transaction handle we got before. Fix this by always > releasing it before doing the jump. Looks good. Reviewed-by: Liu Bo Thanks, -liubo > > Fixes: 2c9fe8355258 ("btrfs: Fix lost-data-profile caused by balance bg") > Signed-off-by: Filipe Manana > --- > fs/btrfs/volumes.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index 3e55e07..f5e5e20 100644 > --- a/fs/btrfs/volumes.c > +++ b/fs/btrfs/volumes.c > @@ -3548,12 +3548,11 @@ again: > > ret = btrfs_force_chunk_alloc(trans, chunk_root, > BTRFS_BLOCK_GROUP_DATA); > + btrfs_end_transaction(trans, chunk_root); > if (ret < 0) { > mutex_unlock(_info->delete_unused_bgs_mutex); > goto error; > } > - > - btrfs_end_transaction(trans, chunk_root); > chunk_reserved = 1; > } > > -- > 2.1.3 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: subvols and parents - how?
On Sat, 2015-12-12 at 03:32 +0100, Christoph Anton Mitterer wrote: > What's still missing now, IMHO, is: > - a guide when one should make subvole (e.g. keeping things on the > root > fs together, unless it's separate like /var/www is usually, but > /var/lib typically "corresponds" to a state of /etc and /usr. I just added that: https://btrfs.wiki.kernel.org/index.php/SysadminGuide#When_To_Make_Subvolumes Chris. smime.p7s Description: S/MIME cryptographic signature