Re: [PATCH 00/26] Btrfs: Add device replace code
On Tue, 13 Nov 2012 17:25:46 +0100, Bart Noordervliet wrote: Hi Stefan, I gave your patchset a whirl and it worked like a charm. Thanks a lot for your work. I was confused for a moment by the fact that the operation doesn't immediately resize btrfs to use all of the new device's available space. But after a bit of thought I suppose that is intentional to make it easy to return to a device of similar size as the original. Yes, replace and resize is an operation that intentionally requires two manual steps, since the step backwards, to shrink a device, can take a long time. One suggestion remains: I would prefer the operation to leave some record of what has happened in the system log. Other operations like device add and device delete do and I think it makes sense to have a log trail of such invasive operations on the filesystem. That's a good idea. I have now added such KERN_INFO log messages. Tested-by: Bart Noordervliet b...@noordervliet.net Thanks for testing and for your feedback :) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
problem with ceph and btrfs patch: set journal_info in async trans commit worker
Hello list, i wanted to try out ceph with latest vanilla kernel 3.7-rc5. I was seeing a massive performance degration. I see around 22x btrfs-endio-write processes every 10-20 seconds and they run a long time while consuming a massive amount of CPU. So my performance of 23.000 iops drops to an up and down of 23.000 iops to 0 - avg is now 2500 iops instead of 23.000. Git bisect shows me commit: e209db7ace281ca347b1ac699bf1fb222eac03fe Btrfs: set journal_info in async trans commit worker as the problematic patch. When i revert this one everything is fine again. Is this known? Greets, Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
kernel BUG at btrfs/scrub.c:638 (kernel 3.6.5)
Hi, I was testing a new HBA (lsi SAS2008 based) in combination with BTRFS and kernel 3.6.5 #mkfs.btrfs -m raid1 -d raid1 /dev/sdf /dev/sdg # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid2 size 298.09GB used 19.01GB path /dev/sdg devid1 size 298.09GB used 19.03GB path /dev/sdf Btrfs v0.19+ I was simulating a faulty disk by physical removing the disk and connecting again. After reconnecting the disk, the disk appeared again but I get some kernel BUG report in dmesg after running a scrub. [ 936.138067] Btrfs loaded [ 936.138252] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 1 transid 3 /dev/sdf [ 936.190574] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 2 transid 3 /dev/sdg [ 950.208483] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 1 transid 4 /dev/sdf [ 950.216385] btrfs: disk space caching is enabled [ 1079.577103] mpt2sas0: log_info(0x3003010a): originator(IOP), code(0x03), sub_code(0x010a) [ 1079.577151] mpt2sas0: log_info(0x3003010a): originator(IOP), code(0x03), sub_code(0x010a) [ 1079.577416] mpt2sas0: log_info(0x30030101): originator(IOP), code(0x03), sub_code(0x0101) = after disconnection of one disk [ 1253.17] sd 8:0:1:0: [sdg] Synchronizing SCSI cache [ 1253.42] sd 8:0:1:0: [sdg] [ 1253.44] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK [ 1253.444588] mpt2sas0: removing handle(0x000a), sas_addr(0x443322110600) testsan:/btrfs # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid1 size 298.09GB used 19.03GB path /dev/sdf *** Some devices missing Btrfs v0.19+ = after connecting the same disk again = it seems that the disk is now sdh instead of sdg, could be that I've connected the disk on another port of the HBA # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid2 size 298.09GB used 19.01GB path /dev/sdh devid1 size 298.09GB used 19.03GB path /dev/sdf Btrfs v0.19+ After running a scrub command I get now the following errors in dmesg: [ 1253.17] sd 8:0:1:0: [sdg] Synchronizing SCSI cache [ 1253.42] sd 8:0:1:0: [sdg] [ 1253.44] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK [ 1253.444588] mpt2sas0: removing handle(0x000a), sas_addr(0x443322110600) [ 1385.440298] scsi 8:0:2:0: Direct-Access ATA WDC WD3200AAJS-0 3E01 PQ: 0 ANSI: 6 [ 1385.440307] scsi 8:0:2:0: SATA: handle(0x000a), sas_addr(0x443322110600), phy(6), device_name(0x) [ 1385.440310] scsi 8:0:2:0: SATA: enclosure_logical_id(0x500605b0054dc1f0), slot(5) [ 1385.440415] scsi 8:0:2:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) [ 1385.440421] scsi 8:0:2:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 1385.440627] sd 8:0:2:0: Attached scsi generic sg0 type 0 [ 1385.441276] sd 8:0:2:0: [sdh] 625142448 512-byte logical blocks: (320 GB/298 GiB) [ 1385.444743] sd 8:0:2:0: [sdh] Write Protect is off [ 1385.444747] sd 8:0:2:0: [sdh] Mode Sense: 7f 00 10 08 [ 1385.445860] sd 8:0:2:0: [sdh] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 1385.464525] sdh: unknown partition table [ 1385.472633] sd 8:0:2:0: [sdh] Attached SCSI disk [ 1593.048743] [ cut here ] [ 1593.050188] kernel BUG at /usr/src/packages/BUILD/kernel-default-3.6.5/linux-3.6/fs/btrfs/scrub.c:638! [ 1593.051654] invalid opcode: [#1] SMP [ 1593.052712] Modules linked in: btrfs zlib_deflate libcrc32c af_packet cpufreq_conservative cpufreq_userspace cpufreq_powersave dm_mod snd_hda_codec_hdmi iTCO_wdt snd_hda_codec_realtek gpio_ich iTCO_vendor_support sg i2c_i801 acpi_cpufreq mperf coretemp serio_raw pcspkr sr_mod cdrom snd_hda_intel mei lpc_ich mfd_core e1000e kvm_intel kvm microcode snd_hda_codec snd_hwdep snd_pcm snd_timer snd usb_storage tpm_tis tpm hid_generic wmi usbhid soundcore snd_page_alloc tpm_bios edd autofs4 uhci_hcd ehci_hcd usbcore usb_common i915 drm_kms_helper drm i2c_algo_bit video button processor thermal_sys scsi_dh_hp_sw scsi_dh_rdac scsi_dh_alua scsi_dh_emc scsi_dh mpt2sas scsi_transport_sas raid_class ata_generic [ 1593.052712] CPU 2 [ 1593.052712] Pid: 2823, comm: btrfs-scrub-1 Not tainted 3.6.5-0-default #1 Acer Veriton M67WS/EQ45M [ 1593.052712] RIP: 0010:[a0526032] [a0526032] scrub_handle_errored_block+0x972/0x980 [btrfs] [ 1593.052712] RSP: 0018:88022d6c1ca0 EFLAGS: 00010246 [ 1593.052712] RAX: 0007 RBX: 88022d012800 RCX: 0001 [ 1593.052712] RDX: RSI: 88022d4a79a0 RDI: 88022d012800 [ 1593.052712] RBP: 88022d4a71f0 R08: 88022d6c R09: dead00100100 [ 1593.052712] R10: dead00200200 R11: 0001 R12: 0001 [ 1593.052712] R13: R14: 88022d4a7278 R15: [
[PATCH] Btrfs: cleanup for btrfs_btree_balance_dirty
- 'nr' is no more used.
- btrfs_btree_balance_dirty() and __btrfs_btree_balance_dirty() can share
a bunch of code.
Signed-off-by: Liu Bo bo.li@oracle.com
---
fs/btrfs/delayed-inode.c |5 +
fs/btrfs/disk-io.c | 29 ++---
fs/btrfs/disk-io.h |4 ++--
fs/btrfs/file.c |9 +++--
fs/btrfs/inode.c | 42 +++---
fs/btrfs/relocation.c| 22 ++
fs/btrfs/transaction.c |4 +---
7 files changed, 34 insertions(+), 81 deletions(-)
diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c
index 478f66b..52b2a88 100644
--- a/fs/btrfs/delayed-inode.c
+++ b/fs/btrfs/delayed-inode.c
@@ -1255,7 +1255,6 @@ static void btrfs_async_run_delayed_node_done(struct
btrfs_work *work)
struct btrfs_delayed_node *delayed_node = NULL;
struct btrfs_root *root;
struct btrfs_block_rsv *block_rsv;
- unsigned long nr = 0;
int need_requeue = 0;
int ret;
@@ -1316,11 +1315,9 @@ static void btrfs_async_run_delayed_node_done(struct
btrfs_work *work)
delayed_node);
mutex_unlock(delayed_node-mutex);
- nr = trans-blocks_used;
-
trans-block_rsv = block_rsv;
btrfs_end_transaction_dmeta(trans, root);
- __btrfs_btree_balance_dirty(root, nr);
+ btrfs_btree_balance_dirty_nodelay(root);
free_path:
btrfs_free_path(path);
out:
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 7cda519..b1f38e5 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -3399,7 +3399,8 @@ void btrfs_mark_buffer_dirty(struct extent_buffer *buf)
}
}
-void btrfs_btree_balance_dirty(struct btrfs_root *root, unsigned long nr)
+static void __btrfs_btree_balance_dirty(struct btrfs_root *root,
+ int flush_delayed)
{
/*
* looks as though older kernels can get into trouble with
@@ -3411,7 +3412,8 @@ void btrfs_btree_balance_dirty(struct btrfs_root *root,
unsigned long nr)
if (current-flags PF_MEMALLOC)
return;
- btrfs_balance_delayed_items(root);
+ if (flush_delayed)
+ btrfs_balance_delayed_items(root);
num_dirty = root-fs_info-dirty_metadata_bytes;
@@ -3422,25 +3424,14 @@ void btrfs_btree_balance_dirty(struct btrfs_root *root,
unsigned long nr)
return;
}
-void __btrfs_btree_balance_dirty(struct btrfs_root *root, unsigned long nr)
+void btrfs_btree_balance_dirty(struct btrfs_root *root)
{
- /*
-* looks as though older kernels can get into trouble with
-* this code, they end up stuck in balance_dirty_pages forever
-*/
- u64 num_dirty;
- unsigned long thresh = 32 * 1024 * 1024;
-
- if (current-flags PF_MEMALLOC)
- return;
-
- num_dirty = root-fs_info-dirty_metadata_bytes;
+ __btrfs_btree_balance_dirty(root, 1);
+}
- if (num_dirty thresh) {
- balance_dirty_pages_ratelimited_nr(
- root-fs_info-btree_inode-i_mapping, 1);
- }
- return;
+void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root)
+{
+ __btrfs_btree_balance_dirty(root, 0);
}
int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid)
diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h
index 2025a91..305c33e 100644
--- a/fs/btrfs/disk-io.h
+++ b/fs/btrfs/disk-io.h
@@ -62,8 +62,8 @@ struct btrfs_root *btrfs_read_fs_root_no_radix(struct
btrfs_root *tree_root,
struct btrfs_root *btrfs_read_fs_root_no_name(struct btrfs_fs_info *fs_info,
struct btrfs_key *location);
int btrfs_cleanup_fs_roots(struct btrfs_fs_info *fs_info);
-void btrfs_btree_balance_dirty(struct btrfs_root *root, unsigned long nr);
-void __btrfs_btree_balance_dirty(struct btrfs_root *root, unsigned long nr);
+void btrfs_btree_balance_dirty(struct btrfs_root *root);
+void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root);
void btrfs_free_fs_root(struct btrfs_fs_info *fs_info, struct btrfs_root
*root);
void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index 9ab1bed..37940cd 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -1349,7 +1349,7 @@ static noinline ssize_t __btrfs_buffered_write(struct
file *file,
balance_dirty_pages_ratelimited_nr(inode-i_mapping,
dirty_pages);
if (dirty_pages (root-leafsize PAGE_CACHE_SHIFT) + 1)
- btrfs_btree_balance_dirty(root, 1);
+ btrfs_btree_balance_dirty(root);
pos += copied;
num_written += copied;
@@ -1803,7 +1803,6 @@ static int btrfs_punch_hole(struct inode *inode,
[PATCH] Btrfs: protect devices list with its mutex
Since we've kill the bigger one volume_mutex, we need to add devices
list mutex back.
Signed-off-by: Liu Bo bo.li@oracle.com
---
fs/btrfs/volumes.c |9 +
1 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 0f5ebb7..a6a7596 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1678,16 +1678,17 @@ int btrfs_init_new_device(struct btrfs_root *root, char
*device_path)
filemap_write_and_wait(bdev-bd_inode-i_mapping);
devices = root-fs_info-fs_devices-devices;
- /*
-* we have the volume lock, so we don't need the extra
-* device list mutex while reading the list here.
-*/
+
+ mutex_lock(root-fs_info-fs_devices-device_list_mutex);
list_for_each_entry(device, devices, dev_list) {
if (device-bdev == bdev) {
ret = -EEXIST;
+ mutex_unlock(
+ root-fs_info-fs_devices-device_list_mutex);
goto error;
}
}
+ mutex_unlock(root-fs_info-fs_devices-device_list_mutex);
device = kzalloc(sizeof(*device), GFP_NOFS);
if (!device) {
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at btrfs/scrub.c:638 (kernel 3.6.5)
On Wed, 14 Nov 2012 15:27:28 +0100, Joeri Vanthienen wrote: Hi, I was testing a new HBA (lsi SAS2008 based) in combination with BTRFS and kernel 3.6.5 #mkfs.btrfs -m raid1 -d raid1 /dev/sdf /dev/sdg # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid2 size 298.09GB used 19.01GB path /dev/sdg devid1 size 298.09GB used 19.03GB path /dev/sdf Btrfs v0.19+ I was simulating a faulty disk by physical removing the disk and connecting again. After reconnecting the disk, the disk appeared again but I get some kernel BUG report in dmesg after running a scrub. [ 936.138067] Btrfs loaded [ 936.138252] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 1 transid 3 /dev/sdf [ 936.190574] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 2 transid 3 /dev/sdg [ 950.208483] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 1 transid 4 /dev/sdf [ 950.216385] btrfs: disk space caching is enabled [ 1079.577103] mpt2sas0: log_info(0x3003010a): originator(IOP), code(0x03), sub_code(0x010a) [ 1079.577151] mpt2sas0: log_info(0x3003010a): originator(IOP), code(0x03), sub_code(0x010a) [ 1079.577416] mpt2sas0: log_info(0x30030101): originator(IOP), code(0x03), sub_code(0x0101) = after disconnection of one disk [ 1253.17] sd 8:0:1:0: [sdg] Synchronizing SCSI cache [ 1253.42] sd 8:0:1:0: [sdg] [ 1253.44] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK [ 1253.444588] mpt2sas0: removing handle(0x000a), sas_addr(0x443322110600) testsan:/btrfs # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid1 size 298.09GB used 19.03GB path /dev/sdf *** Some devices missing Btrfs v0.19+ = after connecting the same disk again = it seems that the disk is now sdh instead of sdg, could be that I've connected the disk on another port of the HBA # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid2 size 298.09GB used 19.01GB path /dev/sdh devid1 size 298.09GB used 19.03GB path /dev/sdf Btrfs v0.19+ After running a scrub command I get now the following errors in dmesg: [ 1253.17] sd 8:0:1:0: [sdg] Synchronizing SCSI cache [ 1253.42] sd 8:0:1:0: [sdg] [ 1253.44] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK [ 1253.444588] mpt2sas0: removing handle(0x000a), sas_addr(0x443322110600) [ 1385.440298] scsi 8:0:2:0: Direct-Access ATA WDC WD3200AAJS-0 3E01 PQ: 0 ANSI: 6 [ 1385.440307] scsi 8:0:2:0: SATA: handle(0x000a), sas_addr(0x443322110600), phy(6), device_name(0x) [ 1385.440310] scsi 8:0:2:0: SATA: enclosure_logical_id(0x500605b0054dc1f0), slot(5) [ 1385.440415] scsi 8:0:2:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) [ 1385.440421] scsi 8:0:2:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 1385.440627] sd 8:0:2:0: Attached scsi generic sg0 type 0 [ 1385.441276] sd 8:0:2:0: [sdh] 625142448 512-byte logical blocks: (320 GB/298 GiB) [ 1385.444743] sd 8:0:2:0: [sdh] Write Protect is off [ 1385.444747] sd 8:0:2:0: [sdh] Mode Sense: 7f 00 10 08 [ 1385.445860] sd 8:0:2:0: [sdh] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 1385.464525] sdh: unknown partition table [ 1385.472633] sd 8:0:2:0: [sdh] Attached SCSI disk [ 1593.048743] [ cut here ] [ 1593.050188] kernel BUG at /usr/src/packages/BUILD/kernel-default-3.6.5/linux-3.6/fs/btrfs/scrub.c:638! [ 1593.051654] invalid opcode: [#1] SMP [ 1593.052712] Modules linked in: btrfs zlib_deflate libcrc32c af_packet cpufreq_conservative cpufreq_userspace cpufreq_powersave dm_mod snd_hda_codec_hdmi iTCO_wdt snd_hda_codec_realtek gpio_ich iTCO_vendor_support sg i2c_i801 acpi_cpufreq mperf coretemp serio_raw pcspkr sr_mod cdrom snd_hda_intel mei lpc_ich mfd_core e1000e kvm_intel kvm microcode snd_hda_codec snd_hwdep snd_pcm snd_timer snd usb_storage tpm_tis tpm hid_generic wmi usbhid soundcore snd_page_alloc tpm_bios edd autofs4 uhci_hcd ehci_hcd usbcore usb_common i915 drm_kms_helper drm i2c_algo_bit video button processor thermal_sys scsi_dh_hp_sw scsi_dh_rdac scsi_dh_alua scsi_dh_emc scsi_dh mpt2sas scsi_transport_sas raid_class ata_generic [ 1593.052712] CPU 2 [ 1593.052712] Pid: 2823, comm: btrfs-scrub-1 Not tainted 3.6.5-0-default #1 Acer Veriton M67WS/EQ45M [ 1593.052712] RIP: 0010:[a0526032] [a0526032] scrub_handle_errored_block+0x972/0x980 [btrfs] [ 1593.052712] RSP: 0018:88022d6c1ca0 EFLAGS: 00010246 [ 1593.052712] RAX: 0007 RBX: 88022d012800 RCX: 0001 [ 1593.052712] RDX: RSI: 88022d4a79a0 RDI: 88022d012800 [ 1593.052712] RBP: 88022d4a71f0 R08: 88022d6c R09: dead00100100
fs/btrfs/volumes.c:1505:1: warning: label 'error_close' defined but not used
tree: git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git
master
head: 0cb8658e96f1ba2ff2d730224cb31e231edd318d
commit: 7017ac470ed5b029bc683883714a7f4a4ed34c33 Btrfs: move some common code
into a subfunction
date: 2 days ago
config: make ARCH=x86_64 allyesconfig
All warnings:
fs/btrfs/volumes.c: In function 'btrfs_rm_device':
fs/btrfs/volumes.c:1505:1: warning: label 'error_close' defined but not used
[-Wunused-label]
vim +1505 +/error_close fs/btrfs/volumes.c
2b82032c Yan Zheng 2008-11-17 1489 * at this point, the device is
zero sized. We want to
2b82032c Yan Zheng 2008-11-17 1490 * remove it from the devices
list and zero out the old super
2b82032c Yan Zheng 2008-11-17 1491 */
1f78160c Xiao Guangrong 2011-04-20 1492if (clear_super) {
dfe25020 Chris Mason2008-05-13 1493/* make sure this
device isn't detected as part of
dfe25020 Chris Mason2008-05-13 1494 * the FS anymore
dfe25020 Chris Mason2008-05-13 1495 */
dfe25020 Chris Mason2008-05-13 1496
memset(disk_super-magic, 0, sizeof(disk_super-magic));
dfe25020 Chris Mason2008-05-13 1497set_buffer_dirty(bh);
dfe25020 Chris Mason2008-05-13 1498sync_dirty_buffer(bh);
dfe25020 Chris Mason2008-05-13 1499}
a061fc8d Chris Mason2008-05-07 1500
a061fc8d Chris Mason2008-05-07 1501ret = 0;
a061fc8d Chris Mason2008-05-07 1502
a061fc8d Chris Mason2008-05-07 1503 error_brelse:
a061fc8d Chris Mason2008-05-07 1504brelse(bh);
a061fc8d Chris Mason2008-05-07 @1505 error_close:
dfe25020 Chris Mason2008-05-13 1506if (bdev)
e525fd89 Tejun Heo 2010-11-13 1507blkdev_put(bdev,
FMODE_READ | FMODE_EXCL);
a061fc8d Chris Mason2008-05-07 1508 out:
a061fc8d Chris Mason2008-05-07 1509mutex_unlock(uuid_mutex);
a061fc8d Chris Mason2008-05-07 1510return ret;
9b3517e9 Ilya Dryomov 2011-02-15 1511 error_undo:
9b3517e9 Ilya Dryomov 2011-02-15 1512if (device-writeable) {
0c1daee0 Xiao Guangrong 2011-04-20 1513lock_chunks(root);
---
0-DAY kernel build testing backend Open Source Technology Center
Fengguang Wu, Yuanhan Liu Intel Corporation
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS, getting darn slower everyday
On 2012-10-07 15:19, Swâmi Petaramesh wrote: Hi again ;-) Le 07/10/2012 14:33, Alex a écrit : 1. Convert to a 16k or 32k leafsize. How should I do this ? Can I do this on a live FS, and isn't this going to double my on-disk used space (I have active snapshots...) 2. defragment (each non-trivial file) every now and again I believed that the autodefrag mount option was supposed to avoid the need for manual defrags...? Plus, with respect to snapshots, isn't this going to increase a lot my used disk space ? 3. Note that btrfs in kernel 3.6 (vs 3.5) does appear to be quite a bit quicker. I'm not sure that a bit quicker will be enough, when I see my BTRFS is currently about 8x slower than what ext4 user to... Here is a graph to reinforce what Alex said about performance after 3.5.x. This graph was posted in #btrfs on freenode. At the end of the graph, the poster said that he upgraded from a 3.5.x to a 3.6.x https://pipebreaker.pl/dump/loadl.png I noticed the same thing going from 3.5.0 to 3.7.0rc2; system usage is much lower. Before this I would often find huge lags and then check and see btrfs-cleaner, btrfs-transactionhelpersomething, etc. taking up 100% io. So if your system usage is high, this will help. Also possibly not relevant for you: on my desktop, (btrfs or ext4, with consumer disks) I find that deadline scheduler and nr_requests of 8 is vastly superior for reducing GUI lags. (default is scheduler cfq and 128 nr_requests). But for servers, I don't know what to recommend. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at btrfs/scrub.c:638 (kernel 3.6.5)
Sure Stefan. Thank you! Looking forward to the fix. On Wed, Nov 14, 2012 at 4:46 PM, Stefan Behrens sbehr...@giantdisaster.de wrote: On Wed, 14 Nov 2012 15:27:28 +0100, Joeri Vanthienen wrote: Hi, I was testing a new HBA (lsi SAS2008 based) in combination with BTRFS and kernel 3.6.5 #mkfs.btrfs -m raid1 -d raid1 /dev/sdf /dev/sdg # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid2 size 298.09GB used 19.01GB path /dev/sdg devid1 size 298.09GB used 19.03GB path /dev/sdf Btrfs v0.19+ I was simulating a faulty disk by physical removing the disk and connecting again. After reconnecting the disk, the disk appeared again but I get some kernel BUG report in dmesg after running a scrub. [ 936.138067] Btrfs loaded [ 936.138252] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 1 transid 3 /dev/sdf [ 936.190574] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 2 transid 3 /dev/sdg [ 950.208483] device fsid fe542409-7346-4ea1-af04-fd1765b6a1a2 devid 1 transid 4 /dev/sdf [ 950.216385] btrfs: disk space caching is enabled [ 1079.577103] mpt2sas0: log_info(0x3003010a): originator(IOP), code(0x03), sub_code(0x010a) [ 1079.577151] mpt2sas0: log_info(0x3003010a): originator(IOP), code(0x03), sub_code(0x010a) [ 1079.577416] mpt2sas0: log_info(0x30030101): originator(IOP), code(0x03), sub_code(0x0101) = after disconnection of one disk [ 1253.17] sd 8:0:1:0: [sdg] Synchronizing SCSI cache [ 1253.42] sd 8:0:1:0: [sdg] [ 1253.44] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK [ 1253.444588] mpt2sas0: removing handle(0x000a), sas_addr(0x443322110600) testsan:/btrfs # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid1 size 298.09GB used 19.03GB path /dev/sdf *** Some devices missing Btrfs v0.19+ = after connecting the same disk again = it seems that the disk is now sdh instead of sdg, could be that I've connected the disk on another port of the HBA # btrfs filesystem show Label: none uuid: fe542409-7346-4ea1-af04-fd1765b6a1a2 Total devices 2 FS bytes used 123.02MB devid2 size 298.09GB used 19.01GB path /dev/sdh devid1 size 298.09GB used 19.03GB path /dev/sdf Btrfs v0.19+ After running a scrub command I get now the following errors in dmesg: [ 1253.17] sd 8:0:1:0: [sdg] Synchronizing SCSI cache [ 1253.42] sd 8:0:1:0: [sdg] [ 1253.44] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK [ 1253.444588] mpt2sas0: removing handle(0x000a), sas_addr(0x443322110600) [ 1385.440298] scsi 8:0:2:0: Direct-Access ATA WDC WD3200AAJS-0 3E01 PQ: 0 ANSI: 6 [ 1385.440307] scsi 8:0:2:0: SATA: handle(0x000a), sas_addr(0x443322110600), phy(6), device_name(0x) [ 1385.440310] scsi 8:0:2:0: SATA: enclosure_logical_id(0x500605b0054dc1f0), slot(5) [ 1385.440415] scsi 8:0:2:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) [ 1385.440421] scsi 8:0:2:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 1385.440627] sd 8:0:2:0: Attached scsi generic sg0 type 0 [ 1385.441276] sd 8:0:2:0: [sdh] 625142448 512-byte logical blocks: (320 GB/298 GiB) [ 1385.444743] sd 8:0:2:0: [sdh] Write Protect is off [ 1385.444747] sd 8:0:2:0: [sdh] Mode Sense: 7f 00 10 08 [ 1385.445860] sd 8:0:2:0: [sdh] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 1385.464525] sdh: unknown partition table [ 1385.472633] sd 8:0:2:0: [sdh] Attached SCSI disk [ 1593.048743] [ cut here ] [ 1593.050188] kernel BUG at /usr/src/packages/BUILD/kernel-default-3.6.5/linux-3.6/fs/btrfs/scrub.c:638! [ 1593.051654] invalid opcode: [#1] SMP [ 1593.052712] Modules linked in: btrfs zlib_deflate libcrc32c af_packet cpufreq_conservative cpufreq_userspace cpufreq_powersave dm_mod snd_hda_codec_hdmi iTCO_wdt snd_hda_codec_realtek gpio_ich iTCO_vendor_support sg i2c_i801 acpi_cpufreq mperf coretemp serio_raw pcspkr sr_mod cdrom snd_hda_intel mei lpc_ich mfd_core e1000e kvm_intel kvm microcode snd_hda_codec snd_hwdep snd_pcm snd_timer snd usb_storage tpm_tis tpm hid_generic wmi usbhid soundcore snd_page_alloc tpm_bios edd autofs4 uhci_hcd ehci_hcd usbcore usb_common i915 drm_kms_helper drm i2c_algo_bit video button processor thermal_sys scsi_dh_hp_sw scsi_dh_rdac scsi_dh_alua scsi_dh_emc scsi_dh mpt2sas scsi_transport_sas raid_class ata_generic [ 1593.052712] CPU 2 [ 1593.052712] Pid: 2823, comm: btrfs-scrub-1 Not tainted 3.6.5-0-default #1 Acer Veriton M67WS/EQ45M [ 1593.052712] RIP: 0010:[a0526032] [a0526032] scrub_handle_errored_block+0x972/0x980 [btrfs] [ 1593.052712] RSP: 0018:88022d6c1ca0 EFLAGS: 00010246 [ 1593.052712] RAX: 0007 RBX: 88022d012800 RCX: 0001 [ 1593.052712] RDX:
[PATCH] Btrfs: fix BUG() in scrub when first superblock reading gives EIO
This fixes a very special case that can be reproduced by just
disconnecting a disk at runtime, and without unmounting the
filesystem first, start scrub on the filesystem with the
disconnected disk. All read and write EIOs are handled
correctly, only the first superblock is an exception and gives
a BUG() in a subfunction. The BUG() is correct, it would crash
later otherwise. The subfunction must not be called for
superblocks and this is what the fix changes.
Reported-by: Joeri Vanthienen m...@joerivanthienen.be
Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de
---
This patch is for btrfs-next, not for 3.7rc.
fs/btrfs/scrub.c | 11 +++
1 file changed, 11 insertions(+)
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index 30ba997..37c6443 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -785,6 +785,17 @@ static int scrub_handle_errored_block(struct scrub_block
*sblock_to_check)
BUG_ON(sblock_to_check-page_count 1);
fs_info = sctx-dev_root-fs_info;
+ if (sblock_to_check-pagev[0]-flags BTRFS_EXTENT_FLAG_SUPER) {
+ /*
+* if we find an error in a super block, we just report it.
+* They will get written with the next transaction commit
+* anyway
+*/
+ spin_lock(sctx-stat_lock);
+ ++sctx-stat.super_errors;
+ spin_unlock(sctx-stat_lock);
+ return 0;
+ }
length = sblock_to_check-page_count * PAGE_SIZE;
logical = sblock_to_check-pagev[0]-logical;
generation = sblock_to_check-pagev[0]-generation;
--
1.8.0
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bulk discard doesn't work after add/delete of devices
Hi, the issue I raised in this thread is still not fixed. So, to get things going, I prepared a patch myself which I will send in a separate email. The subject will be really fix trim 0 bytes after a device delete. I would be grateful if you could consider the patch -- with it applied I finally could trim my disks successfully. Greetings, Lutz -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: really fix trim 0 bytes after a device delete
Commit 2cac13e41bf5b99ffc426bd28dfd2248df1dfa67, fix trim 0 bytes after
a device delete, said:
A user reported a bug of btrfs's trim, that is we will trim 0 bytes
after a device delete.
The commit didn't attack the root of the problem so did not fix the bug
except for a special case.
For block discard, btrfs_trim_fs directly compares the range passed in
against the filesystem's objectids. The former is bounded by the sum of
the sizes of the devices of the filesystem, the latter is a completely
unrelated set of intervals of 64-bit integers. The bug reported occurred
as the smallest objectid was larger than the sum of the device sizes.
The previous commit only fixes the case where the smallest objectid is
nonzero and the largest objectid less than the sum of the device sizes,
but it would still trim too little if the largest objectid is larger
than that and nothing in the reported situation.
The current mapping between the given range and the objectids is thus
clearly broken, so, to fix the bug and as a first step towards a
complete solution, simply ignore the range parameter's start and length
fields and always trim the whole filesystem. (While this makes it
impossible to only partly trim a filesystem, due to the broken mapping
this often didn't work anyway.)
Reported-by: Lutz Euler lutz.eu...@freenet.de
Signed-off-by: Lutz Euler lutz.eu...@freenet.de
---
fs/btrfs/extent-tree.c | 55 +--
1 files changed, 25 insertions(+), 30 deletions(-)
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 3d3e2c1..9c2bb59 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -8092,44 +8092,39 @@ int btrfs_trim_fs(struct btrfs_root *root, struct
fstrim_range *range)
u64 start;
u64 end;
u64 trimmed = 0;
- u64 total_bytes = btrfs_super_total_bytes(fs_info-super_copy);
int ret = 0;
/*
-* try to trim all FS space, our block group may start from non-zero.
+* The range passed in is a subinterval of the interval from 0
+* to the sum of the sizes of the devices of the filesystem.
+* The objectid's used in the filesystem can span any set of
+* subintervals of the interval from 0 to (u64)-1. As there is
+* neither a simple nor an agreed upon mapping between these
+* two ranges we ignore the range parameter's start and len
+* fields and always trim the whole filesystem (that is, only
+* the free space in allocated chunks).
*/
- if (range-len == total_bytes)
- cache = btrfs_lookup_first_block_group(fs_info, range-start);
- else
- cache = btrfs_lookup_block_group(fs_info, range-start);
+ cache = btrfs_lookup_first_block_group(fs_info, 0);
while (cache) {
- if (cache-key.objectid = (range-start + range-len)) {
- btrfs_put_block_group(cache);
- break;
- }
-
- start = max(range-start, cache-key.objectid);
- end = min(range-start + range-len,
- cache-key.objectid + cache-key.offset);
+ start = cache-key.objectid;
+ end = cache-key.objectid + cache-key.offset;
- if (end - start = range-minlen) {
- if (!block_group_cache_done(cache)) {
- ret = cache_block_group(cache, NULL, root, 0);
- if (!ret)
- wait_block_group_cache_done(cache);
- }
- ret = btrfs_trim_block_group(cache,
-group_trimmed,
-start,
-end,
-range-minlen);
+ if (!block_group_cache_done(cache)) {
+ ret = cache_block_group(cache, NULL, root, 0);
+ if (!ret)
+ wait_block_group_cache_done(cache);
+ }
+ ret = btrfs_trim_block_group(cache,
+group_trimmed,
+start,
+end,
+range-minlen);
- trimmed += group_trimmed;
- if (ret) {
- btrfs_put_block_group(cache);
- break;
- }
+ trimmed += group_trimmed;
+ if (ret) {
+ btrfs_put_block_group(cache);
+ break;
}
cache = next_block_group(fs_info-tree_root, cache);
--
1.7.4.1
--
To unsubscribe from this list: send the line
Re: btrfs send/receive symlink bug
On Wed, Nov 14, 2012 at 5:34 PM, David Arendt ad...@prnet.org wrote: Hi, I am using kernel 3.7.0-rc5 and latest btrfs-progs git. I am trying btrfs send/receive. When I have a filesystem containing a symlink pointing to a nonexistent destination or a destination created after the symlink was created, btrfs receive aborts with: ERROR: chown link1.txt failed. No such file or directory. Steps to reproduce (/dev/loop0 and /dev/loop1 are test images): mkfs.btrfs /dev/loop0 mount /dev/loop0 /u00 btrfs subvolume create /u00/test cd /u00/test ln -s test1.txt link1.txt btrfs subvolume snapshot -r /u00/test /u00/test.snapshot mkfs.btrfs /dev/loop1 mount /dev/loop1 /u01 btrfs send /u00/test.snapshot | btrfs receive /u01 ERROR: chown link1.txt failed. No such file or directory A patch [1] for btrfs-progs that solves this issue was posted a month ago but has landed yet on the btrfs-progs git repository. [1] http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg19539.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/3] btrfs-progs: check malloc() result
Signed-off-by: Kenji Okimoto okim...@clear-code.com
---
disk-io.c |3 +++
1 file changed, 3 insertions(+)
diff --git a/disk-io.c b/disk-io.c
index 32f45dc..c4d4631 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -458,6 +458,9 @@ static int find_and_setup_log_root(struct btrfs_root
*tree_root,
u64 blocknr = btrfs_super_log_root(disk_super);
struct btrfs_root *log_root = malloc(sizeof(struct btrfs_root));
+ if (!log_root)
+ return -ENOMEM;
+
if (blocknr == 0) {
free(log_root);
return 0;
--
1.7.10.4
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] btrfs-progs: plug memory leaks in btrfs_scan_one_dir() reported by cppcheck.
[utils.c:983]: (error) Memory leak: fullpath
Signed-off-by: Kenji Okimoto okim...@clear-code.com
---
utils.c |3 +++
1 file changed, 3 insertions(+)
diff --git a/utils.c b/utils.c
index 205e667..3efb3af 100644
--- a/utils.c
+++ b/utils.c
@@ -980,6 +980,7 @@ again:
dirp = opendir(dirname);
if (!dirp) {
fprintf(stderr, Unable to open %s for scanning\n, dirname);
+ free(fullpath);
return -ENOENT;
}
while(1) {
@@ -1030,6 +1031,7 @@ again:
free(pending);
pending = list_entry(pending_list.next, struct pending_dir,
list);
+ free(fullpath);
list_del(pending-list);
closedir(dirp);
dirp = NULL;
@@ -1038,6 +1040,7 @@ again:
ret = 0;
fail:
free(pending);
+ free(fullpath);
if (dirp)
closedir(dirp);
return ret;
--
1.7.10.4
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/3] btrfs-progs: plug a memory leak reported by cppcheck
[src/btrfs/disk-io.c:462]: (error) Memory leak: log_root
Signed-off-by: Kenji Okimoto okim...@clear-code.com
---
disk-io.c | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/disk-io.c b/disk-io.c
index 0395205..32f45dc 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -458,8 +458,10 @@ static int find_and_setup_log_root(struct btrfs_root
*tree_root,
u64 blocknr = btrfs_super_log_root(disk_super);
struct btrfs_root *log_root = malloc(sizeof(struct btrfs_root));
- if (blocknr == 0)
+ if (blocknr == 0) {
+ free(log_root);
return 0;
+ }
blocksize = btrfs_level_size(tree_root,
btrfs_super_log_root_level(disk_super));
@@ -474,8 +476,12 @@ static int find_and_setup_log_root(struct btrfs_root
*tree_root,
fs_info-log_root_tree = log_root;
- if (!extent_buffer_uptodate(log_root-node))
+ if (!extent_buffer_uptodate(log_root-node)) {
+ free(log_root);
return -EIO;
+ }
+
+ free(log_root);
return 0;
}
--
1.7.10.4
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/3] btrfs-progs: fix problems reported by cppcheck
I fixed some problems reported by cppcheck: [disk-io.c:462]: (error) Memory leak: log_root [utils.c:983]: (error) Memory leak: fullpath Patch set based on: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git#master commit 91d9eec1ff044394f2b98ee7fcb76713dd33b994 Author: Chris Mason chris.ma...@fusionio.com Date: Thu Oct 4 20:35:31 2012 -0400 Kenji Okimoto (3): btrfs-progs: plug a memory leak reported by cppcheck btrfs-progs: check malloc() result btrfs-progs: plug memory leaks in btrfs_scan_one_dir() reported by cppcheck. disk-io.c | 13 +++-- utils.c |3 +++ 2 files changed, 14 insertions(+), 2 deletions(-) -- 1.7.10.4 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: problem with ceph and btrfs patch: set journal_info in async trans commit worker
Hi, Stefan On wed, 14 Nov 2012 14:42:07 +0100, Stefan Priebe - Profihost AG wrote: Hello list, i wanted to try out ceph with latest vanilla kernel 3.7-rc5. I was seeing a massive performance degration. I see around 22x btrfs-endio-write processes every 10-20 seconds and they run a long time while consuming a massive amount of CPU. So my performance of 23.000 iops drops to an up and down of 23.000 iops to 0 - avg is now 2500 iops instead of 23.000. Git bisect shows me commit: e209db7ace281ca347b1ac699bf1fb222eac03fe Btrfs: set journal_info in async trans commit worker as the problematic patch. When i revert this one everything is fine again. Is this known? Could you try the following patch? http://marc.info/?l=linux-btrfsm=135175512030453w=2 I think the patch Btrfs: set journal_info in async trans commit worker is not the real reason that caused the regression. I guess it is caused by the bug of the reservation. When we join the same transaction handle more than 2 times, the pointer of the reservation in the transaction handle would be lost, and the statistical data in the reservation would be corrupted. And then we would trigger the space flush, which may block your tasks. Thanks Miao Greets, Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs send/receive symlink bug
Hi, thanks, this patch fixes the problem Bye, David Arendt On 11/15/12 05:21, Marios Titas wrote: On Wed, Nov 14, 2012 at 5:34 PM, David Arendt ad...@prnet.org wrote: Hi, I am using kernel 3.7.0-rc5 and latest btrfs-progs git. I am trying btrfs send/receive. When I have a filesystem containing a symlink pointing to a nonexistent destination or a destination created after the symlink was created, btrfs receive aborts with: ERROR: chown link1.txt failed. No such file or directory. Steps to reproduce (/dev/loop0 and /dev/loop1 are test images): mkfs.btrfs /dev/loop0 mount /dev/loop0 /u00 btrfs subvolume create /u00/test cd /u00/test ln -s test1.txt link1.txt btrfs subvolume snapshot -r /u00/test /u00/test.snapshot mkfs.btrfs /dev/loop1 mount /dev/loop1 /u01 btrfs send /u00/test.snapshot | btrfs receive /u01 ERROR: chown link1.txt failed. No such file or directory A patch [1] for btrfs-progs that solves this issue was posted a month ago but has landed yet on the btrfs-progs git repository. [1] http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg19539.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
