Re: experiences running btrfs on external USB disks?

2018-12-03 Thread Tomasz Chmielewski
tfix/qmgr[2745]: 13BBE460F86: from=, size=404, nrcpt=1 (queue active) Dec 4 05:23:14 step postfix/pickup[8993]: 40A964603EC: uid=0 from= [...some emails follow, usual CRON messages etc., but noting at all generated by the kernel, no hardware issue reported...] Tomasz Chmielewski

experiences running btrfs on external USB disks?

2018-12-03 Thread Tomasz Chmielewski
0 [/dev/sda1].corruption_errs 0 [/dev/sda1].generation_errs 0 Tomasz Chmielewski

Re: btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Tomasz Chmielewski
ms the issue was unneeded discard option, so not pasting unnecessary SMART data, thanks for finding this out. Tomasz Chmielewski

btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Tomasz Chmielewski
8-a644-430b-9dd9-2ad3d381448a Total devices 3 FS bytes used 987.12GiB devid1 size 1.73TiB used 804.03GiB path /dev/sda2 devid2 size 1.73TiB used 804.06GiB path /dev/sdb2 devid3 size 1.73TiB used 804.03GiB path /dev/sdc2 Tomasz Chmielewski https://lxadm.com

Re: unable to mount btrfs after upgrading from 4.16.1 to 4.19.1

2018-11-09 Thread Tomasz Chmielewski
On 2018-11-10 04:20, Tomasz Chmielewski wrote: On 2018-11-10 04:15, Tomasz Chmielewski wrote: On 2018-11-10 03:20, Roman Mamedov wrote: On Sat, 10 Nov 2018 03:08:01 +0900 Tomasz Chmielewski wrote: After upgrading from kernel 4.16.1 to 4.19.1 and a clean restart, the fs no longer mounts

Re: unable to mount btrfs after upgrading from 4.16.1 to 4.19.1

2018-11-09 Thread Tomasz Chmielewski
On 2018-11-10 04:15, Tomasz Chmielewski wrote: On 2018-11-10 03:20, Roman Mamedov wrote: On Sat, 10 Nov 2018 03:08:01 +0900 Tomasz Chmielewski wrote: After upgrading from kernel 4.16.1 to 4.19.1 and a clean restart, the fs no longer mounts: Did you try rebooting back to 4.16.1 to see

Re: unable to mount btrfs after upgrading from 4.16.1 to 4.19.1

2018-11-09 Thread Tomasz Chmielewski
On 2018-11-10 03:20, Roman Mamedov wrote: On Sat, 10 Nov 2018 03:08:01 +0900 Tomasz Chmielewski wrote: After upgrading from kernel 4.16.1 to 4.19.1 and a clean restart, the fs no longer mounts: Did you try rebooting back to 4.16.1 to see if it still mounts there? Yes, just did

unable to mount btrfs after upgrading from 4.16.1 to 4.19.1

2018-11-09 Thread Tomasz Chmielewski
open_ctree failed Any advice how to recover? Tomasz Chmielewski https://lxadm.com

very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Tomasz Chmielewski
? Tomasz Chmielewski https://lxadm.com

Re: fatal database corruption with btrfs "out of space" with ~50 GB left

2018-02-19 Thread Tomasz Chmielewski
On 2018-02-19 13:29, Anand Jain wrote: On 02/14/2018 10:19 PM, Tomasz Chmielewski wrote: Just FYI, how dangerous running btrfs can be - we had a fatal, unrecoverable MySQL corruption when btrfs decided to do one of these "I have ~50 GB left, so let's do out of space (and corrupt some

Re: fatal database corruption with btrfs "out of space" with ~50 GB left

2018-02-14 Thread Tomasz Chmielewski
On 2018-02-15 16:02, Tomasz Chmielewski wrote: On 2018-02-15 13:32, Qu Wenruo wrote: Is there any kernel message like kernel warning or backtrace? I see there was this one: Feb 13 13:53:32 lxd01 kernel: [9351710.878404] [ cut here ] Feb 13 13:53:32 lxd01 kernel

Re: fatal database corruption with btrfs "out of space" with ~50 GB left

2018-02-14 Thread Tomasz Chmielewski
or so, and was not even 50% done. And the IO load the balance was adding was slowing the things down a lot. Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordo

Re: fatal database corruption with btrfs "out of space" with ~50 GB left

2018-02-14 Thread Tomasz Chmielewski
On 2018-02-15 10:47, Qu Wenruo wrote: On 2018年02月14日 22:19, Tomasz Chmielewski wrote: Just FYI, how dangerous running btrfs can be - we had a fatal, unrecoverable MySQL corruption when btrfs decided to do one of these "I have ~50 GB left, so let's do out of space (and corrupt some

fatal database corruption with btrfs "out of space" with ~50 GB left

2018-02-14 Thread Tomasz Chmielewski
Just FYI, how dangerous running btrfs can be - we had a fatal, unrecoverable MySQL corruption when btrfs decided to do one of these "I have ~50 GB left, so let's do out of space (and corrupt some files at the same time, ha ha!)". Running btrfs RAID-1 with kernel 4.14. Tomasz C

Re: again "out of space" and remount read only, with 4.14

2017-11-26 Thread Tomasz Chmielewski
On 2017-11-27 00:37, Martin Raiber wrote: On 26.11.2017 08:46 Tomasz Chmielewski wrote: Got this one on a 4.14-rc7 filesystem with some 400 GB left: Hi, I guess it is too late now, but I guess the "btrfs fi usage" output of the file system (especially after it went ro) would

again "out of space" and remount read only, with 4.14

2017-11-25 Thread Tomasz Chmielewski
rror 4 in libmailutils.so.4.0.0[7f0c547f2000+b] [2217605.435864] BTRFS error (device sda4): pending csums is 22380544 Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org Mor

Re: 4.14 balance: kernel BUG at /home/kernel/COD/linux/fs/btrfs/ctree.c:1856!

2017-11-17 Thread Tomasz Chmielewski
On 2017-11-18 10:08, Hans van Kranenburg wrote: On 11/18/2017 01:49 AM, Tomasz Chmielewski wrote: I'm getting the following BUG when running balance on one of my systems: [ 3458.698704] BTRFS info (device sdb3): relocating block group 306045779968 flags data|raid1 [ 3466.892933] BTRFS info

4.14 balance: kernel BUG at /home/kernel/COD/linux/fs/btrfs/ctree.c:1856!

2017-11-17 Thread Tomasz Chmielewski
d5fb88 [ 3498.167892] ---[ end trace 6a751a3020dd3086 ]--- [ 3499.572729] BTRFS info (device sdb3): relocating block group 304972038144 flags data|raid1 [ 3504.068432] BTRFS info (device sdb3): found 2037 extents [ 3538.281808] BTRFS info (device sdb3): found 2037 extents Tomasz Chmielewski

Re: how to run balance successfully (No space left on device)?

2017-11-09 Thread Tomasz Chmielewski
uldn't "Balance" be marked as "mostly OK" or "Unstable" here? Giving it "OK" status is misleading. https://btrfs.wiki.kernel.org/index.php/Status Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrf

Re: how to run balance successfully (No space left on device)?

2017-11-06 Thread Tomasz Chmielewski
On 2017-10-31 23:18, Tomasz Chmielewski wrote: On 2017-09-18 17:20, Tomasz Chmielewski wrote: # df -h /var/lib/lxd FWIW, standard (aka util-linux) df is effectively useless in a situation such as this, as it really doesn't give you the information you need (it can say you have lots of space

Re: how to run balance successfully (No space left on device)?

2017-10-31 Thread Tomasz Chmielewski
On 2017-10-31 23:18, Tomasz Chmielewski wrote: On a different server, however, it failed badly: # time btrfs balance start /srv WARNING: Full balance without filters requested. This operation is very intense and takes potentially very long. It is recommended to use

Re: how to run balance successfully (No space left on device)?

2017-10-31 Thread Tomasz Chmielewski
On 2017-09-18 17:20, Tomasz Chmielewski wrote: # df -h /var/lib/lxd FWIW, standard (aka util-linux) df is effectively useless in a situation such as this, as it really doesn't give you the information you need (it can say you have lots of space available, but if btrfs has all of it allocated

Re: yet another "out of space" on a filesystem with >100 GB free space, and strange files which exist but don't exist

2017-10-04 Thread Tomasz Chmielewski
On 2017-10-04 20:20, Austin S. Hemmelgarn wrote: On 2017-10-04 07:13, Tomasz Chmielewski wrote: Kernel: 4.13.4, btrfs RAID-1. Disk usage more or less like below (yes, I know about btrfs fi df / show / usage): Filesystem  Size  Used Avail Use% Mounted on /dev/sda3   424G  262G  161G

yet another "out of space" on a filesystem with >100 GB free space, and strange files which exist but don't exist

2017-10-04 Thread Tomasz Chmielewski
ry What is this file, and why does it exist if it doesn't? How do I remove it? Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.

Re: how to run balance successfully (No space left on device)?

2017-09-18 Thread Tomasz Chmielewski
But I've removed some 65 GB data, so now it's 171 GB free, or, 60% used filesystem. The balance still fails. Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More maj

Re: how to run balance successfully (No space left on device)?

2017-09-18 Thread Tomasz Chmielewski
On 2017-09-18 17:29, Andrei Borzenkov wrote: On Mon, Sep 18, 2017 at 11:20 AM, Tomasz Chmielewski <man...@wpkg.org> wrote: # df -h /var/lib/lxd FWIW, standard (aka util-linux) df is effectively useless in a situation such as this, as it really doesn't give you the information yo

Re: how to run balance successfully (No space left on device)?

2017-09-18 Thread Tomasz Chmielewski
But chances are pretty good that one you get that patch integrated, whether by integrating it yourself to what you have currently, or by trying 4.14-rc1 or waiting until it hits release or stable, that bug will have been squashed! =:^) OK, will wait for 4.14. Tomasz Chmielewski https://lx

how to run balance successfully (No space left on device)?

2017-09-17 Thread Tomasz Chmielewski
[46914.188044] BTRFS info (device sda3): found 7299 extents [46914.303476] BTRFS info (device sda3): relocating block group 2947936616448 flags metadata|raid1 [46939.570810] BTRFS info (device sda3): found 42022 extents [46945.053488] BTRFS info (device sda3): 2 enospc errors during balance Tomasz

Re: 4.13: No space left with plenty of free space (/home/kernel/COD/linux/fs/btrfs/extent-tree.c:6989 __btrfs_free_extent.isra.62+0xc2c/0xdb0)

2017-09-08 Thread Tomasz Chmielewski
uot;, when there is a lot of space left? Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: 4.13: No space left with plenty of free space (/home/kernel/COD/linux/fs/btrfs/extent-tree.c:6989 __btrfs_free_extent.isra.62+0xc2c/0xdb0)

2017-09-07 Thread Tomasz Chmielewski
On 2017-09-08 13:33, Tomasz Chmielewski wrote: Just got this one in dmesg with btrfs RAID-1 on top of Linux software RAID-5. Should say: with btrfs _single_ on top of Linux software RAID-5. Why does it say "No space left" if we have 9 TB free there? [233787.920933] BTRFS: T

4.13: No space left with plenty of free space (/home/kernel/COD/linux/fs/btrfs/extent-tree.c:6989 __btrfs_free_extent.isra.62+0xc2c/0xdb0)

2017-09-07 Thread Tomasz Chmielewski
Global reserve: 512.00MiB (used: 0.00B) Data,single: Size:12.84TiB, Used:7.13TiB /dev/md2 12.84TiB Metadata,DUP: Size:79.00GiB, Used:77.87GiB /dev/md2 158.00GiB System,DUP: Size:8.00MiB, Used:1.48MiB /dev/md2 16.00MiB Unallocated: /dev/md2 3.31TiB Tomasz C

btrfs hang with 4.11.1 kernel

2017-05-17 Thread Tomasz Chmielewski
6047] RDX: 7f86700ada30 RSI: 000a RDI: 043c May 17 07:47:53 lxd02 kernel: [43865.596049] RBP: 7f86700ac620 R08: R09: 3df7938e May 17 07:47:53 lxd02 kernel: [43865.596050] R10: 7f4cf401ca80 R11: 0293 R12: 01207fe0 May 17 07:

Re: how to understand "btrfs fi show" output? "No space left" issues

2016-11-13 Thread Tomasz Chmielewski
free_of_min_used 7979008 block_group_of_min_used 915121242112 balance block group (915121242112) can reduce the number of data block group Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message

Re: how to understand "btrfs fi show" output? "No space left" issues

2016-09-26 Thread Tomasz Chmielewski
rading to 4.8rc8, "used" value dropped, so hopefully it's fixed now. Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

WARNING: CPU: 7 PID: 23122 at /home/kernel/COD/linux/lib/list_debug.c:59 btrfs_clear_bit_hook+0x2b9/0x350 [btrfs]

2016-08-24 Thread Tomasz Chmielewski
6e9c12 ]--- Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Input/output error, nothing appended in dmesg

2016-08-05 Thread Tomasz Chmielewski
On 2016-08-06 00:45, Tomasz Chmielewski wrote: And, miracle cure O_o # file ./2016-08-02/serverX/syslog.log ERROR: cannot read `./2016-08-02/serverX/syslog.log' (Input/output error) # echo 3 > /proc/sys/vm/drop_caches # file 2016-08-02/serverX/syslog.log 2016-08-02/serverX/syslog.

Re: Input/output error, nothing appended in dmesg

2016-08-05 Thread Tomasz Chmielewski
ted? It is mounted, yes. Does btrfs-debug-tree need an unmounted FS? I'm not able to unmount it unfortunately (in sense, the system has to work). Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a mess

Re: Input/output error, nothing appended in dmesg

2016-08-05 Thread Tomasz Chmielewski
On 2016-08-06 00:38, Tomasz Chmielewski wrote: Too big for the known problem though. Still, can you btrfs-debug-tree and just make sure it doesn't have inline items? Hmmm # btrfs-debug-tree /dev/xvdb > /root/debug.tree parent transid verify failed on 355229302784 wanted 49943295 fo

Re: Input/output error, nothing appended in dmesg

2016-08-05 Thread Tomasz Chmielewski
ree(btrfs_print_tree+0x26a)[0x41acf6] btrfs-debug-tree(main+0x9a5)[0x432589] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f2369de0f45] btrfs-debug-tree[0x4070e9] Tomasz Chmielewski https://lxadm.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs"

Re: Input/output error, nothing appended in dmesg

2016-08-05 Thread Tomasz Chmielewski
On 2016-08-05 23:26, Chris Mason wrote: On 08/05/2016 07:42 AM, Tomasz Chmielewski wrote: I'm getting occasional (every few weeks) input/output errors on a btrfs filesystem with compress-force=zlib, running on Amazon EC2, with 4.5.2 kernel: # cat 2016-08-02/serverX/syslog.log cat: 2016-08-02

Input/output error, nothing appended in dmesg

2016-08-05 Thread Tomasz Chmielewski
/dev/xvdb # btrfs fi df /var/log/remote/ Data, single: total=149.00GiB, used=144.50GiB System, single: total=4.00MiB, used=48.00KiB Metadata, single: total=4.01GiB, used=2.05GiB GlobalReserve, single: total=512.00MiB, used=0.00B Tomasz Chmielewski https://lxadm.com -- To unsubscribe from

Re: btrfs won't mount to /home

2016-07-11 Thread Tomasz Chmielewski
On 2016-07-11 22:56, Roman Mamedov wrote: On Mon, 11 Jul 2016 22:45:13 +0900 Tomasz Chmielewski <t...@virtall.com> wrote: So, weird, isn't it? What's wrong there? Your systemd unmounts it immediately from /home, search the archives there's been a funny story like that recently.

btrfs won't mount to /home

2016-07-11 Thread Tomasz Chmielewski
esg -c [382190.199363] BTRFS info (device sda4): disk space caching is enabled [382190.199370] BTRFS: has skinny extents What's wrong there? Tomasz Chmielewski https://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger

Re: filesystem read only after power outage

2016-07-09 Thread Tomasz Chmielewski
On 2016-07-05 15:55, Tomasz Chmielewski wrote: On 2016-07-05 14:56, Tomasz Chmielewski wrote: Getting this lengthy output logged, and the fs mounter read-only after a power outage. Tried also 4.6.3, but it ends just alike. Jul 5 02:04:20 bkp011 kernel: [ 799.298303] [ cut here

Re: filesystem read only after power outage

2016-07-05 Thread Tomasz Chmielewski
On 2016-07-05 14:56, Tomasz Chmielewski wrote: Getting this lengthy output logged, and the fs mounter read-only after a power outage. Tried also 4.6.3, but it ends just alike. Jul 5 02:04:20 bkp011 kernel: [ 799.298303] [ cut here ] Jul 5 02:04:20 bkp011 kernel

filesystem read only after power outage

2016-07-05 Thread Tomasz Chmielewski
Getting this lengthy output logged, and the fs mounter read-only after a power outage. Tried also 4.6.3, but it ends just alike. Jul 5 02:04:20 bkp011 kernel: [ 799.298303] [ cut here ] Jul 5 02:04:20 bkp011 kernel: [ 799.298335] WARNING: CPU: 0 PID: 1896 at

can't use btrfs on USB-stick (write errors)

2016-06-21 Thread Tomasz Chmielewski
fo (device sdb1): delayed_refs has NO entry Jun 14 07:50:26 ativ kernel: [57362.034220] BTRFS error (device sdb1): commit super ret -5 Jun 14 07:50:26 ativ kernel: [57362.034339] BTRFS error (device sdb1): cleaner transaction attach returned -30 Tomasz Chmielewski http://wpkg.org -- To unsubs

btrfs RAID-1 vs md RAID-1?

2016-05-15 Thread Tomasz Chmielewski
read requests across all devices to maximise performance. Raid5 (...) This also allows more parallelism when reading, as read requests are distributed over all the devices in the array instead of all but one. Are there any plans to improve this is btrfs? Tomasz

Re: 4.4.0 - no space left with >1.7 TB free space left

2016-05-12 Thread Tomasz Chmielewski
On 2016-05-12 15:03, Tomasz Chmielewski wrote: FYI, I'm still getting this with 4.5.3, which probably means the fix was not yet included ("No space left" at snapshot time): /var/log/postgresql/postgresql-9.3-main.log:2016-05-11 06:06:10 UTC LOG: could not close temporary stati

Re: 4.4.0 - no space left with >1.7 TB free space left

2016-05-12 Thread Tomasz Chmielewski
porary statistics file "pg_stat_tmp/global.tmp": No space left on device I've tried mounting with space_cache=v2, but it didn't help. Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a messa

Re: 4.4.0 - no space left with >1.7 TB free space left

2016-04-08 Thread Tomasz Chmielewski
.spinics.net/lists/linux-btrfs/msg52478.html But maybe that meant 4.5 or 4.6 only? While the bug is affecting people on 4.4.x today. Does it mean 4.5 also doesn't have it yet? Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs&qu

Re: 4.4.0 - no space left with >1.7 TB free space left

2016-04-08 Thread Tomasz Chmielewski
g? If yes, then you should try the patch https://patchwork.kernel.org/patch/7967161/ (Too bad this was not included into 4.4.1.) By the way - was it included in any later kernel? I'm running 4.4.5 on that server, but still hitting the same issue. Tomasz Chmielewski http://wpkg.org -- To unsubs

Re: btrfs and containers

2016-03-10 Thread Tomasz Chmielewski
is what you want! LXD is particularly easy to run unprivileged containers: https://linuxcontainers.org/ (starts containers as unprivileged by default, and has lots of many goodies in general). Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe

4.4.0 - no space left with >1.7 TB free space left

2016-02-08 Thread Tomasz Chmielewski
ze 2.63TiB used 1.26TiB path /dev/sdb4 btrfs-progs v4.0.1 Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: 4.4.0 - no space left with >1.7 TB free space left

2016-02-08 Thread Tomasz Chmielewski
On 2016-02-08 20:24, Roman Mamedov wrote: On Mon, 08 Feb 2016 18:22:34 +0900 Tomasz Chmielewski <t...@virtall.com> wrote: Linux 4.4.0 - btrfs is mainly used to host lots of test containers, often snapshots, and at times, there is heavy IO in many of them for extended periods of time.

compression disk space saving - what are your results?

2015-12-02 Thread Tomasz Chmielewski
20% reasonable saving for zlib? Typically text compresses much better with that algorithm, although I understand that we have several limitations when applying that on a filesystem level. Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe li

Re: compression disk space saving - what are your results?

2015-12-02 Thread Tomasz Chmielewski
% text files/logs. Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: compression disk space saving - what are your results?

2015-12-02 Thread Tomasz Chmielewski
=68bb462d42a963169bf7acbe106aae08c17129a5 http://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/commit/?id=4bcbb33255131adbe481c0467df26d654ce3bc78 Linux 4.3.0. Tomasz Chmielewski http://wpkg.org/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" i

Re: kernel crashes with btrfs and busy database IO - how to debug?

2015-06-15 Thread Tomasz Chmielewski
[45705.855796] [8180cbb2] system_call_fastpath+0x16/0x75 Also the hang seems to be highly related to the bug, would you please send a new mail reporting the hang? Thanks, Qu 在 2015年06月14日 15:58, Tomasz Chmielewski 写道: On 2015-06-14 09:30, Tomasz Chmielewski wrote: On 2015-06-13 08:23, Tomasz

Re: kernel crashes with btrfs and busy database IO - how to debug?

2015-06-14 Thread Tomasz Chmielewski
On 2015-06-14 09:30, Tomasz Chmielewski wrote: On 2015-06-13 08:23, Tomasz Chmielewski wrote: I did get it from /var/crash/ though - is it more useful? I don't have vmlinux for this kernel though, but have just built 4.1-rc7 with the same config, can try to get the crash there. I've uploaded

Re: kernel crashes with btrfs and busy database IO - how to debug?

2015-06-13 Thread Tomasz Chmielewski
On 2015-06-13 08:23, Tomasz Chmielewski wrote: I did get it from /var/crash/ though - is it more useful? I don't have vmlinux for this kernel though, but have just built 4.1-rc7 with the same config, can try to get the crash there. I've uploaded a crash dump and vmlinux here: http

Re: kernel crashes with btrfs and busy database IO - how to debug?

2015-06-12 Thread Tomasz Chmielewski
to debug for us. It's quite possible that some codes go mad and pass a NULL pointer, and then wait_event() is called on the NULL-some_member. Anyway, backtrace is needed to debug this. If syslog can't help, what about kdump + crash to get the backtrace? I'll try to get a kdump + crash. -- Tomasz

Re: kernel crashes with btrfs and busy database IO - how to debug?

2015-06-12 Thread Tomasz Chmielewski
89 ef 4c 8d 73 88 48 89 42 08 48 89 10 48 89 1b 48 89 5b 08 e8 4f 3b 6b c1 e8 3a [45705.855906] RIP [c0158b8e] btrfs_wait_pending_ordered+0x5e/0x110 [btrfs] [45705.855944] RSP 8800a0623d18 [45705.855959] CR2: 0008 -- Tomasz Chmielewski http://wpkg.org

kernel crashes with btrfs and busy database IO - how to debug?

2015-06-11 Thread Tomasz Chmielewski
(wait_queue_head_t *q, wait_queue_t *wait, int state) kernel/sched/wait.c:EXPORT_SYMBOL(prepare_to_wait_event); -- Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info

4.1-rc6 - kernel crash after doing chattr +C

2015-06-06 Thread Tomasz Chmielewski
48 89 10 48 89 1b 48 89 5b 08 e8 bf 3a 6b c1 e8 aa [57860.150959] RIP [c0158b8e] btrfs_wait_pending_ordered+0x5e/0x110 [btrfs] [57860.150998] RSP 8805d5c3bd18 [57860.151014] CR2: 0008 [57860.151186] ---[ end trace f41cd52aa31494ac ]--- -- Tomasz Chmielewski http

3.19.3: fs/btrfs/super.c:260 __btrfs_abort_transaction+0x4c/0x10e [btrfs]()

2015-04-03 Thread Tomasz Chmielewski
(device sdb5): forced readonly [67344.634010] BTRFS warning (device sdb5): Skipping commit of aborted transaction. [67344.634098] BTRFS: error (device sdb5) in cleanup_transaction:1670: errno=-22 unknown [67344.634186] BTRFS info (device sdb5): delayed_refs has NO entry -- Tomasz Chmielewski http

BTRFS: unable to add free space :-17

2015-03-23 Thread Tomasz Chmielewski
server1 kernel: [ 861.791031] BTRFS: unable to add free space :-17 Mar 23 13:13:29 server1 kernel: [ 911.761852] BTRFS: unable to add free space :-17 Mar 23 13:13:53 server1 kernel: [ 936.124674] BTRFS: unable to add free space :-17 -- Tomasz Chmielewski http://www.sslrack.com

Re: BTRFS: unable to add free space :-17

2015-03-23 Thread Tomasz Chmielewski
On 2015-03-23 22:48, Chris Mason wrote: On Mon, Mar 23, 2015 at 8:35 AM, Chris Mason c...@fb.com wrote: On Mon, Mar 23, 2015 at 8:19 AM, Tomasz Chmielewski t...@virtall.com wrote: Got this with 4.0.0-rc5 when doing a degraded mount: Do you get this every time, even after going back to rc4

Re: 3.19 - unable to replace a failed drive - 100% CPU usage in kworker and btrfs-transacti

2015-02-16 Thread Tomasz Chmielewski
On 2015-02-16 19:40, Liu Bo wrote: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 6269 root 20 0 000 R 92.5 0.0 2769:33 btrfs-transacti 22247 root 20 0 000 R 92.5 0.0 42:38.65 kworker/u16:16 Can you cat /proc/22247/stack and

3.19 - unable to replace a failed drive - 100% CPU usage in kworker and btrfs-transacti

2015-02-15 Thread Tomasz Chmielewski
I had a failed drive in RAID-1, so it was replaced with a good one, followed by: btrfs device add /dev/sdb4 /home btrfs device delete missing /home 4 days later, it got to a state when there is no IO anymore (according to iostat), btrfs device delete missing did not complete: # uptime

Re: WARNING: CPU: 1 PID: 2436 at fs/btrfs/qgroup.c:1414 btrfs_delayed_qgroup_accounting+0x9f1/0xa0b [btrfs]()

2015-02-12 Thread Tomasz Chmielewski
] [8104ff9a] ? kthread_freezable_should_stop+0x43/0x43 [197051.342944] ---[ end trace 5d57d07bb94831a2 ]--- Tomasz Chmielewski http://www.sslrack.com On 2015-01-04 07:58, Tomasz Chmielewski wrote: Got this with 3.18.1 and qgroups enabled. Not sure how to reproduce. [1262648.802286

Re: Kernel bug in 3.19-rc4

2015-01-15 Thread Tomasz Chmielewski
); BUG_ON(ret); /* -ENOSPC in reservation; Logic error? JDM */ } -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo

Re: kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123!

2015-01-07 Thread Tomasz Chmielewski
[15948.761942] ---[ end trace 0ccd21c265dce56b ]--- # ls bigfile2.img bigfile.img # touch 1 (...never returned...) Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo

WARNING: CPU: 1 PID: 2436 at fs/btrfs/qgroup.c:1414 btrfs_delayed_qgroup_accounting+0x9f1/0xa0b [btrfs]()

2015-01-03 Thread Tomasz Chmielewski
] ? kthread_freezable_should_stop+0x43/0x43 [1262648.804264] [813b59ec] ret_from_fork+0x7c/0xb0 [1262648.804311] [8104eb1a] ? kthread_freezable_should_stop+0x43/0x43 [1262648.804360] ---[ end trace b76fd72b4be63515 ]--- -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send

Re: Btrfs progs release 3.18

2014-12-30 Thread Tomasz Chmielewski
/sdb4, ID: 2 Device size: 2.63TiB Data,RAID1:252.00GiB Metadata,RAID1: 3.00GiB System,RAID1:8.00MiB Unallocated: 2.38TiB -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe

Re: 3.18.0: kernel BUG at fs/btrfs/relocation.c:242!

2014-12-19 Thread Tomasz Chmielewski
On 2014-12-19 22:47, Josef Bacik wrote: On 12/12/2014 09:37 AM, Tomasz Chmielewski wrote: FYI, still seeing this with 3.18 (scrub passes fine on this filesystem). # time btrfs balance start /mnt/lxc2 Segmentation fault Ok now I remember why I haven't fix this yet, the images you gave me

kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123!

2014-12-19 Thread Tomasz Chmielewski
bigfile2.img bigfile.img # touch 1 (...never returned...) -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: 3.18.0: kernel BUG at fs/btrfs/relocation.c:242!

2014-12-15 Thread Tomasz Chmielewski
On 2014-12-15 21:07, Josef Bacik wrote: On 12/12/2014 09:37 AM, Tomasz Chmielewski wrote: FYI, still seeing this with 3.18 (scrub passes fine on this filesystem). # time btrfs balance start /mnt/lxc2 Segmentation fault real322m32.153s user0m0.000s sys 16m0.930s Sorry Tomasz

Re: 3.18.0: kernel BUG at fs/btrfs/relocation.c:242!

2014-12-13 Thread Tomasz Chmielewski
referenced 3446026993664 Btrfs v3.17.3 real76m27.845s user19m1.470s sys 2m55.690s -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http

Re: 3.18.0: kernel BUG at fs/btrfs/relocation.c:242!

2014-12-13 Thread Tomasz Chmielewski
1-2 weeks (once the system grows in size / complexity). So hopefully, btrfsck had fixed that fs - once it is running stable for a week or two, I might be brave to re-enable btrfs quotas (was another system freezer, at least a few kernels back). -- Tomasz Chmielewski http://www.sslrack.com

Re: 3.18.0: kernel BUG at fs/btrfs/relocation.c:242!

2014-12-13 Thread Tomasz Chmielewski
in this tool), but definitely not BUG. -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Balance scrub defrag

2014-12-12 Thread Tomasz Chmielewski
manually, it was possible to finish btrfs device delete missing. Probably it makes sense to run scrub occasionally (just like mdraid is doing on most distributions). -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs

Re: 3.18.0: kernel BUG at fs/btrfs/relocation.c:242!

2014-12-12 Thread Tomasz Chmielewski
[btrfs] [20307.032754] RSP 8807e919bb18 [20307.033068] ---[ end trace 18be77360e49d59d ]--- On 2014-11-25 23:33, Tomasz Chmielewski wrote: I'm still seeing this when running balance with 3.18-rc6: [95334.066898] BTRFS info (device sdd1): relocating block group 6468350771200 flags 17

Re: 3.18.0: kernel BUG at fs/btrfs/relocation.c:242!

2014-12-12 Thread Tomasz Chmielewski
. -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: 3.18.0: kernel BUG at fs/btrfs/relocation.c:242!

2014-12-12 Thread Tomasz Chmielewski
On 2014-12-12 23:34, Robert White wrote: On 12/12/2014 01:46 PM, Tomasz Chmielewski wrote: On 2014-12-12 22:36, Robert White wrote: In another thread [that was discussing SMART] you talked about replacing a drive and then needing to do some patching-up of the result because of drive failures

Re: btrfs device delete missing - Input/output error

2014-12-07 Thread Tomasz Chmielewski
On 2014-12-07 06:26, Chris Murphy wrote: On Sat, Dec 6, 2014 at 2:17 AM, Tomasz Chmielewski t...@virtall.com wrote: After we run again btrfs device delete missing /home, the newly created directory eventually (/home/backup/ma-int/weekly.tmp) is being detected as csum failed ino

Re: btrfs device delete missing - Input/output error

2014-12-06 Thread Tomasz Chmielewski
, the newly created directory eventually (/home/backup/ma-int/weekly.tmp) is being detected as csum failed ino -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More

Re: btrfs device delete missing - Input/output error

2014-12-05 Thread Tomasz Chmielewski
, dev sdb, sector 2262535088 [ 8177.757542] ata4: EH complete -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo

Re: btrfs device delete missing - Input/output error

2014-12-05 Thread Tomasz Chmielewski
67112960s68161535s1048576sraid 3 68161536s2215645183s 2147483648s raid 4 2215645184s 5860533134s 3644887951s btrfs raid -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send

btrfs device delete missing - Input/output error

2014-12-04 Thread Tomasz Chmielewski
[26691.003163] end_request: I/O error, dev sdb, sector 2262535088 [26691.003215] btrfs_dev_stat_print_on_error: 36 callbacks suppressed [26691.003268] BTRFS: bdev /dev/sdb4 errs: wr 0, rd 60, flush 0, corrupt 0, gen 0 [26691.003383] ata4: EH complete -- Tomasz Chmielewski http

Re: 3.17.0-rc7: kernel BUG at fs/btrfs/relocation.c:931!

2014-11-25 Thread Tomasz Chmielewski
00 00 00 74 02 0f 0b 83 bb 24 01 00 00 00 74 02 0f 0b 48 8b 73 18 48 8b 7b 08 [95373.579226] RIP [a0323144] relocate_block_group+0x432/0x4de [btrfs] [95373.579352] RSP 8807da1bbb18 On 2014-10-04 00:06, Tomasz Chmielewski wrote: On 2014-10-03 20:17 (Fri), Josef Bacik wrote

Re: 5 _thousand_ snapshots? even 160? (was: device balance times)

2014-10-22 Thread Tomasz Chmielewski
monthly, for each data source * number of data sources. So while it probably will make a difference (5000 snapshots from one source, vs 5000 snapshots made from many sources) for balance times, I wouldn't call a large number of snapshots that unusual. -- Tomasz Chmielewski http://www.sslrack.com

device balance times

2014-10-21 Thread Tomasz Chmielewski
seconds = 133.86 MB/sec How long does the balance take for others with many snapshots? -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http

Re: 3.17.0-rc7: kernel BUG at fs/btrfs/relocation.c:931!

2014-10-04 Thread Tomasz Chmielewski
Hi, is btrfs-image with single -s flag OK? I.e. btrfs-image -s -c 9 -t 32 /dev/sdc1 /root/btrfs-2.img ? Tomasz Chmielewski On 2014-10-04 00:09 (Sat), Josef Bacik wrote: Can you make a btrfs-image of this fs and send it to me? Thanks, Josef Tomasz Chmielewski t...@virtall.com wrote

Re: 3.17.0-rc7: kernel BUG at fs/btrfs/relocation.c:931!

2014-10-03 Thread Tomasz Chmielewski
On 2014-10-03 20:17 (Fri), Josef Bacik wrote: On 10/02/2014 03:27 AM, Tomasz Chmielewski wrote: Got this when running balance with 3.17.0-rc7: Give these two patches a try https://patchwork.kernel.org/patch/4938281/ https://patchwork.kernel.org/patch/4939761/ With these two patches

Re: kernel BUG at fs/btrfs/extent-tree.c:7727! with 3.17-rc3

2014-10-03 Thread Tomasz Chmielewski
-09-08 20:04 (Mon), Tomasz Chmielewski wrote: On 2014-09-03 19:42 (Wed), Tomasz Chmielewski wrote: Got the following with 3.17-rc3 and running balance (had to power cycle after that): I'm seeing similar BUG with 3.17-rc4: [ 1049.755843] BTRFS info (device sdb5): found 35715 extents [ 1050.257075

3.17.0-rc7: kernel BUG at fs/btrfs/relocation.c:931!

2014-10-02 Thread Tomasz Chmielewski
[173475.416482] ---[ end trace 17e512e0d6dc91d7 ]--- -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: 3.17.0-rc6 system freeze

2014-09-29 Thread Tomasz Chmielewski
On 2014-09-29 16:10 (Mon), Liu Bo wrote: Hi Tomasz, On Mon, Sep 29, 2014 at 02:00:18PM +0200, Tomasz Chmielewski wrote: System froze under 3.17.0-rc6 with btrfs. It had to be hard rebooted. How does this happen? A stressful write with compression? Rsync (with --inplace - can be stressful

how long should btrfs device delete missing ... take?

2014-09-11 Thread Tomasz Chmielewski
, 2 users, load average: 3.22, 3.53, 3.55 I've tried running this on the latest 3.16.x kernel earlier, but since the progress was so slow, rebooted after about a week to see if the latest RC will be any faster. -- Tomasz Chmielewski http://www.sslrack.com -- To unsubscribe from this list

Re: how long should btrfs device delete missing ... take?

2014-09-11 Thread Tomasz Chmielewski
After a disk died and was replaced, btrfs device delete missing is taking more than 10 days on an otherwise idle server: Something isn't right though, because it's clearly neither reading nor writing at \ anywhere close to 1/2 the drive read throughput. I'm curious what 'iotop -d30 -o' \

  1   2   3   >