block group xxx has wrong amount of free space

[Михаил Гаврилов]

$ journalctl -b -1 --no-pager | grep BTRFS
May 08 01:28:23 localhost kernel: BTRFS: device fsid
a22dd8bc-24ef-4f97-bb62-01fcd7c3f11c devid 1 transid 25 /dev/sda4
May 08 01:28:23 localhost kernel: BTRFS: device fsid
eaa531cd-25f4-4e00-b31f-22665faa9768 devid 1 transid 333 /dev/sdb
May 08 01:28:23 localhost kernel: BTRFS: device label home devid 1
transid 2326870 /dev/sdc1
May 08 01:28:26 localhost.localdomain kernel: BTRFS info (device
sdc1): disk space caching is enabled
May 08 01:28:26 localhost.localdomain kernel: BTRFS info (device
sdc1): bdev /dev/sdc1 errs: wr 0, rd 14, flush 0, corrupt 0, gen 0
May 08 01:28:39 localhost.localdomain kernel: BTRFS error (device
sdc1): could not find root 8
May 08 01:40:30 localhost.localdomain kernel: BTRFS warning (device
sdc1): block group 2286562574336 has wrong amount of free space
May 08 01:40:30 localhost.localdomain kernel: BTRFS warning (device
sdc1): failed to load free space cache for block group 2286562574336,
rebuilding it now
May 08 02:06:09 localhost.localdomain kernel: BTRFS warning (device
sdc1): block group 3844117364736 has wrong amount of free space
May 08 02:06:09 localhost.localdomain kernel: BTRFS warning (device
sdc1): failed to load free space cache for block group 3844117364736,
rebuilding it now


These messages appear on every boot.
That they means? And how fix it?

$ uname -r
4.5.3-300.fc24.x86_64

--
Best Regards,
Mike Gavrilov.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: task btrfs-cleaner:770 blocked for more than 120 seconds.

[Михаил Гаврилов]

2016-03-16 14:01 GMT+05:00 Roman Mamedov :
> On Tue, 15 Mar 2016 09:18:14 +0500
> Mikhail Gavrilov  wrote:
>
>> Yesterday btrfs again hangs on kernel 4.4.5
>> It's the same or another problem?
>
> Seems different to me, as there's also defrag involved. Try removing and not
> using the defrag mount option.

How then defragment FS?

>
>> This messages say that restore failed?
>
> So you can't mount the FS anymore? Did you try btrfsck, what does it report?
>

FS is mounted, but after failure mount takes a long time (more than 4
min) and Fedora  stops boot  by timeout (system suggest enter under
root for maintance) after reboot again all fine.

I'm just afraid of losing data after this messages:
[  449.949206] BTRFS warning (device sdc1): failed to load free space
cache for block group 2286562574336, rebuilding it now
[  452.619318] BTRFS: open_ctree failed

or it does not mean anything terrible?

--
Best Regards,
Mike Gavrilov.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: task btrfs-cleaner:770 blocked for more than 120 seconds.

[Михаил Гаврилов]

Sorry, I have not yet had time to apply your patch.

And get hang again when launch web browser.

Here new logs: http://btrfs.sy24.ru/kernel-sysrqw-btrfscleaner770blocked-3.txt

Addition info: when btrfs partition hangs also hang GUI it is normal?
For grab logs I switch to tty3 (Ctrl - Alt - F3)



--
Best Regards,
Mike Gavrilov.


2016-02-13 1:34 GMT+05:00 Liu Bo <bo.li@oracle.com>:
> On Sat, Feb 13, 2016 at 12:15:08AM +0500, Михаил Гаврилов wrote:
>> 2016-02-12 8:22 GMT+05:00 Liu Bo <bo.li@oracle.com>:
>> > You can try it on your 4.2.3 kernel or the latest 4.5, but I guess it
>> > doesn't not fix the real deadlock you're hitting...
>>
>> It means it is not final patch? you continue investigate problem? Can
>> I help you?
>
> Yeah, it's not the final patch, you can apply it and see if the deadlock
> will happen again.
>
> Unless we can get a way to reproduce it, I'm afraid there's little things we 
> can  do here.
>
> Thanks,
>
> -liubo
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: task btrfs-cleaner:770 blocked for more than 120 seconds.

[Михаил Гаврилов]

2016-02-12 8:22 GMT+05:00 Liu Bo :
> You can try it on your 4.2.3 kernel or the latest 4.5, but I guess it
> doesn't not fix the real deadlock you're hitting...

It means it is not final patch? you continue investigate problem? Can
I help you?

--
Best Regards,
Mike Gavrilov.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: task btrfs-cleaner:770 blocked for more than 120 seconds.

[Михаил Гаврилов]

Thanks guys, I appreciate your's work.
In which kernel this patch would landed?

--
Best Regards,
Mike Gavrilov.


2016-02-12 0:18 GMT+05:00 Liu Bo :
>
> Really appreciate for collecting these, it should be helpful.
>
> Unfortunately I still could not figure out who's holding fs tree's root 
> WRITE_LOCK so that others are blocked.
>
> A possible bug in log code (the follwing patch addressed it),
>
> - log_new_dir_dentries() is holding log tree's leaf READ_LOCK and may try
>   to get fs tree's READ_LOCK via btrfs_iget() -> btrfs_lookup().
>   (This is shown in the backtrac)
>
> - btrfs_log_inode() can call btrfs_search_forward() to get fs tree's
>   leaf READ_LOCK and then call copy_items() -> btrfs_insert_empty_items()
>   to acquire WRITE_LOCK of log tree's leaf and leaf's parent.
>   (In the backtrace, this is blocked by item 1 because log_new_dir_dentries is
>   holding a log tree leaf's READ_LOCK() which happens to be sibling to
>   the leaf that btrfs_insert_empty_items() is accessing, when doing
>   split_leaf() it needs to get the sibling's WRITE_LOCK(). )
>
> Thanks,
>
> -liubo
>
> diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
> index 323e12c..4a64fdd 100644
> --- a/fs/btrfs/tree-log.c
> +++ b/fs/btrfs/tree-log.c
> @@ -4956,6 +4956,7 @@ process_leaf:
> if (di_key.type == BTRFS_ROOT_ITEM_KEY)
> continue;
>
> +   btrfs_release_path(path);
> di_inode = btrfs_iget(root->fs_info->sb, _key,
>   root, NULL);
> if (IS_ERR(di_inode)) {
> @@ -4971,7 +4972,6 @@ process_leaf:
> ctx->log_new_dentries = false;
> if (type == BTRFS_FT_DIR)
> log_mode = LOG_INODE_ALL;
> -   btrfs_release_path(path);
> ret = btrfs_log_inode(trans, root, di_inode,
>   log_mode, 0, LLONG_MAX, ctx);
> iput(di_inode);
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: task btrfs-cleaner:770 blocked for more than 120 seconds.

[Михаил Гаврилов]

2016-02-03 9:48 GMT+05:00 Chris Murphy :
> Mike: From your attachment, looks like you rebooted. So do this:
>
> echo 1 > /proc/sys/kernel/sysrq
> Reproduce the problem where you get blocked task messages in dmesg
> echo w > /proc/sysrq-trigger
> journalctl -k > kernel-sysrqw-btrfscleaner770blocked-2.txt
>
>
> Make sure you use the same mount options. Looks like you're using
> autodefrag, and inode_cache. Are there others? And can you say what
> the workload is? Especially because inode_cache is not a default mount
> option and isn't recommended except for certain workloads, but still I
> think it shouldn't hang. But that's a question for Liu Bo.
>
>
> Chris Murphy

Thanks Chris for clarification.
I am not have exactly algorithm for reproducing this.
But it happens with my btrfs partition again.

*Hang occured here*
echo 1 > /proc/sys/kernel/sysrq
Reproduce the problem where you get blocked task messages in dmesg
echo w > /proc/sysrq-trigger
journalctl -k > kernel-sysrqw-btrfscleaner770blocked-2.txt

Here full log: http://btrfs.sy24.ru/kernel-sysrqw-btrfscleaner770blocked-2.txt

I am so sorry if this log is useless.
If "sysrq" is needed enabled before hang then I need set this
permanently because as I said I not having exactly reproducing this.

My mount options:
UUID=82df2d84-bf54-46cb-84ba-c88e93677948 /home btrfs
subvolid=5,autodefrag,noatime,space_cache,inode_cache 0 0


--
Best Regards,
Mike Gavrilov.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

task btrfs-cleaner:770 blocked for more than 120 seconds.

[Михаил Гаврилов]

Hi all! I have problem with hanging btrfs file system.

[17047.652816] INFO: task btrfs-cleaner:770 blocked for more than 120 seconds.
[17047.652837]   Not tainted 4.3.5-300.fc23.x86_64+debug #1
[17047.652853] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[17047.652875] btrfs-cleaner   D 8807fe3d7a18 10264   770  2 0x0080
[17047.652880]  8807e50f7a50 0092 811067cd
8807
[17047.652882]  8807fa344000 8807e68ec000 8807e50f8000
880003753e90
[17047.652884]  8807e68ec000 8807e68ec000 8807e50f7a90
8807e50f7a68
[17047.652886] Call Trace:
[17047.652891]  [] ? trace_hardirqs_on+0xd/0x10
[17047.652894]  [] schedule+0x3a/0x90
[17047.652909]  [] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[17047.652912]  [] ? wake_atomic_t_function+0x70/0x70
[17047.652918]  [] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[17047.652923]  [] btrfs_search_forward+0x59/0x350 [btrfs]
[17047.652927]  [] ? kmem_cache_alloc+0x281/0x310
[17047.652931]  [] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
[17047.652940]  []
find_new_extents.constprop.43+0xa6/0x2d0 [btrfs]
[17047.652942]  [] ? trace_hardirqs_on_caller+0x129/0x1b0
[17047.652943]  [] ? trace_hardirqs_on+0xd/0x10
[17047.652952]  [] btrfs_defrag_file+0x8d8/0xd60 [btrfs]
[17047.652955]  [] ? __sb_start_write+0xc9/0x110
[17047.652963]  [] btrfs_run_defrag_inodes+0x242/0x390 [btrfs]
[17047.652971]  [] ?
btrfs_run_defrag_inodes+0xf0/0x390 [btrfs]
[17047.652978]  [] cleaner_kthread+0xcb/0x210 [btrfs]
[17047.652984]  [] ?
btree_read_extent_buffer_pages.constprop.54+0x110/0x110 [btrfs]
[17047.652986]  [] kthread+0x101/0x120
[17047.652987]  [] ? trace_hardirqs_on_caller+0x129/0x1b0
[17047.652989]  [] ? kthread_create_on_node+0x250/0x250
[17047.652991]  [] ret_from_fork+0x3f/0x70
[17047.652992]  [] ? kthread_create_on_node+0x250/0x250
[17047.652993] 1 lock held by btrfs-cleaner/770:
[17047.652994]  #0:  (sb_writers#16){.+.+.+}, at: []
__sb_start_write+0xc9/0x110
[17047.652999] INFO: task btrfs-transacti:771 blocked for more than 120 seconds.
[17047.653019]   Not tainted 4.3.5-300.fc23.x86_64+debug #1
[17047.653032] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[17047.653053] btrfs-transacti D 8807fe1d7a18  9912   771  2 0x0080
[17047.653056]  8807e50fbb20 0096 811067cd
8807
[17047.653058]  8807fa33c000 8807f714 8807e50fc000
880003753e90
[17047.653060]  8807f714 8807f714 8807e50fbb60
8807e50fbb38
[17047.653062] Call Trace:
[17047.653063]  [] ? trace_hardirqs_on+0xd/0x10
[17047.653065]  [] schedule+0x3a/0x90
[17047.653074]  [] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[17047.653076]  [] ? wake_atomic_t_function+0x70/0x70
[17047.653081]  [] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[17047.653086]  [] btrfs_search_slot+0x73f/0x9f0 [btrfs]
[17047.653092]  [] btrfs_insert_empty_items+0x71/0xc0 [btrfs]
[17047.653101]  []
__btrfs_run_delayed_items+0x135/0x600 [btrfs]
[17047.653104]  [] ? local_clock+0x1c/0x20
[17047.653113]  [] btrfs_run_delayed_items+0x13/0x20 [btrfs]
[17047.653120]  []
btrfs_commit_transaction+0x2f0/0xbd0 [btrfs]
[17047.653127]  [] transaction_kthread+0x208/0x270 [btrfs]
[17047.653134]  [] ?
btrfs_cleanup_transaction+0x610/0x610 [btrfs]
[17047.653135]  [] kthread+0x101/0x120
[17047.653137]  [] ? trace_hardirqs_on_caller+0x129/0x1b0
[17047.653138]  [] ? kthread_create_on_node+0x250/0x250
[17047.653140]  [] ret_from_fork+0x3f/0x70
[17047.653141]  [] ? kthread_create_on_node+0x250/0x250
[17047.653142] 2 locks held by btrfs-transacti/771:
[17047.653143]  #0:  (_info->transaction_kthread_mutex){+.+...},
at: [] transaction_kthread+0x72/0x270 [btrfs]
[17047.653151]  #1:  (_node->mutex){+.+.-.}, at:
[] __btrfs_run_delayed_items+0xfc/0x600 [btrfs]
[17047.653190] INFO: task systemd:2059 blocked for more than 120 seconds.
[17047.653209]   Not tainted 4.3.5-300.fc23.x86_64+debug #1
[17047.653225] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[17047.653246] systemd D 8807fe5d7a18 12696  2059  1 0x
[17047.653250]  8807f6d9f9b8 0082 81f62300
0004
[17047.653252]  880725b84000 8807c0afc000 8807f6da
880003753e90
[17047.653253]  8807c0afc000 8807c0afc000 8807f6d9f9f8
8807f6d9f9d0
[17047.653255] Call Trace:
[17047.653257]  [] schedule+0x3a/0x90
[17047.653266]  [] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[17047.653268]  [] ? wake_atomic_t_function+0x70/0x70
[17047.653273]  [] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[17047.653278]  [] btrfs_search_slot+0x73f/0x9f0 [btrfs]
[17047.653286]  [] ? btrfs_crc32c+0x68/0x90 [btrfs]
[17047.653292]  [] btrfs_lookup_dir_item+0x78/0xc0 [btrfs]
[17047.653300]  [] btrfs_lookup_dentry+0xb6/0x550 [btrfs]
[17047.653302]  [] ? _raw_spin_unlock+0x27/0x40
[17047.653309]  [] btrfs_lookup+0x12/0x40 [btrfs]
[17047.653311]  [] 

Re: Very various speed of grep operation on btrfs partition

[Михаил Гаврилов]

Ok, I am make another experiment. I am buy new HDD and format it with
btrfs file system. Also I increased size of grep data and make bash script
wich automate testing:

#!/bin/bash

#For testing on windows machine
#grep_path='/cygdrive/e/Sources/inside'
#For testing on new HDD
#grep_path='/run/media/mikhail/eaa531cd-25f4-4e00-b31f-22665faa9768/sources/inside'
#For testing in real life
grep_path='/home/mikhail/sources/inside'
command="grep -rn 'float:left;display: block;height: 24px;line-height:
1.2em;position: relative;text-align: center;white-space: nowrap;width:
80px;' '$grep_path'"
log_file='res.log'

exec 3>&1 1>>${log_file} 2>&1
while [ 1 = 1 ]
do

   (( count++ ))
   echo "PASS: $count" at `date +"%T"` | tee /dev/fd/3
   echo $command | tee /dev/fd/3
   eval "{ time $command > /dev/null; } |& tee /dev/fd/3"
done


And get very interesting results:

Linux btrfs with NEW HDD: 6.441s (result as in syntetic tests)
Linux btrfs with real data HDD (used 94%): 16m52.036s Very bad why???
Data are same with first variant.
Windows ntfs NEW HDD: 1m27.643s

I am really disappointed why in real life (home folder) have so bad results
It's possible HDD which is used 94% optimise speed as on empty hard drive?
Both hard disk are same. This is ST4000NM0033-9ZM170.


--
Best Regards,
Mike Gavrilov.

Re: Very various speed of grep operation on btrfs partition

[Михаил Гаврилов]

2015-12-04 17:59 GMT+05:00 Austin S Hemmelgarn :
> Well, what other things are accessing the filesystem at the same time? If
> you've got something like KDE running with the 'semantic desktop' stuff
> turned on, than that will seriously impact the performance of other things
> using that filesystem.
>
> The other thing to keep in mind, is that caching may be impacting things
> somewhat.  To really get a good idea of performance for something like this,
> you should run 'sync' followed by 'echo 3 > /proc/sys/vm/drop_caches'
> (you'll need to be root for the second one) prior to each run, and ideally
> have nothing else running on that filesystem.

Thanks for clarifying.

I was able to further clarify:

After resetting the cache on a clean machine after a reboot grep
operation was take:
real 2m54.549s
user 0m0.662s
sys 0m1.062s

After turning off the indexing service (tracker) result improved:
real 2m12.182s
user 0m0.657s
sys 0m1.021s


If the cache is not cleaned:
real 0m0.575s
user 0m0.467s
sys 0m0.108s


And the result is stable and all subsequent launches, even when the
indexing service is enabled.

A day later noticed that the effect of the cache is missing:
real 4m33.940s
user 0m0.862s
sys 0m1.711s


As I understand to solve my problem just need to do the cache is
always effective, even if memory occupied by other applications.

Is possible to specify minimal size of disk cache?

Pity that I can't do 'echo 3 > /proc/sys/vm/drop_caches' on Windows
machine. It be interesting how fast grep would be work without cache.

> On a separate note, if you're either running on a 64-bit system, or have
> less than about 2^31 files on the FS, inode_cache will slow things down.
> It's intended for stuff like mail spools where you have billions of files
> being created and deleted over a few weeks, and quickly use up the inode
> numbers.  On almost all systems, it will make things run slower, and
> possibly result in non-=deterministic filesystem performance like what you
> are seeing here.

Hmm, less than about 2^31 ? Maybe you mean more?

> Additionally, do you have some particular reason that you absolutely _need_
> nodatacow to be enabled for the FS?  It usually has no impact on
> performance, but it removes any kind of error correction for file data
> (checksums can't be used safely without COW semantics).  It probably has no
> direct impact on what you're seeing here, but it is something that really
> shouldn't be used in most cases at the filesystem level (it can be done on
> given subvolumes or directories, and that's the recommended way to do it if
> you don't want to go down to the per-file level).
>

I see that some issue with btrfs still not closed:
https://code.google.com/p/chromium/issues/detail?id=284738
And gnome-boxes still very slow when COW is enable.



--
Best Regards,
Mike Gavrilov.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Very various speed of grep operation on btrfs partition

[Михаил Гаврилов]

Today on work I needed searching some strings in repository. Only
machine with windows was available. I am was using grep from Cygwin
for this task and I am was surprised about speed of NTFS partition.I
decided to repeat this task on my home Linux workstation.

[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 3m21.262s
user 0m0.914s
sys 0m2.288s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 0m49.921s
user 0m0.736s
sys 0m0.957s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 0m1.077s
user 0m0.667s
sys 0m0.409s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 0m1.062s
user 0m0.657s
sys 0m0.386s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 0m1.020s
user 0m0.641s
sys 0m0.373s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 0m0.941s
user 0m0.593s
sys 0m0.335s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 0m0.918s
user 0m0.594s
sys 0m0.322s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 0m1.053s
user 0m0.620s
sys 0m0.401s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 0m1.049s
user 0m0.625s
sys 0m0.411s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 2m51.858s
user 0m0.885s
sys 0m1.863s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 3m22.615s
user 0m0.876s
sys 0m2.160s
[mikhail@localhost ~]$ time grep -rn 'float:left;display:
block;height: 24px;line-height: 1.2em;position: relative;text-align:
center;white-space: nowrap;width: 80px;'
"/home/mikhail/sources/inside/Модули интерфейса/"
/home/mikhail/sources/inside/Модули интерфейса/Активные
продажи/Темы/Разводящая группы
тем/CRMGroupTheme/CRMGroupTheme.xhtml:38:  

real 1m16.100s
user 0m0.773s
sys 

kernel BUG at fs/btrfs/extent-tree.c:1833! [btrfs]

[Михаил Гаврилов]

kernel BUG at fs/btrfs/extent-tree.c:1833!
invalid opcode:  [#1] SMP
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4
tun nls_utf8 isofs rfcomm fuse nf_conntrack_netbios_ns
nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6
xt_conntrack ebtable_nat ebtable_broute bridge ebtable_filter ebtables
ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6
nf_defrag_ipv6 nf_nat_ipv6 ip6table_security ip6table_filter
ip6_tables iptable_mangle iptable_raw iptable_nat nf_conntrack_ipv4
nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_security bnep
snd_usb_audio snd_usbmidi_lib snd_rawmidi hid_logitech_hidpp btusb
btrtl btbcm btintel bluetooth gspca_zc3xx gspca_main videodev uas
media joydev usb_storage hid_logitech_dj rfkill btrfs xor intel_rapl
iosf_mbi x86_pkg_temp_thermal snd_hda_codec_hdmi snd_hda_codec_realtek
coretemp
 snd_hda_codec_generic snd_hda_codec_ca0132 iTCO_wdt
iTCO_vendor_support kvm_intel vfat ppdev fat kvm snd_hda_intel
snd_hda_codec snd_hda_core snd_hwdep snd_seq crct10dif_pclmul
snd_seq_device crc32_pclmul crc32c_intel snd_pcm raid6_pq snd_timer
snd mei_me mei soundcore shpchp i2c_i801 lpc_ich parport_pc
tpm_infineon parport tpm_tis tpm nfsd auth_rpcgss nfs_acl lockd grace
sunrpc binfmt_misc i915 i2c_algo_bit drm_kms_helper drm 8021q garp stp
llc mrp serio_raw r8169 mii video
CPU: 7 PID: 10775 Comm: kworker/u16:15 Not tainted 4.2.6-301.fc23.x86_64 #1
Hardware name: Gigabyte Technology Co., Ltd. Z87M-D3H/Z87M-D3H, BIOS
F11 08/12/2014
Workqueue: btrfs-extent-refs btrfs_extent_refs_helper [btrfs]
task: 8805c042bb00 ti: 88074d748000 task.ti: 88074d748000
RIP: 0010:[]  []
insert_inline_extent_backref+0xe7/0xf0 [btrfs]
RSP: 0018:88074d74baa8  EFLAGS: 00010293
RAX:  RBX:  RCX: 
RDX: 8800 RSI: 0001 RDI: 
RBP: 88074d74bb28 R08: 4000 R09: 88074d74b9a0
R10:  R11: 0003 R12: 8807ec932800
R13: 88048185d090 R14:  R15: 
FS:  () GS:88081e3c() knlGS:
CS:  0010 DS:  ES:  CR0: 80050033
CR2: 13dd12294020 CR3: 01c0b000 CR4: 001406e0
Stack:
  0005  
 0001 81200206 88074d74bb08 a05dcd4a
 33e8 4e7e87f3 8807ea905800 
Call Trace:
 [] ? kmem_cache_alloc+0x1d6/0x210
 [] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
 [] __btrfs_inc_extent_ref.isra.52+0xa9/0x270 [btrfs]
 [] __btrfs_run_delayed_refs+0xc84/0x1080 [btrfs]
 [] ? mempool_free_slab+0x17/0x20
 [] ? kmem_cache_alloc+0x193/0x210
 [] btrfs_run_delayed_refs.part.73+0x74/0x270 [btrfs]
 [] delayed_ref_async_start+0x7e/0x90 [btrfs]
 [] btrfs_scrubparity_helper+0xc2/0x260 [btrfs]
 [] btrfs_extent_refs_helper+0xe/0x10 [btrfs]
 [] process_one_work+0x19e/0x3f0
 [] worker_thread+0x4e/0x450
 [] ? process_one_work+0x3f0/0x3f0
 [] kthread+0xd8/0xf0
 [] ? kthread_worker_fn+0x160/0x160
 [] ret_from_fork+0x3f/0x70
 [] ? kthread_worker_fn+0x160/0x160
Code: 10 49 89 d9 48 8b 55 c0 4c 89 7c 24 10 4c 89 f1 4c 89 ee 4c 89
e7 89 44 24 08 48 8b 45 20 48 89 04 24 e8 5d d5 ff ff 31 c0 eb ac <0f>
0b e8 92 47 ab e0 66 90 0f 1f 44 00 00 55 48 89 e5 41 57 41
RIP  [] insert_inline_extent_backref+0xe7/0xf0 [btrfs]
 RSP 



Report from downstream: https://bugzilla.redhat.com/show_bug.cgi?id=1287508


--
Best Regards,
Mike Gavrilov.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

INFO: task tracker-store:2686 blocked for more than 120 seconds.

[Михаил Гаврилов]

[  239.881241] INFO: task tracker-store:2686 blocked for more than 120 seconds.
[  239.881245]   Not tainted 4.2.6-300.fc23.x86_64+debug #1
[  239.881246] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  239.881247] tracker-store   D 8807fe1d76d8 11528  2686  1 0x
[  239.881254]  8807a7aef1e8 0096 8807f997b800

[  239.881256]  8807f61faa40 8807a79c5480 8807a7aef1e8
8807a7af
[  239.881258]  8807fe1d76c0 7fff 8186d450
8807a7aef3b0
[  239.881260] Call Trace:
[  239.881267]  [] ? bit_wait+0x50/0x50
[  239.881268]  [] schedule+0x3e/0x90
[  239.881270]  [] schedule_timeout+0x255/0x410
[  239.881274]  [] ? mark_held_locks+0x7d/0xb0
[  239.881279]  [] ? ktime_get+0x7d/0x140
[  239.881281]  [] ? bit_wait+0x50/0x50
[  239.881283]  [] ? trace_hardirqs_on_caller+0x129/0x1b0
[  239.881287]  [] ? read_tsc+0x9/0x10
[  239.881289]  [] ? ktime_get+0xac/0x140
[  239.881292]  [] ? __delayacct_blkio_start+0x23/0x30
[  239.881294]  [] ? bit_wait+0x50/0x50
[  239.881296]  [] io_schedule_timeout+0xa4/0x110
[  239.881297]  [] bit_wait_io+0x35/0x50
[  239.881299]  [] __wait_on_bit+0x5d/0x90
[  239.881303]  [] wait_on_page_bit+0xcf/0xf0
[  239.881305]  [] ? autoremove_wake_function+0x40/0x40
[  239.881329]  []
read_extent_buffer_pages+0x2c9/0x300 [btrfs]
[  239.881340]  [] ? free_root_pointers+0x60/0x60 [btrfs]
[  239.881347]  []
btree_read_extent_buffer_pages.constprop.54+0xac/0x110 [btrfs]
[  239.881354]  [] read_tree_block+0x41/0x70 [btrfs]
[  239.881360]  []
read_block_for_search.isra.34+0x13e/0x380 [btrfs]
[  239.881366]  [] btrfs_search_slot+0x384/0xa10 [btrfs]
[  239.881369]  [] ? trace_hardirqs_on_caller+0x129/0x1b0
[  239.881375]  [] btrfs_lookup_csum+0x4e/0x150 [btrfs]
[  239.881382]  []
__btrfs_lookup_bio_sums.isra.6+0x1b5/0x510 [btrfs]
[  239.881389]  [] ? btrfs_bio_wq_end_io+0x2c/0x80 [btrfs]
[  239.881395]  [] btrfs_lookup_bio_sums+0x36/0x40 [btrfs]
[  239.881402]  [] btrfs_submit_bio_hook+0x131/0x1b0 [btrfs]
[  239.881410]  [] submit_one_bio+0x6e/0xa0 [btrfs]
[  239.881417]  [] submit_extent_page+0xe5/0x1e0 [btrfs]
[  239.881423]  [] __do_readpage+0x39e/0x940 [btrfs]
[  239.881429]  [] ?
btrfs_create_repair_bio+0x100/0x100 [btrfs]
[  239.881436]  [] ?
btrfs_writepage_end_io_hook+0x300/0x300 [btrfs]
[  239.881443]  []
__extent_readpages.constprop.40+0x2c4/0x2e0 [btrfs]
[  239.881445]  [] ? __add_to_page_cache_locked+0x408/0x470
[  239.881451]  [] ?
btrfs_writepage_end_io_hook+0x300/0x300 [btrfs]
[  239.881458]  [] extent_readpages+0x197/0x1f0 [btrfs]
[  239.881464]  [] ?
btrfs_writepage_end_io_hook+0x300/0x300 [btrfs]
[  239.881469]  [] btrfs_readpages+0x1f/0x30 [btrfs]
[  239.881472]  [] __do_page_cache_readahead+0x2af/0x330
[  239.881473]  [] ? __do_page_cache_readahead+0x17d/0x330
[  239.881475]  [] ? local_clock+0x1c/0x20
[  239.881477]  [] ondemand_readahead+0xe3/0x460
[  239.881480]  [] ? inode_congested+0x122/0x210
[  239.881482]  [] page_cache_async_readahead+0x73/0x80
[  239.881483]  [] generic_file_read_iter+0x452/0x610
[  239.881485]  [] ? mark_held_locks+0x7d/0xb0
[  239.881487]  [] __vfs_read+0xcc/0x100
[  239.881488]  [] vfs_read+0x8a/0x140
[  239.881490]  [] SyS_read+0x58/0xd0
[  239.881493]  [] entry_SYSCALL_64_fastpath+0x12/0x76
[  239.881495] 1 lock held by tracker-store/2686:
[  239.881495]  #0:  (>f_pos_lock){+.+.+.}, at:
[] __fdget_pos+0x49/0x50




--
Best Regards,
Mike Gavrilov.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs driver hangs again

[Михаил Гаврилов]

_one_work+0x19b/0x840
[  360.086228]  [] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  360.086231]  [] ? worker_thread+0xd5/0x450
[  360.086235]  [] worker_thread+0x4e/0x450
[  360.086238]  [] ? process_one_work+0x840/0x840
[  360.086241]  [] kthread+0x104/0x120
[  360.086246]  [] ? kthread_create_on_node+0x250/0x250
[  360.086250]  [] ret_from_fork+0x3f/0x70
[  360.086253]  [] ? kthread_create_on_node+0x250/0x250
[  360.086256] 3 locks held by kworker/u16:7/155:
[  360.086258]  #0:  ("%s-%s""btrfs", name){.+.+..}, at:
[] process_one_work+0x19b/0x840
[  360.086265]  #1:  ((>normal_work)){+.+...}, at:
[] process_one_work+0x19b/0x840
[  360.086271]  #2:  (sb_internal){.+.+..}, at: []
start_transaction+0x41c/0x6c0 [btrfs]
[  360.086290] INFO: task kworker/u16:8:192 blocked for more than 120 seconds.
[  360.086293]   Not tainted 4.2.5-300.fc23.x86_64+debug #1
[  360.086294] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  360.086297] kworker/u16:8   D 8807fe1d76d8 11304   192  2 0x
[  360.086316] Workqueue: btrfs-endio-write btrfs_endio_write_helper [btrfs]
[  360.086318]  8807f4f3f848 0096 8807f4f3f818

[  360.086323]  8807fa32aa40 8807f4e6aa40 8807f4f3f868
8807f4f4
[  360.086328]  8807e46f0b90 8807e46f0be8 8807f4f3f890
8807e46f0b48
[  360.086333] Call Trace:
[  360.086337]  [] schedule+0x3e/0x90
[  360.086353]  [] btrfs_tree_lock+0x7d/0x270 [btrfs]
[  360.086357]  [] ? wake_atomic_t_function+0x70/0x70
[  360.086367]  [] btrfs_lock_root_node+0x34/0x50 [btrfs]
[  360.086377]  [] btrfs_search_slot+0x905/0xa10 [btrfs]
[  360.086381]  [] ? mark_held_locks+0x7d/0xb0
[  360.086394]  [] btrfs_lookup_file_extent+0x49/0x60 [btrfs]
[  360.086409]  [] __btrfs_drop_extents+0x187/0xde0 [btrfs]
[  360.086413]  [] ? kmem_cache_alloc+0x33e/0x360
[  360.086422]  [] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
[  360.086436]  []
insert_reserved_file_extent.constprop.62+0xb5/0x310 [btrfs]
[  360.086449]  [] ?
join_transaction.isra.12+0x10e/0x420 [btrfs]
[  360.086463]  [] btrfs_finish_ordered_io+0x2ff/0x650 [btrfs]
[  360.086468]  [] ? sched_clock+0x9/0x10
[  360.086480]  [] finish_ordered_fn+0x15/0x20 [btrfs]
[  360.086495]  [] btrfs_scrubparity_helper+0xf5/0x870 [btrfs]
[  360.086510]  [] btrfs_endio_write_helper+0xe/0x10 [btrfs]
[  360.086513]  [] process_one_work+0x232/0x840
[  360.086517]  [] ? process_one_work+0x19b/0x840
[  360.086520]  [] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  360.086522]  [] ? worker_thread+0xd5/0x450
[  360.086527]  [] worker_thread+0x4e/0x450
[  360.086530]  [] ? process_one_work+0x840/0x840
[  360.086534]  [] kthread+0x104/0x120
[  360.086538]  [] ? kthread_create_on_node+0x250/0x250
[  360.086542]  [] ret_from_fork+0x3f/0x70
[  360.086546]  [] ? kthread_create_on_node+0x250/0x250
[  360.086549] 3 locks held by kworker/u16:8/192:
[  360.086551]  #0:  ("%s-%s""btrfs", name){.+.+..}, at:
[] process_one_work+0x19b/0x840
[  360.086558]  #1:  ((>normal_work)){+.+...}, at:
[] process_one_work+0x19b/0x840
[  360.086564]  #2:  (sb_internal){.+.+..}, at: []
start_transaction+0x41c/0x6c0 [btrfs]
[  360.086594] INFO: task btrfs-transacti:785 blocked for more than 120 seconds.
[  360.086597]   Not tainted 4.2.5-300.fc23.x86_64+debug #1
[  360.086599] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  360.086601] btrfs-transacti D 8807fe1d76d8 10632   785  2 0x0080
[  360.086607]  88003ec63ac8 0092 88003ec63a98

[  360.086612]  8807fa32aa40 8807ecadaa40 88003ec63ae8
88003ec64000
[  360.086617]  88079eb78970 88079eb789c8 88003ec63b10
88079eb78928
[  360.086622] Call Trace:
[  360.086626]  [] schedule+0x3e/0x90
[  360.086642]  [] btrfs_tree_lock+0x7d/0x270 [btrfs]
[  360.086646]  [] ? wake_atomic_t_function+0x70/0x70
[  360.086657]  [] btrfs_search_slot+0x6f8/0xa10 [btrfs]
[  360.086661]  [] ? mutex_lock_nested+0x279/0x3e0
[  360.086676]  [] ?
__btrfs_run_delayed_items+0xfc/0x600 [btrfs]
[  360.086687]  [] btrfs_insert_empty_items+0x71/0xc0 [btrfs]
[  360.086690]  [] ? mutex_unlock+0xe/0x10
[  360.086705]  []
__btrfs_run_delayed_items+0x135/0x600 [btrfs]
[  360.086709]  [] ? sched_clock+0x9/0x10
[  360.086724]  [] btrfs_run_delayed_items+0x13/0x20 [btrfs]
[  360.086738]  []
btrfs_commit_transaction+0x2f0/0xc80 [btrfs]
[  360.086751]  [] transaction_kthread+0x208/0x270 [btrfs]
[  360.086764]  [] ?
btrfs_cleanup_transaction+0x6e0/0x6e0 [btrfs]
[  360.086767]  [] kthread+0x104/0x120
[  360.086772]  [] ? kthread_create_on_node+0x250/0x250
[  360.086776]  [] ret_from_fork+0x3f/0x70
[  360.086780]  [] ? kthread_create_on_node+0x250/0x250
[  360.086783] 2 locks held by btrfs-transacti/785:
[  360.086785]  #0:  (_info->transaction_kthread_mutex){+.+...},
at: [] transaction_kthread+0x72/0x270 [btrfs]
[  360.086800]  #1:  (_node->mutex){+.

btrfs driver hangs again

[Михаил Гаврилов]

Hi!
please, help me.


[  479.827555] INFO: task kworker/u16:2:140 blocked for more than 120 seconds.
[  479.827558]   Not tainted 4.2.5-300.fc23.x86_64+debug #1
[  479.827559] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  479.827560] kworker/u16:2   D 8807fe3d76d8 11800   140  2 0x
[  479.827578] Workqueue: btrfs-endio-write btrfs_endio_write_helper [btrfs]
[  479.827579]  88009c56b848 0096 88009c56b818

[  479.827581]  8807fa32d480 8807f531d480 88009c56b868
88009c56c000
[  479.827583]  8806c347d1f0 8807f531d480 8807f531d480
88009c56b890
[  479.827585] Call Trace:
[  479.827588]  [] schedule+0x3e/0x90
[  479.827597]  [] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[  479.827600]  [] ? wake_atomic_t_function+0x70/0x70
[  479.827604]  [] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[  479.827608]  [] btrfs_search_slot+0x74e/0xa10 [btrfs]
[  479.827610]  [] ? __lock_acquire+0xcb9/0x1d00
[  479.827616]  [] btrfs_lookup_file_extent+0x49/0x60 [btrfs]
[  479.827622]  [] __btrfs_drop_extents+0x187/0xde0 [btrfs]
[  479.827625]  [] ? kmem_cache_alloc+0x33e/0x360
[  479.827628]  [] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
[  479.827634]  []
insert_reserved_file_extent.constprop.62+0xb5/0x310 [btrfs]
[  479.827639]  [] ?
join_transaction.isra.12+0x10e/0x420 [btrfs]
[  479.827645]  [] btrfs_finish_ordered_io+0x2ff/0x650 [btrfs]
[  479.827647]  [] ? sched_clock+0x9/0x10
[  479.827652]  [] finish_ordered_fn+0x15/0x20 [btrfs]
[  479.827659]  [] btrfs_scrubparity_helper+0xf5/0x870 [btrfs]
[  479.827665]  [] btrfs_endio_write_helper+0xe/0x10 [btrfs]
[  479.827667]  [] process_one_work+0x232/0x840
[  479.827668]  [] ? process_one_work+0x19b/0x840
[  479.827669]  [] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  479.827670]  [] ? worker_thread+0xd5/0x450
[  479.827672]  [] worker_thread+0x4e/0x450
[  479.827673]  [] ? process_one_work+0x840/0x840
[  479.827674]  [] kthread+0x104/0x120
[  479.827676]  [] ? kthread_create_on_node+0x250/0x250
[  479.827678]  [] ret_from_fork+0x3f/0x70
[  479.827679]  [] ? kthread_create_on_node+0x250/0x250
[  479.827680] 3 locks held by kworker/u16:2/140:
[  479.827681]  #0:  ("%s-%s""btrfs", name){.+.+..}, at:
[] process_one_work+0x19b/0x840
[  479.827684]  #1:  ((>normal_work)){+.+...}, at:
[] process_one_work+0x19b/0x840
[  479.827687]  #2:  (sb_internal){.+.+..}, at: []
start_transaction+0x41c/0x6c0 [btrfs]
[  479.827694] INFO: task kworker/u16:3:141 blocked for more than 120 seconds.
[  479.827695]   Not tainted 4.2.5-300.fc23.x86_64+debug #1
[  479.827695] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  479.827696] kworker/u16:3   D 8807fe1d76d8 11240   141  2 0x
[  479.827705] Workqueue: btrfs-endio-write btrfs_endio_write_helper [btrfs]
[  479.827706]  88009c56f848 0096 88009c56f818

[  479.827708]  8807fa32aa40 8807f5318000 88009c56f868
88009c57
[  479.827709]  8806c347d1f0 8807f5318000 8807f5318000
88009c56f890
[  479.827711] Call Trace:
[  479.827712]  [] schedule+0x3e/0x90
[  479.827719]  [] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[  479.827720]  [] ? wake_atomic_t_function+0x70/0x70
[  479.827724]  [] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[  479.827728]  [] btrfs_search_slot+0x74e/0xa10 [btrfs]
[  479.827729]  [] ? __lock_acquire+0xcb9/0x1d00
[  479.827735]  [] btrfs_lookup_file_extent+0x49/0x60 [btrfs]
[  479.827741]  [] __btrfs_drop_extents+0x187/0xde0 [btrfs]
[  479.827742]  [] ? kmem_cache_alloc+0x33e/0x360
[  479.827746]  [] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
[  479.827751]  []
insert_reserved_file_extent.constprop.62+0xb5/0x310 [btrfs]
[  479.827756]  [] ?
join_transaction.isra.12+0x10e/0x420 [btrfs]
[  479.827761]  [] btrfs_finish_ordered_io+0x2ff/0x650 [btrfs]
[  479.827763]  [] ? sched_clock+0x9/0x10
[  479.827768]  [] finish_ordered_fn+0x15/0x20 [btrfs]
[  479.827774]  [] btrfs_scrubparity_helper+0xf5/0x870 [btrfs]
[  479.827779]  [] btrfs_endio_write_helper+0xe/0x10 [btrfs]
[  479.827780]  [] process_one_work+0x232/0x840
[  479.827781]  [] ? process_one_work+0x19b/0x840
[  479.827782]  [] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  479.827783]  [] ? worker_thread+0xd5/0x450
[  479.827785]  [] worker_thread+0x4e/0x450
[  479.827786]  [] ? process_one_work+0x840/0x840
[  479.827787]  [] kthread+0x104/0x120
[  479.827789]  [] ? kthread_create_on_node+0x250/0x250
[  479.827790]  [] ret_from_fork+0x3f/0x70
[  479.827792]  [] ? kthread_create_on_node+0x250/0x250
[  479.827793] 3 locks held by kworker/u16:3/141:
[  479.827793]  #0:  ("%s-%s""btrfs", name){.+.+..}, at:
[] process_one_work+0x19b/0x840
[  479.827796]  #1:  ((>normal_work)){+.+...}, at:
[] process_one_work+0x19b/0x840
[  479.827798]  #2:  (sb_internal){.+.+..}, at: []
start_transaction+0x41c/0x6c0 [btrfs]
[  479.827806] INFO: task kworker/u16:7:155 blocked for more than 120 seconds.
[  

btrfs file system stop working

[Михаил Гаврилов]

I see this is the second once in my life:

[  480.007335] INFO: task kworker/u16:1:106 blocked for more than 120 seconds.
[  480.007338]   Not tainted 4.2.3-300.fc23.x86_64+debug #1
[  480.007339] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  480.007340] kworker/u16:1   D 8807fe9d76d8 2   106  2 0x
[  480.007348] Workqueue: writeback wb_workfn (flush-btrfs-1)
[  480.007350]  8807f539f438 0092 8807f539f408

[  480.007351]  8807fa365480 8807f53a 8807f539f458
8807f53a
[  480.007353]  8806b7939eb0 8807f53a 8807f53a
8807f539f480
[  480.007355] Call Trace:
[  480.007358]  [] schedule+0x3e/0x90
[  480.007372]  [] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[  480.007374]  [] ? wake_atomic_t_function+0x70/0x70
[  480.007379]  [] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[  480.007384]  [] btrfs_search_slot+0x74e/0xa10 [btrfs]
[  480.007385]  [] ? _raw_spin_unlock+0x2b/0x40
[  480.007391]  [] btrfs_lookup_file_extent+0x49/0x60 [btrfs]
[  480.007398]  [] run_delalloc_nocow+0x15d/0xb40 [btrfs]
[  480.007400]  [] ? local_clock+0x1c/0x20
[  480.007407]  [] ? test_range_bit+0x48/0x130 [btrfs]
[  480.007410]  [] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  480.007416]  [] ? test_range_bit+0xe5/0x130 [btrfs]
[  480.007422]  [] run_delalloc_range+0x3a0/0x400 [btrfs]
[  480.007429]  [] ?
find_lock_delalloc_range.constprop.46+0x1e7/0x210 [btrfs]
[  480.007435]  []
writepage_delalloc.isra.34+0x105/0x170 [btrfs]
[  480.007441]  [] __extent_writepage+0xf5/0x4a0 [btrfs]
[  480.007446]  []
extent_write_cache_pages.isra.31.constprop.44+0x30a/0x400 [btrfs]
[  480.007452]  [] extent_writepages+0x5c/0x90 [btrfs]
[  480.007458]  [] ?
btrfs_writepage_end_io_hook+0x300/0x300 [btrfs]
[  480.007464]  [] btrfs_writepages+0x28/0x30 [btrfs]
[  480.007466]  [] do_writepages+0x21/0x30
[  480.007468]  [] __writeback_single_inode+0x70/0xa20
[  480.007469]  [] writeback_sb_inodes+0x2d6/0x6d0
[  480.007471]  [] __writeback_inodes_wb+0x8c/0xc0
[  480.007473]  [] wb_writeback+0x3cd/0x720
[  480.007475]  [] ? find_next_bit+0x15/0x20
[  480.007476]  [] wb_workfn+0x405/0x8d0
[  480.007479]  [] process_one_work+0x232/0x840
[  480.007480]  [] ? process_one_work+0x19b/0x840
[  480.007482]  [] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  480.007484]  [] ? worker_thread+0xd5/0x450
[  480.007485]  [] worker_thread+0x4e/0x450
[  480.007487]  [] ? process_one_work+0x840/0x840
[  480.007488]  [] kthread+0x104/0x120
[  480.007490]  [] ? kthread_create_on_node+0x250/0x250
[  480.007491]  [] ret_from_fork+0x3f/0x70
[  480.007493]  [] ? kthread_create_on_node+0x250/0x250
[  480.007494] 4 locks held by kworker/u16:1/106:
[  480.007495]  #0:  ("writeback"){.+.+.+}, at: []
process_one_work+0x19b/0x840
[  480.007499]  #1:  ((&(>dwork)->work)){+.+.+.}, at:
[] process_one_work+0x19b/0x840
[  480.007501]  #2:  (>s_umount_key#57){.+.+..}, at:
[] trylock_super+0x1b/0x50
[  480.007505]  #3:  (sb_internal#2){.+.+..}, at: []
start_transaction+0x41c/0x6c0 [btrfs]
[  480.007514] INFO: task kworker/u16:2:140 blocked for more than 120 seconds.
[  480.007515]   Not tainted 4.2.3-300.fc23.x86_64+debug #1
[  480.007515] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  480.007516] kworker/u16:2   D 8807fe5d76d8 11800   140  2 0x
[  480.007526] Workqueue: btrfs-endio-write btrfs_endio_write_helper [btrfs]
[  480.007527]  8807f4dc3848 0096 8807f4dc3818

[  480.007529]  8807fa36 8807f534aa40 8807f4dc3868
8807f4dc4000
[  480.007530]  8806b7939eb0 8807f534aa40 8807f534aa40
8807f4dc3890
[  480.007532] Call Trace:
[  480.007533]  [] schedule+0x3e/0x90
[  480.007539]  [] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[  480.007541]  [] ? wake_atomic_t_function+0x70/0x70
[  480.007545]  [] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[  480.007549]  [] btrfs_search_slot+0x74e/0xa10 [btrfs]
[  480.007551]  [] ? __lock_acquire+0xcb9/0x1d00
[  480.007556]  [] btrfs_lookup_file_extent+0x49/0x60 [btrfs]
[  480.007562]  [] __btrfs_drop_extents+0x187/0xde0 [btrfs]
[  480.007564]  [] ? kmem_cache_alloc+0x33e/0x360
[  480.007567]  [] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
[  480.007573]  []
insert_reserved_file_extent.constprop.62+0xb5/0x310 [btrfs]
[  480.007578]  [] ?
join_transaction.isra.12+0x10e/0x420 [btrfs]
[  480.007583]  [] btrfs_finish_ordered_io+0x2ff/0x650 [btrfs]
[  480.007586]  [] ? sched_clock+0x9/0x10
[  480.007591]  [] finish_ordered_fn+0x15/0x20 [btrfs]
[  480.007598]  [] btrfs_scrubparity_helper+0xf5/0x870 [btrfs]
[  480.007603]  [] btrfs_endio_write_helper+0xe/0x10 [btrfs]
[  480.007605]  [] process_one_work+0x232/0x840
[  480.007607]  [] ? process_one_work+0x19b/0x840
[  480.007608]  [] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  480.007610]  [] ? worker_thread+0xd5/0x450
[  480.007612]  [] worker_thread+0x4e/0x450
[  480.007613]  [] ? 

INFO: task btrfs-transacti:753 blocked for more than 120 seconds. (occurred not first time)

[Михаил Гаврилов]

After this system hangs.
dmesg log I am get through ssh and root user.
For /home partition used btrfs file system.


[174914.234404] INFO: task btrfs-transacti:753 blocked for more than
120 seconds.
[174914.234408]   Not tainted 4.2.0-0.rc8.git0.1.fc23.x86_64+debug #1
[174914.234408] echo 0  /proc/sys/kernel/hung_task_timeout_secs
disables this message.
[174914.234409] btrfs-transacti D 8807fe1d76d8  9640   753  2 0x0080
[174914.234414]  8807e2c23a38 0092 8807e2c23a08

[174914.234416]  8807fa342a40 8807f4f72a40 8807e2c23a58
8807e2c24000
[174914.234418]  8801837aab70 8807f4f72a40 8807f4f72a40
8807e2c23a80
[174914.234420] Call Trace:
[174914.234424]  [8186ad9e] schedule+0x3e/0x90
[174914.234440]  [a07ac342] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[174914.234442]  [810fbf00] ? wake_atomic_t_function+0x70/0x70
[174914.234447]  [a073f834]
btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[174914.234451]  [a0744bbe] btrfs_search_slot+0x74e/0xa10 [btrfs]
[174914.234452]  [81107edd] ? mark_held_locks+0x7d/0xb0
[174914.234458]  [a076181e] btrfs_lookup_inode+0x3e/0xc0 [btrfs]
[174914.234460]  [811080cd] ? trace_hardirqs_on+0xd/0x10
[174914.234468]  [a07ca0d9]
__btrfs_update_delayed_inode+0x69/0x210 [btrfs]
[174914.234475]  [a07ca7a8] ?
__btrfs_run_delayed_items+0x528/0x600 [btrfs]
[174914.234477]  [811080cd] ? trace_hardirqs_on+0xd/0x10
[174914.234483]  [a07ca7cd]
__btrfs_run_delayed_items+0x54d/0x600 [btrfs]
[174914.234485]  [81026d49] ? sched_clock+0x9/0x10
[174914.234491]  [a07caed3] btrfs_run_delayed_items+0x13/0x20 [btrfs]
[174914.234498]  [a076d213]
btrfs_commit_transaction+0x2e3/0xc70 [btrfs]
[174914.234504]  [a07681a8] transaction_kthread+0x208/0x270 [btrfs]
[174914.234509]  [a0767fa0] ?
btrfs_cleanup_transaction+0x6e0/0x6e0 [btrfs]
[174914.234511]  [810d3694] kthread+0x104/0x120
[174914.234513]  [810d3590] ? kthread_create_on_node+0x250/0x250
[174914.234515]  [81871cdf] ret_from_fork+0x3f/0x70
[174914.234516]  [810d3590] ? kthread_create_on_node+0x250/0x250
[174914.234517] 2 locks held by btrfs-transacti/753:
[174914.234518]  #0:  (fs_info-transaction_kthread_mutex){+.+...},
at: [a0768012] transaction_kthread+0x72/0x270 [btrfs]
[174914.234525]  #1:  (delayed_node-mutex){+.+.-.}, at:
[a07ca7a8] __btrfs_run_delayed_items+0x528/0x600 [btrfs]
[174914.234554] INFO: task kworker/u16:9:22966 blocked for more than
120 seconds.
[174914.234555]   Not tainted 4.2.0-0.rc8.git0.1.fc23.x86_64+debug #1
[174914.234556] echo 0  /proc/sys/kernel/hung_task_timeout_secs
disables this message.
[174914.234556] kworker/u16:9   D 8807fe5d76d8  9112 22966  2 0x
[174914.234566] Workqueue: btrfs-endio-write btrfs_endio_write_helper [btrfs]
[174914.234567]  88005e6c7848 0096 88005e6c7818

[174914.234569]  8807fa36 8800185caa40 88005e6c7868
88005e6c8000
[174914.234570]  8801837aab70 8800185caa40 8800185caa40
88005e6c7890
[174914.234572] Call Trace:
[174914.234574]  [8186ad9e] schedule+0x3e/0x90
[174914.234580]  [a07ac342] btrfs_tree_read_lock+0x102/0x160 [btrfs]
[174914.234582]  [810fbf00] ? wake_atomic_t_function+0x70/0x70
[174914.234586]  [a073f834]
btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[174914.234590]  [a0744bbe] btrfs_search_slot+0x74e/0xa10 [btrfs]
[174914.234591]  [811090e9] ? __lock_acquire+0xcb9/0x1d00
[174914.234597]  [a075f8d9] btrfs_lookup_file_extent+0x49/0x60 [btrfs]
[174914.234604]  [a0781ff7] __btrfs_drop_extents+0x187/0xde0 [btrfs]
[174914.234606]  [8124ce1e] ? kmem_cache_alloc+0x33e/0x360
[174914.234609]  [a073f3da] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
[174914.234615]  [a07709e5]
insert_reserved_file_extent.constprop.62+0xb5/0x310 [btrfs]
[174914.234621]  [a076bcbe] ?
join_transaction.isra.12+0x10e/0x420 [btrfs]
[174914.234627]  [a0776caf]
btrfs_finish_ordered_io+0x2ff/0x650 [btrfs]
[174914.234629]  [81026d49] ? sched_clock+0x9/0x10
[174914.234634]  [a0777295] finish_ordered_fn+0x15/0x20 [btrfs]
[174914.234641]  [a07a17b5]
btrfs_scrubparity_helper+0xf5/0x870 [btrfs]
[174914.234647]  [a07a1fbe] btrfs_endio_write_helper+0xe/0x10 [btrfs]
[174914.234649]  [810cc3f2] process_one_work+0x232/0x840
[174914.234651]  [810cc35b] ? process_one_work+0x19b/0x840
[174914.234653]  [811284cd] ? debug_lockdep_rcu_enabled+0x1d/0x20
[174914.234655]  [810ccad5] ? worker_thread+0xd5/0x450
[174914.234656]  [810cca4e] worker_thread+0x4e/0x450
[174914.234658]  [810cca00] ? process_one_work+0x840/0x840
[174914.234659]  [810d3694] kthread+0x104/0x120
[174914.234661]  [810d3590] ? 

btrfs check --repair couldn't solve problem with btrfs partition

[Михаил Гаврилов]

Hi!

Several times I try repair my btrfs partition with follow commands:
[root@localhost ~]# umount /home

root@localhost ~]# btrfs check --repair /dev/sdb1
enabling repair mode
Fixed 0 roots.
Checking filesystem on /dev/sdb1
UUID: 82df2d84-bf54-46cb-84ba-c88e93677948
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
root 5 inode 14121739 errors 2000, link count wrong
unresolved ref dir 12843363 index 36917 namelen 11 name Preferences
filetype 0 errors 3, no dir item, no dir index
root 5 inode 14121759 errors 2000, link count wrong
unresolved ref dir 139955 index 340700 namelen 18 name
session_state.xspf filetype 0 errors 3, no dir item, no dir index
found 2897923331138 bytes used err is 1
total csum bytes: 3100962412
total tree bytes: 4638130176
total fs tree bytes: 950992896
total extent tree bytes: 142475264
btree space waste bytes: 608962027
file data blocks allocated: 33183423946752
 referenced 3177308782592

Btrfs v3.17


# rpm -q btrfs-progs
btrfs-progs-3.17-1.fc21.x86_64


And every time I see this messages.



--
Best Regards,
Mike Gavrilov.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html