Re: [PATCH] btrfs: remove pointless debugfs interface

[Eric Sandeen]

On 8/31/16 2:08 PM, David Sterba wrote:
> On Wed, Aug 31, 2016 at 10:13:49AM -0500, Eric Sandeen wrote:
>> A /sys/kernel/debug/btrfs/test file was added nearly
>> two and a half years ago, but it serves no purpose;
> 
> It does. Introduced in 1bae30982bc86ab66d61ccb6e22792593b45d44d says
> something about helping developers to easily export information from the
> filesystem, to aid debugging. Writing the debugfs support code is not
> obviously trivial, so it's idling in the source. Exporing a new value is
> as easy as copy and update 3 lines of code. If you have no use for it,
> fine.

I had thought that Documentation/filesystems/debugfs.txt would suffice,
but if you keep stuff lying around in btrfs just in case somebody needs
to export a global variable in the future, I suppose that's cool too.  ;)

>> it stores and returns a value, but nothing in the btrfs
>> code uses this value in any way.  There are no other btrfs
>> files in this debugfs dir.
>>
>> This was brought to my attention because it is world-writable;
>> it is the only such file under /sys/kernel/debug, and without
>> knowledge of its purpose, some users were alarmed by this.
> 
> So let's fix the permissions.

*shrug* ok.

-Eric

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: remove pointless debugfs interface

[Jeff Mahoney]

On 8/31/16 3:08 PM, David Sterba wrote:
> On Wed, Aug 31, 2016 at 10:13:49AM -0500, Eric Sandeen wrote:
>> A /sys/kernel/debug/btrfs/test file was added nearly
>> two and a half years ago, but it serves no purpose;
> 
> It does. Introduced in 1bae30982bc86ab66d61ccb6e22792593b45d44d says
> something about helping developers to easily export information from the
> filesystem, to aid debugging. Writing the debugfs support code is not
> obviously trivial, so it's idling in the source. Exporing a new value is
> as easy as copy and update 3 lines of code. If you have no use for it,
> fine.
> 
>> it stores and returns a value, but nothing in the btrfs
>> code uses this value in any way.  There are no other btrfs
>> files in this debugfs dir.
>>
>> This was brought to my attention because it is world-writable;
>> it is the only such file under /sys/kernel/debug, and without
>> knowledge of its purpose, some users were alarmed by this.
> 
> So let's fix the permissions.

Perhaps we can also just stick it behind a CONFIG option as well if the
intention is to keep it around for developer debugging purposes.

-Jeff

-- 
Jeff Mahoney
SUSE Labs



signature.asc
Description: OpenPGP digital signature

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

[Ronan Arraes Jardim Chagas]

Hi guys!

And the problem happened again. This time, I was only using Mozilla
Firefox. I could get the very first message after the error. I hope it
brings more information:

[28039.672199] [ cut here ]
[28039.672253] WARNING: CPU: 3 PID: 31800 at ../fs/btrfs/qgroup.c:2667
btrfs_qgroup_free_meta+0x88/0x90 [btrfs]
[28039.672255] Modules linked in: fuse nf_log_ipv6 xt_pkttype
nf_log_ipv4 nf_log_common xt_LOG xt_limit af_packet iscsi_ibft
iscsi_boot_sysfs msr ip6t_REJECT nf_reject_ipv6 xt_tcpudp
nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_raw ipt_REJECT nf_reject_ipv4
iptable_raw xt_CT nvidia_drm(PO) nvidia_modeset(PO) iptable_filter
nvidia(PO) ip6table_mangle nf_conntrack_netbios_ns
nf_conntrack_broadcast drm_kms_helper nf_conntrack_ipv4 drm
nf_defrag_ipv4 fb_sys_fops snd_hda_codec_hdmi joydev
snd_hda_codec_realtek ip_tables syscopyarea snd_hda_codec_generic
xt_conntrack snd_hda_intel sysfillrect intel_rapl sb_edac edac_core
snd_hda_codec hp_wmi x86_pkg_temp_thermal intel_powerclamp snd_hda_core
snd_hwdep nf_conntrack sparse_keymap sysimgblt coretemp kvm_intel kvm
rfkill irqbypass snd_pcm snd_timer crct10dif_pclmul
[28039.672305]  e1000e crc32_pclmul ghash_clmulni_intel snd aesni_intel
ip6table_filter aes_x86_64 lrw gf128mul glue_helper ablk_helper
iTCO_wdt iTCO_vendor_support mei_wdt ioatdma pcspkr cryptd ip6_tables
ptp lpc_ich fjes i2c_i801 dca mfd_core soundcore pps_core shpchp
tpm_infineon tpm_tis tpm mei_me mei x_tables btrfs xor raid6_pq
hid_generic usbhid crc32c_intel serio_raw xhci_pci ehci_pci sr_mod
firewire_ohci xhci_hcd ehci_hcd cdrom firewire_core crc_itu_t isci
usbcore usb_common libsas ata_generic mpt3sas raid_class
scsi_transport_sas wmi button sg
[28039.672373] CPU: 3 PID: 31800 Comm: gnome-terminal- Tainted:
PW  O4.7.1-1-default #1
[28039.672375] Hardware name: Hewlett-Packard HP Z820 Workstation/158B,
BIOS J63 v03.65 12/19/2013
[28039.672378]   81393104 

[28039.672382]  8107ca1e 881008780800 00014000
881008780800
[28039.672386]  ffe4 88100b297c00 88053b7e3540
a02c9f58
[28039.672390] Call Trace:
[28039.672406]  [] dump_trace+0x5e/0x320
[28039.672413]  [] show_stack_log_lvl+0x10c/0x180
[28039.672419]  [] show_stack+0x21/0x40
[28039.672425]  [] dump_stack+0x5c/0x78
[28039.672430]  [] __warn+0xbe/0xe0
[28039.672461]  [] btrfs_qgroup_free_meta+0x88/0x90
[btrfs]
[28039.672492]  [] start_transaction+0x3c3/0x4f0
[btrfs]
[28039.672521]  [] btrfs_create+0x38/0x1d0 [btrfs]
[28039.672528]  [] path_openat+0x139b/0x14a0
[28039.672535]  [] do_filp_open+0x7e/0xe0
[28039.672541]  [] do_sys_open+0x124/0x1f0
[28039.672547]  []
entry_SYSCALL_64_fastpath+0x1e/0xa8
[28039.676186] DWARF2 unwinder stuck at
entry_SYSCALL_64_fastpath+0x1e/0xa8

Best regards,
Ronan
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

[Chris Murphy]

On Wed, Aug 31, 2016 at 2:49 PM, Ronan Arraes Jardim Chagas
 wrote:
> Hi guys!
>
> And the problem happened again. This time, I was only using Mozilla
> Firefox. I could get the very first message after the error. I hope it
> brings more information:
>
> [28039.672199] [ cut here ]
> [28039.672253] WARNING: CPU: 3 PID: 31800 at ../fs/btrfs/qgroup.c:2667
> btrfs_qgroup_free_meta+0x88/0x90 [btrfs]


Does this file system have quota enabled?

I'm testing this right now and can't even figure out how to determine
when quota is enabled on a Btrfs file system. There's enable, disable,
and rescan. If it's enabled or disabled, I get the same message if I
rescan. If I mount the file system with quota previously enabled,
there is no mount time notification that quota is enabled.

I sincerely hope opensuse isn't enabled quota by default.



-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs-progs 4.7, check reports many "incorrect local backref count" messages

[Chris Murphy]

This is still happening with btrfs-progs 4.7.1 and there is zero
information in the long result what to do about the problem, and
whether it's sane to try have --repair fix it, let alone what the
original cause of the problem was.



-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

[Chris Murphy]

OK it looks like with -w flag I can get a reliable indication of
whether quota is enabled or not:

[root@f24s ~]# btrfs quota enable /mnt/0
[root@f24s ~]# btrfs quota rescan -w /mnt/0
quota rescan started
[root@f24s ~]# btrfs quota disable /mnt/0
[root@f24s ~]# btrfs quota rescan -w /mnt/0
ERROR: quota rescan failed: Invalid argument


So if you did not enable quota support, and aren't sure if it's
enabled you can try 'btrfs quota rescan -w ' but this might
actually be a bad idea, a rescan could take a while if you're actually
using quotas, I have no idea because I don't use them.

Perhaps someone can point out an easier way to determine whether
quotas are enabled?



Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2] btrfs: fix perms on demonstration debugfs interface

[Eric Sandeen]

btrfs provides a helpful demonstration of how to export
a global variable via debugfs; however, it is unique among
other debugfs files in that it is world-writable, which causes
some concern to people who are not familiar with its purpose.

Fix it so that it is only user-writable.

Signed-off-by: Eric Sandeen 
---

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 4879656..fb84685 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -834,7 +834,7 @@ static int btrfs_init_debugfs(void)
if (!btrfs_debugfs_root_dentry)
return -ENOMEM;
 
-   debugfs_create_u64("test", S_IRUGO | S_IWUGO, btrfs_debugfs_root_dentry,
+   debugfs_create_u64("test", S_IRUGO | S_IWUSR, btrfs_debugfs_root_dentry,
_debugfs_test);
 #endif
return 0;

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: fix endless loop in balancing block groups

[Liu Bo]

On Thu, Sep 01, 2016 at 08:32:00AM +0800, Qu Wenruo wrote:
> 
> 
> At 09/01/2016 07:43 AM, Liu Bo wrote:
> > Qgroup function may overwrite the saved error 'err' with 0
> > in case quota is not enabled, and this ends up with a
> > endless loop in balance because we keep going back to balance
> > the same block group.
> 
> In which case?
> 
> If join_trans() fails, we won't go through qgroup fix.
> And before join_trans(), they all go to out_free/out tag.
> Nothing to do with qgroup fix.

I don't think so.

It doesn't always go to out_free, in the while() loop, it'd break the
loop on any error and record it in @err, then go through
prepare_to_merge() + merge_reloc_roots() and other stuff.

Here's an example for keeping err after the above while loop(),
...
err = prepare_to_merge(rc, err);
...

Thanks,

-liubo

> 
> And if we hit qgroup_fix_relocated_data_extents(), then 'err' is alreayd 0,
> nothing we need to save.
> 
> And even for quota disabled case, qgroup_fix_relocated_data_extents() will
> just return 0. Nothing wrong at all.
> 
> Or did I miss something?
> 
> Thanks,
> Qu
> > 
> > It really should use 'ret' instead.
> > 
> > Signed-off-by: Liu Bo 
> > ---
> >  fs/btrfs/relocation.c | 8 +---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> > 
> > diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
> > index 8a2c2a0..c0c13dc 100644
> > --- a/fs/btrfs/relocation.c
> > +++ b/fs/btrfs/relocation.c
> > @@ -4200,9 +4200,11 @@ restart:
> > err = PTR_ERR(trans);
> > goto out_free;
> > }
> > -   err = qgroup_fix_relocated_data_extents(trans, rc);
> > -   if (err < 0) {
> > -   btrfs_abort_transaction(trans, err);
> > +   ret = qgroup_fix_relocated_data_extents(trans, rc);
> > +   if (ret < 0) {
> > +   btrfs_abort_transaction(trans, ret);
> > +   if (!err)
> > +   err = ret;
> > goto out_free;
> > }
> > btrfs_commit_transaction(trans, rc->extent_root);
> > 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

[Chris Murphy]

On Wed, Aug 31, 2016 at 5:03 PM, Jeff Mahoney  wrote:
> On 8/31/16 6:58 PM, Chris Murphy wrote:

> Does Ronan's call trace showing
>> /fs/btrfs/qgroup.c:2667
>>> btrfs_qgroup_free_meta implicate qgroups as a possible source of his 
>>> problem? That trace would only happen if quotas were enabled, right?
>>
>
> Yeah.  That warning doesn't get checked unless they're enabled.

OK so Ronan, I'm gonna guess the simplest work around for your problem
is to disable quota support, and see if the problem happens again.

If it doesn't happen again then it sounds like the reproduce steps are:

a. enable quota support
b. do something metadata heavy workload that's also maybe hitting
fsync; from opensuse list the example that sometimes causes it:


  osc co home:Ronis_BR/julia
  cd home:Ronis_BR/julia
  osc build --root=`pwd`/jail openSUSE_Tumbleweed x86_64

I wonder if it's easier to hit it on a hard drive, slower fsyncs?

-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[no subject]

[Fennec Fox]

Linux Titanium 4.7.2-1-MANJARO #1 SMP PREEMPT Sun Aug 21 15:04:37 UTC
2016 x86_64 GNU/Linux
btrfs-progs v4.7

Data, single: total=30.01GiB, used=18.95GiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=1.01GiB, used=422.17MiB
GlobalReserve, single: total=144.00MiB, used=0.00B

{02:50} Wed Aug 31
[fennectech@Titanium ~]$  sudo fstrim -v /
[sudo] password for fennectech:
Sorry, try again.
[sudo] password for fennectech:
/: 99.8 GiB (107167244288 bytes) trimmed

{03:08} Wed Aug 31
[fennectech@Titanium ~]$  sudo fstrim -v /
[sudo] password for fennectech:
/: 99.9 GiB (107262181376 bytes) trimmed

  I ran these commands minutes after echother ane each time it is
trimming the entire free space

Anyone else seen this?   the filesystem is the root FS and is compressed

-- 
Fennec
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

[Chris Murphy]

On Wed, Aug 31, 2016 at 4:47 PM, Jeff Mahoney  wrote:
> On 8/31/16 5:48 PM, Chris Murphy wrote:
>> OK it looks like with -w flag I can get a reliable indication of
>> whether quota is enabled or not:
>>
>> [root@f24s ~]# btrfs quota enable /mnt/0
>> [root@f24s ~]# btrfs quota rescan -w /mnt/0
>> quota rescan started
>> [root@f24s ~]# btrfs quota disable /mnt/0
>> [root@f24s ~]# btrfs quota rescan -w /mnt/0
>> ERROR: quota rescan failed: Invalid argument
>>
>>
>> So if you did not enable quota support, and aren't sure if it's
>> enabled you can try 'btrfs quota rescan -w ' but this might
>> actually be a bad idea, a rescan could take a while if you're actually
>> using quotas, I have no idea because I don't use them.
>
> It can take a while, but the code is smart enough not to get too much in
> the way of other activity.  It maintains a progress marker and only does
> live accounting on extents that have already been scanned.
>
>> Perhaps someone can point out an easier way to determine whether
>> quotas are enabled?
>
> btrfs qgroup show 

Wow, thanks but that's not obvious at all. man btrfs quota is
described as "btrfs-quota - control the global quota status of a btrfs
filesystem" so it stands to reason the state command for whether it's
enabled or disabled would be in that subcommand not in some other
subcommand.

But this is sidetracking. Does Ronan's call trace showing
/fs/btrfs/qgroup.c:2667
> btrfs_qgroup_free_meta implicate qgroups as a possible source of his problem? 
> That trace would only happen if quotas were enabled, right?

-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

[Jeff Mahoney]

On 8/31/16 6:58 PM, Chris Murphy wrote:
> On Wed, Aug 31, 2016 at 4:47 PM, Jeff Mahoney  wrote:
>> On 8/31/16 5:48 PM, Chris Murphy wrote:
>>> OK it looks like with -w flag I can get a reliable indication of
>>> whether quota is enabled or not:
>>>
>>> [root@f24s ~]# btrfs quota enable /mnt/0
>>> [root@f24s ~]# btrfs quota rescan -w /mnt/0
>>> quota rescan started
>>> [root@f24s ~]# btrfs quota disable /mnt/0
>>> [root@f24s ~]# btrfs quota rescan -w /mnt/0
>>> ERROR: quota rescan failed: Invalid argument
>>>
>>>
>>> So if you did not enable quota support, and aren't sure if it's
>>> enabled you can try 'btrfs quota rescan -w ' but this might
>>> actually be a bad idea, a rescan could take a while if you're actually
>>> using quotas, I have no idea because I don't use them.
>>
>> It can take a while, but the code is smart enough not to get too much in
>> the way of other activity.  It maintains a progress marker and only does
>> live accounting on extents that have already been scanned.
>>
>>> Perhaps someone can point out an easier way to determine whether
>>> quotas are enabled?
>>
>> btrfs qgroup show 
> 
> Wow, thanks but that's not obvious at all. man btrfs quota is
> described as "btrfs-quota - control the global quota status of a btrfs
> filesystem" so it stands to reason the state command for whether it's
> enabled or disabled would be in that subcommand not in some other
> subcommand.

Agreed.  The tools interface has some warts.

> But this is sidetracking. Does Ronan's call trace showing
> /fs/btrfs/qgroup.c:2667
>> btrfs_qgroup_free_meta implicate qgroups as a possible source of his 
>> problem? That trace would only happen if quotas were enabled, right?
> 

Yeah.  That warning doesn't get checked unless they're enabled.

-Jeff

-- 
Jeff Mahoney
SUSE Labs



signature.asc
Description: OpenPGP digital signature

Re:

[Jeff Mahoney]

On 8/31/16 10:02 PM, Fennec Fox wrote:
> Linux Titanium 4.7.2-1-MANJARO #1 SMP PREEMPT Sun Aug 21 15:04:37 UTC
> 2016 x86_64 GNU/Linux
> btrfs-progs v4.7
> 
> Data, single: total=30.01GiB, used=18.95GiB
> System, single: total=4.00MiB, used=16.00KiB
> Metadata, single: total=1.01GiB, used=422.17MiB
> GlobalReserve, single: total=144.00MiB, used=0.00B
> 
> {02:50} Wed Aug 31
> [fennectech@Titanium ~]$  sudo fstrim -v /
> [sudo] password for fennectech:
> Sorry, try again.
> [sudo] password for fennectech:
> /: 99.8 GiB (107167244288 bytes) trimmed
> 
> {03:08} Wed Aug 31
> [fennectech@Titanium ~]$  sudo fstrim -v /
> [sudo] password for fennectech:
> /: 99.9 GiB (107262181376 bytes) trimmed
> 
>   I ran these commands minutes after echother ane each time it is
> trimming the entire free space
> 
> Anyone else seen this?   the filesystem is the root FS and is compressed
> 

Yes.  It's working as intended.  We don't track what space has already
been trimmed anywhere, so it trims all unallocated space.

-Jeff

-- 
Jeff Mahoney
SUSE Labs



signature.asc
Description: OpenPGP digital signature

[PATCH] btrfs: add dynamic debug support

[Jeff Mahoney]

We can re-use the dynamic debugging descriptor to make use of the dynamic
debugging mechanism but still use our own printk interface.

Defining the DEBUG macro works as it did before.  When it's defined,
all of the messages default to print.  We can also enable all debug
messages at boot or module-load time using the 'dyndbg' and
'btrfs.dyndbg' options.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/ctree.h | 31 ++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index eff3993..abb274d 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -37,6 +37,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "extent_io.h"
 #include "extent_map.h"
 #include "async-thread.h"
@@ -3314,7 +3315,35 @@ void btrfs_printk(const struct btrfs_fs_info *fs_info, 
const char *fmt, ...)
btrfs_printk_ratelimited(fs_info, KERN_NOTICE fmt, ##args)
 #define btrfs_info_rl(fs_info, fmt, args...) \
btrfs_printk_ratelimited(fs_info, KERN_INFO fmt, ##args)
-#ifdef DEBUG
+
+#if defined(CONFIG_DYNAMIC_DEBUG)
+#define btrfs_debug(fs_info, fmt, args...) \
+do {   \
+DEFINE_DYNAMIC_DEBUG_METADATA(descriptor, fmt);\
+if (unlikely(descriptor.flags & _DPRINTK_FLAGS_PRINT)) \
+   btrfs_printk(fs_info, KERN_DEBUG fmt, ##args);  \
+} while (0)
+#define btrfs_debug_in_rcu(fs_info, fmt, args...)  \
+do {   \
+DEFINE_DYNAMIC_DEBUG_METADATA(descriptor, fmt);\
+if (unlikely(descriptor.flags & _DPRINTK_FLAGS_PRINT)) 
\
+   btrfs_printk_in_rcu(fs_info, KERN_DEBUG fmt, ##args);   \
+} while (0)
+#define btrfs_debug_rl_in_rcu(fs_info, fmt, args...)   \
+do {   \
+DEFINE_DYNAMIC_DEBUG_METADATA(descriptor, fmt);\
+if (unlikely(descriptor.flags & _DPRINTK_FLAGS_PRINT)) \
+   btrfs_printk_rl_in_rcu(fs_info, KERN_DEBUG fmt, \
+  ##args);\
+} while (0)
+#define btrfs_debug_rl(fs_info, fmt, args...)  \
+do {   \
+DEFINE_DYNAMIC_DEBUG_METADATA(descriptor, fmt);\
+if (unlikely(descriptor.flags & _DPRINTK_FLAGS_PRINT)) \
+   btrfs_printk_ratelimited(fs_info, KERN_DEBUG fmt,   \
+##args);   \
+} while (0)
+#elif defined(DEBUG)
 #define btrfs_debug(fs_info, fmt, args...) \
btrfs_printk(fs_info, KERN_DEBUG fmt, ##args)
 #define btrfs_debug_in_rcu(fs_info, fmt, args...) \
-- 
1.8.5.6


-- 
Jeff Mahoney
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: fix endless loop in balancing block groups

[Qu Wenruo]

At 09/01/2016 07:43 AM, Liu Bo wrote:

Qgroup function may overwrite the saved error 'err' with 0
in case quota is not enabled, and this ends up with a
endless loop in balance because we keep going back to balance
the same block group.


In which case?

If join_trans() fails, we won't go through qgroup fix.
And before join_trans(), they all go to out_free/out tag.
Nothing to do with qgroup fix.

And if we hit qgroup_fix_relocated_data_extents(), then 'err' is alreayd 
0, nothing we need to save.


And even for quota disabled case, qgroup_fix_relocated_data_extents() 
will just return 0. Nothing wrong at all.


Or did I miss something?

Thanks,
Qu


It really should use 'ret' instead.

Signed-off-by: Liu Bo 
---
 fs/btrfs/relocation.c | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 8a2c2a0..c0c13dc 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -4200,9 +4200,11 @@ restart:
err = PTR_ERR(trans);
goto out_free;
}
-   err = qgroup_fix_relocated_data_extents(trans, rc);
-   if (err < 0) {
-   btrfs_abort_transaction(trans, err);
+   ret = qgroup_fix_relocated_data_extents(trans, rc);
+   if (ret < 0) {
+   btrfs_abort_transaction(trans, ret);
+   if (!err)
+   err = ret;
goto out_free;
}
btrfs_commit_transaction(trans, rc->extent_root);




--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

[Jeff Mahoney]

On 8/31/16 5:48 PM, Chris Murphy wrote:
> OK it looks like with -w flag I can get a reliable indication of
> whether quota is enabled or not:
> 
> [root@f24s ~]# btrfs quota enable /mnt/0
> [root@f24s ~]# btrfs quota rescan -w /mnt/0
> quota rescan started
> [root@f24s ~]# btrfs quota disable /mnt/0
> [root@f24s ~]# btrfs quota rescan -w /mnt/0
> ERROR: quota rescan failed: Invalid argument
> 
> 
> So if you did not enable quota support, and aren't sure if it's
> enabled you can try 'btrfs quota rescan -w ' but this might
> actually be a bad idea, a rescan could take a while if you're actually
> using quotas, I have no idea because I don't use them.

It can take a while, but the code is smart enough not to get too much in
the way of other activity.  It maintains a progress marker and only does
live accounting on extents that have already been scanned.

> Perhaps someone can point out an easier way to determine whether
> quotas are enabled?

btrfs qgroup show 

If you get a message like:
ERROR: can't perform the search - No such file or directory
ERROR: can't list qgroups: No such file or directory

... it means there's no quota root and thus quotas aren't enabled.

-Jeff

-- 
Jeff Mahoney
SUSE Labs



signature.asc
Description: OpenPGP digital signature

Re: Recommendation on raid5 drive error resolution

[Gareth Pye]

ro,degraded has mounted it nicely and my rsync of the more useful data
is progressing at the speed of WiFi.

There are repeated read errors from one drive still but the rsync
hasn't bailed yet, which I think means there isn't any overlapping
errors in any of the files it has touched thus far. Am I right or is
their likely to be corrupt data in the files I've synced off?

On Wed, Aug 31, 2016 at 7:46 AM, Gareth Pye  wrote:
> Or I could just once again select the right boot device in the bios. I
> think I want some new hardware :)
>
> On Wed, Aug 31, 2016 at 7:23 AM, Gareth Pye  wrote:
>> On Wed, Aug 31, 2016 at 4:28 AM, Chris Murphy  
>> wrote:
>>> But I'd try a newer kernel before you
>>> give up on it.
>>
>>
>> Any recommendations on liveCDs that have recent kernels & btrfs tools?
>> For no apparent reason system isn't booting normally either, and I'm
>> reluctant to fix that before at least confirming the things I at least
>> partially care about have a recent backup.
>>
>> --
>> Gareth Pye - blog.cerberos.id.au
>> Level 2 MTG Judge, Melbourne, Australia
>
>
>
> --
> Gareth Pye - blog.cerberos.id.au
> Level 2 MTG Judge, Melbourne, Australia



-- 
Gareth Pye - blog.cerberos.id.au
Level 2 MTG Judge, Melbourne, Australia
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Btrfs: fix endless loop in balancing block groups

[Liu Bo]

Qgroup function may overwrite the saved error 'err' with 0
in case quota is not enabled, and this ends up with a
endless loop in balance because we keep going back to balance
the same block group.

It really should use 'ret' instead.

Signed-off-by: Liu Bo 
---
 fs/btrfs/relocation.c | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 8a2c2a0..c0c13dc 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -4200,9 +4200,11 @@ restart:
err = PTR_ERR(trans);
goto out_free;
}
-   err = qgroup_fix_relocated_data_extents(trans, rc);
-   if (err < 0) {
-   btrfs_abort_transaction(trans, err);
+   ret = qgroup_fix_relocated_data_extents(trans, rc);
+   if (ret < 0) {
+   btrfs_abort_transaction(trans, ret);
+   if (!err)
+   err = ret;
goto out_free;
}
btrfs_commit_transaction(trans, rc->extent_root);
-- 
2.5.5

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: fix endless loop in balancing block groups

[Qu Wenruo]

At 09/01/2016 08:54 AM, Liu Bo wrote:

On Thu, Sep 01, 2016 at 08:32:00AM +0800, Qu Wenruo wrote:



At 09/01/2016 07:43 AM, Liu Bo wrote:

Qgroup function may overwrite the saved error 'err' with 0
in case quota is not enabled, and this ends up with a
endless loop in balance because we keep going back to balance
the same block group.


In which case?

If join_trans() fails, we won't go through qgroup fix.
And before join_trans(), they all go to out_free/out tag.
Nothing to do with qgroup fix.


I don't think so.

It doesn't always go to out_free, in the while() loop, it'd break the
loop on any error and record it in @err, then go through
prepare_to_merge() + merge_reloc_roots() and other stuff.

Here's an example for keeping err after the above while loop(),
...
err = prepare_to_merge(rc, err);
...


OK, that's right.
Reviewed-by: Qu Wenruo 

While IMHO it's a larger problem that we didn't handle the error 
immediately after prepare_to_merge() or inside the while(1) loop, but 
postpone it to later parts.


Thanks,
Qu




Thanks,

-liubo



And if we hit qgroup_fix_relocated_data_extents(), then 'err' is alreayd 0,
nothing we need to save.

And even for quota disabled case, qgroup_fix_relocated_data_extents() will
just return 0. Nothing wrong at all.

Or did I miss something?

Thanks,
Qu


It really should use 'ret' instead.

Signed-off-by: Liu Bo 
---
 fs/btrfs/relocation.c | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 8a2c2a0..c0c13dc 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -4200,9 +4200,11 @@ restart:
err = PTR_ERR(trans);
goto out_free;
}
-   err = qgroup_fix_relocated_data_extents(trans, rc);
-   if (err < 0) {
-   btrfs_abort_transaction(trans, err);
+   ret = qgroup_fix_relocated_data_extents(trans, rc);
+   if (ret < 0) {
+   btrfs_abort_transaction(trans, ret);
+   if (!err)
+   err = ret;
goto out_free;
}
btrfs_commit_transaction(trans, rc->extent_root);










--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: remove pointless debugfs interface

[David Sterba]

On Wed, Aug 31, 2016 at 10:13:49AM -0500, Eric Sandeen wrote:
> A /sys/kernel/debug/btrfs/test file was added nearly
> two and a half years ago, but it serves no purpose;

It does. Introduced in 1bae30982bc86ab66d61ccb6e22792593b45d44d says
something about helping developers to easily export information from the
filesystem, to aid debugging. Writing the debugfs support code is not
obviously trivial, so it's idling in the source. Exporing a new value is
as easy as copy and update 3 lines of code. If you have no use for it,
fine.

> it stores and returns a value, but nothing in the btrfs
> code uses this value in any way.  There are no other btrfs
> files in this debugfs dir.
> 
> This was brought to my attention because it is world-writable;
> it is the only such file under /sys/kernel/debug, and without
> knowledge of its purpose, some users were alarmed by this.

So let's fix the permissions.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: fix one bug that process may endlessly wait for ticket in wait_reserve_ticket()

[Wang Xiaoguang]

If can_overcommit() in btrfs_calc_reclaim_metadata_size() returns true,
btrfs_async_reclaim_metadata_space() will not reclaim metadata space, just
return directly and also forget to wake up process which are waiting for
their tickets, so these processes will wait endlessly.

Fstests case generic/172 with mount option "-o compress=lzo" have revealed
this bug in my test machine. Here if we have tickets to handle, we must
handle them first.

Signed-off-by: Wang Xiaoguang 
---
 fs/btrfs/extent-tree.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 0450dc4..8c8a4d1 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4901,11 +4901,6 @@ btrfs_calc_reclaim_metadata_size(struct btrfs_root *root,
u64 expected;
u64 to_reclaim = 0;
 
-   to_reclaim = min_t(u64, num_online_cpus() * SZ_1M, SZ_16M);
-   if (can_overcommit(root, space_info, to_reclaim,
-  BTRFS_RESERVE_FLUSH_ALL))
-   return 0;
-
list_for_each_entry(ticket, _info->tickets, list)
to_reclaim += ticket->bytes;
list_for_each_entry(ticket, _info->priority_tickets, list)
@@ -4913,6 +4908,11 @@ btrfs_calc_reclaim_metadata_size(struct btrfs_root *root,
if (to_reclaim)
return to_reclaim;
 
+   to_reclaim = min_t(u64, num_online_cpus() * SZ_1M, SZ_16M);
+   if (can_overcommit(root, space_info, to_reclaim,
+  BTRFS_RESERVE_FLUSH_ALL))
+   return 0;
+
used = space_info->bytes_used + space_info->bytes_reserved +
   space_info->bytes_pinned + space_info->bytes_readonly +
   space_info->bytes_may_use;
-- 
2.9.0



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: remove pointless debugfs interface

[Eric Sandeen]

A /sys/kernel/debug/btrfs/test file was added nearly
two and a half years ago, but it serves no purpose;
it stores and returns a value, but nothing in the btrfs
code uses this value in any way.  There are no other btrfs
files in this debugfs dir.

This was brought to my attention because it is world-writable;
it is the only such file under /sys/kernel/debug, and without
knowledge of its purpose, some users were alarmed by this.

Another option would be to change the perms, but given that
there is no point to it as far as I can tell, it seems best
to simply remove it.

Signed-off-by: Eric Sandeen 
---

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 4879656..8c3ffb9 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -24,7 +24,6 @@
 #include 
 #include 
 #include 
-#include 
 
 #include "ctree.h"
 #include "disk-io.h"
@@ -728,12 +727,6 @@ int btrfs_sysfs_add_device_link(struct btrfs_fs_devices 
*fs_devices,
 /* /sys/fs/btrfs/ entry */
 static struct kset *btrfs_kset;
 
-/* /sys/kernel/debug/btrfs */
-static struct dentry *btrfs_debugfs_root_dentry;
-
-/* Debugging tunables and exported data */
-u64 btrfs_debugfs_test;
-
 /*
  * Can be called by the device discovery thread.
  * And parent can be specified for seed device
@@ -827,19 +820,6 @@ void btrfs_sysfs_feature_update(struct btrfs_fs_info 
*fs_info,
ret = sysfs_create_group(fsid_kobj, _feature_attr_group);
 }
 
-static int btrfs_init_debugfs(void)
-{
-#ifdef CONFIG_DEBUG_FS
-   btrfs_debugfs_root_dentry = debugfs_create_dir("btrfs", NULL);
-   if (!btrfs_debugfs_root_dentry)
-   return -ENOMEM;
-
-   debugfs_create_u64("test", S_IRUGO | S_IWUGO, btrfs_debugfs_root_dentry,
-   _debugfs_test);
-#endif
-   return 0;
-}
-
 int btrfs_init_sysfs(void)
 {
int ret;
@@ -848,28 +828,19 @@ int btrfs_init_sysfs(void)
if (!btrfs_kset)
return -ENOMEM;
 
-   ret = btrfs_init_debugfs();
-   if (ret)
-   goto out1;
-
init_feature_attrs();
ret = sysfs_create_group(_kset->kobj, _feature_attr_group);
-   if (ret)
-   goto out2;
+   if (ret) {
+   kset_unregister(btrfs_kset);
+   return ret;
+   }
 
return 0;
-out2:
-   debugfs_remove_recursive(btrfs_debugfs_root_dentry);
-out1:
-   kset_unregister(btrfs_kset);
-
-   return ret;
 }
 
 void btrfs_exit_sysfs(void)
 {
sysfs_remove_group(_kset->kobj, _feature_attr_group);
kset_unregister(btrfs_kset);
-   debugfs_remove_recursive(btrfs_debugfs_root_dentry);
 }
 
diff --git a/fs/btrfs/sysfs.h b/fs/btrfs/sysfs.h
index d7da1a4..45bad52 100644
--- a/fs/btrfs/sysfs.h
+++ b/fs/btrfs/sysfs.h
@@ -1,11 +1,6 @@
 #ifndef _BTRFS_SYSFS_H_
 #define _BTRFS_SYSFS_H_
 
-/*
- * Data exported through sysfs
- */
-extern u64 btrfs_debugfs_test;
-
 enum btrfs_feature_set {
FEAT_COMPAT,
FEAT_COMPAT_RO,

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: fix one bug that process may endlessly wait for ticket in wait_reserve_ticket()

[Josef Bacik]

On 08/31/2016 07:46 AM, Wang Xiaoguang wrote:

If can_overcommit() in btrfs_calc_reclaim_metadata_size() returns true,
btrfs_async_reclaim_metadata_space() will not reclaim metadata space, just
return directly and also forget to wake up process which are waiting for
their tickets, so these processes will wait endlessly.

Fstests case generic/172 with mount option "-o compress=lzo" have revealed
this bug in my test machine. Here if we have tickets to handle, we must
handle them first.

Signed-off-by: Wang Xiaoguang 


Yup good catch, you can add

Reviewed-by: Josef Bacik 

Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html