btrfs goes readonly + No space left on 4.3

Hi, while trying to write to my volume btrfs gets readonly: btrfs fi show /vmbackup/ Label: none uuid: f4afaac2-c587-4ff7-87b1-19e6a483215f Total devices 1 FS bytes used 35.56TiB devid1 size 50.93TiB used 35.72TiB path /dev/mapper/stripe0-vmbackup btrfs-progs v4.1.2

Re: Questions about FIEMAP

On Mon, Oct 12, 2015 at 04:37:55AM +, Wang, Zhiye wrote: > Hello everyone, > > After googled a bit, I got information that btrfs supports FIEMAP (as "cp" > needs it), but it's not valid for "write" operation. > > I guess we cannot write to block device directly after get block list using >

Re: [PATCH v5 9/9] btrfs: btrfs_copy_file_range() only supports reflinks

On 11/10/15 15:29, Christoph Hellwig wrote: > On Wed, Sep 30, 2015 at 01:26:53PM -0400, Anna Schumaker wrote: >> Reject copies that don't have the COPY_FR_REFLINK flag set. > > I think a reflink actually is a perfectly valid copy, and I don't buy > the duplicate arguments in earlier threads. We

Re: Questions about FIEMAP

Wang, Zhiye posted on Mon, 12 Oct 2015 04:37:55 + as excerpted: > I guess we cannot write to block device directly after get block list > using FIEMAP. This is because: > > 1. COW feature of btrfs (but this can be disabled using NOCOW) I'm a user not a dev and many of the specifics of this

Re: Issue: With an OverlayFS that has Btrfs as the upper layer, removal of directories silently fails

On Fri, Aug 21, 2015 at 9:23 PM, Neal Gompa wrote: > Hello, > > For those who have received this message twice already, I sincerely apologize. > > Yesterday I filed an issue in the CentOS bug tracker (#9297[1]) and > the Red Hat Bugzilla (#1255512[2]) about OverlayFS

Re: BTRFS with 8TB SMR drives

Hi Warren, from your dmesg I see: Oct 10 07:42:36 cloud.warrenhughes.net kernel: scsi 0:0:1:0: Direct-Access ATA ST8000AS0002-1NA AR13 PQ: 0 ANSI: 5 Oct 10 07:42:36 cloud.warrenhughes.net kernel: sd 0:0:1:0: [sdb] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB) Oct 11 23:57:56

Re: [PATCH] btrfs-progs: Add all missing close_ctree and btrfs_close_all_devices

On Mon, Oct 12, 2015 at 11:27:39AM +0800, Zhao Lei wrote: > This patch add all missing close_ctree and btrfs_close_all_devices > to several tools in btrfs progs, to avoid memory leak. With that many missing callsites, I think it's better to put it right after the command callback: btrfs.c: 245

Re: BTRFS with 8TB SMR drives

On Mon, Oct 12, 2015 at 07:43:50AM +1300, Warren Hughes wrote: > Hi guys, just added a new Seagate Archive 8TB drive to my BTRFS volume > and I'm getting a tonne of errors when balancing or scrubbing. > > A short smartctl test reports fine, running a long one now. Will also > run seatools from a

[PATCH 01/11] btrfs-progs: subvolume: use btrfs_open_dir for btrfs subvolume command

We can use btrfs_open_dir() to check whether target dir is in btrfs's mount point before open, instead of checking it in kernel space of ioctl, and return fuzzy error message. Before patch: # (/mnt/tmp is not btrfs mountpoint) # # btrfs subvolume create /mnt/tmp/123 Create subvolume

[PATCH 07/11] btrfs-progs: qgroup: use btrfs_open_dir for btrfs qgroup command

We can use btrfs_open_dir() to check whether target dir is in btrfs's mount point before open, instead of checking it in kernel space of ioctl, and return fuzzy error message. Before patch: # ./btrfs qgroup create 1/5 /mnt/tmp1 ERROR: unable to create quota group: Inappropriate ioctl for

[PATCH 09/11] btrfs-progs: use btrfs_open_dir in open_path_or_dev_mnt

Use btrfs_open_dir() in open_path_or_dev_mnt() to make the function return error when target is neither block device nor btrfs mount point. Also add "verbose" argument to let function output common error message instead of putting duplicated lines in caller. Before patch: # ./btrfs device

[PATCH 04/11] btrfs-progs: inspect: Bypass unnecessary clean function in open_error

No need to cleanup fd in open_fail case, because it is not opened. Signed-off-by: Zhao Lei --- cmds-inspect.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/cmds-inspect.c b/cmds-inspect.c index fc3db99..879fd43 100644 --- a/cmds-inspect.c +++

[PATCH 02/11] btrfs-progs: filesystem: use btrfs_open_dir for btrfs filesystem command

We can use btrfs_open_dir() to check whether target dir is in btrfs's mount point before open, instead of checking it in kernel space of ioctl, and return fuzzy error message. Before patch: # (/mnt/tmp is not btrfs mountpoint) # # btrfs filesystem df /mnt/tmp ERROR: couldn't get space

[PATCH 08/11] btrfs-progs: quota: use btrfs_open_dir for btrfs quota command

We can use btrfs_open_dir() to check whether target dir is in btrfs's mount point before open, instead of checking it in kernel space of ioctl, and return fuzzy error message. Before patch: # ./btrfs quota enable /mnt/tmp1 ERROR: quota command failed: Inappropriate ioctl for device #

[PATCH 06/11] btrfs-progs: inspect: use btrfs_open_dir for btrfs inspect command

We can use btrfs_open_dir() to check whether target dir is in btrfs's mount point before open, instead of checking it in kernel space of ioctl, and return fuzzy error message. Before patch: # ./btrfs inspect-internal rootid /mnt/tmp1 ERROR: Failed to lookup root id - Inappropriate ioctl for

[PATCH 10/11] btrfs-progs: replace: use btrfs_open_dir for btrfs replace command

We can use btrfs_open_dir() to check whether target dir is in btrfs's mount point before open, instead of checking it in kernel space of ioctl, and return fuzzy error message. Before patch: # ./btrfs replace cancel /mnt/tmp1 ERROR: ioctl(DEV_REPLACE_CANCEL) failed on "/mnt/tmp1":

[PATCH 03/11] btrfs-progs: balance: use btrfs_open_dir for btrfs balance command

We can use btrfs_open_dir() to check whether target dir is in btrfs's mount point before open, instead of checking it in kernel space of ioctl, and return fuzzy error message. Before patch: # btrfs balance start /mnt/tmp ERROR: error during balancing '/mnt/tmp' - Inappropriate ioctl for

[PATCH 11/11] btrfs-progs: fragments: use btrfs_open_dir for btrfs-fragments command

We can use btrfs_open_dir() to check whether target dir is in btrfs's mount point before open, instead of checking it in deeper code, and return fuzzy error message. Before patch: ./btrfs-fragments -o 123 /mnt/tmp1 ERROR: can't perform the search After patch: # ./btrfs-fragments -o 123

Re: Questions about FIEMAP

On Mon, Oct 12, 2015 at 04:37:55AM +, Wang, Zhiye wrote: > After googled a bit, I got information that btrfs supports FIEMAP (as > "cp" needs it), but it's not valid for "write" operation. The FIEMAP output is informative, there's no guarantee that the extent information does not change

Re: Questions about FIEMAP

On 10/11/15 11:37 PM, Wang, Zhiye wrote: > Hello everyone, > > After googled a bit, I got information that btrfs supports FIEMAP (as "cp" > needs it), but it's not valid for "write" operation. cp should not be using fiemap any more. It was for a while, until they realized that copying based

[PATCH 00/11] btrfs-progs: Use btrfs_open_dir to avoid show error of ioctl or tree search

Use btrfs_open_dir() instead of open_file_or_dir(), to show error before real action(in ioctl or tree search), to make the error message exact and unified. It can also make code simple: 85 insertions(+), 185 deletions(-) Also include some small bug fix. Before patch: # grep open_file_or_dir

Re: filesystem goes ro trying to balance. "cpu stuck"

On Mon, Oct 12, 2015 at 12:33 AM, Duncan <1i5t5.dun...@cox.net> wrote: > Donald Pearson posted on Sun, 11 Oct 2015 11:46:14 -0500 as excerpted: > >> Kernel 4.2.2-1.el7.elrepo btrfs-progs v4.2.1 >> >> I'm attempting to convert a filesystem from raid6 to raid10. I didn't >> have any functional

Re: Add stripes filter

On Mon, Sep 28, 2015 at 05:57:05PM +, Gabríel Arthúr Pétursson wrote: > The attached patches to linux and btrfs-progs add support for filtering > based on the number of strips in a block when balancing. FYI, I'm going to make the fixups myself as they're mostly cosmetic and prepare this patch

Re: [PATCH v5 9/9] btrfs: btrfs_copy_file_range() only supports reflinks

On Mon, Oct 12, 2015 at 11:23:05AM +0100, P??draig Brady wrote: > You're right that if the user doesn't notice, then there is no > point exposing this. However I think the user does notice as > there is a difference in the end state of the copy. I.E. generally > if there is a different end state

[PATCH for 4.3] btrfs: check unsupported filters in balance arguments

We don't verify that all the balance filter arguments supplemented by the flags are actually known to the kernel. Thus we let it silently pass and do nothing. At the moment this means only the 'limit' filter, but we're going to add a few more soon so it's better to have that fixed. Also in older

Re: BTRFS with 8TB SMR drives

Yes, correct its drive managed SMR. I have been following this bug: https://bugzilla.kernel.org/show_bug.cgi?id=93581 for a while As a test I compiled/installed 4.3.0-rc4 as it looks like they reverted some kernel patches that (negatively) affect SMR. I ran a complete balance overnight and not

Re: BTRFS with 8TB SMR drives

I get a lot of these from both sdb and sdc Oct 11 23:00:03 cloud.warrenhughes.net kernel: sd 0:0:1:0: [sdb] UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 Oct 11 23:00:03 cloud.warrenhughes.net kernel: sd 0:0:1:0: [sdb] Sense Key : 0x3 [current] Oct 11 23:00:03 cloud.warrenhughes.net

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On Sun, Oct 11, 2015 at 07:22:03AM -0700, Christoph Hellwig wrote: > On Wed, Sep 30, 2015 at 01:26:52PM -0400, Anna Schumaker wrote: > > This allows us to have an in-kernel copy mechanism that avoids frequent > > switches between kernel and user space. This is especially useful so > > NFSD can

Re: [PATCH] btrfs: fix resending received snapshot with parent

On Friday, October 9, 2015 4:24:10 PM EDT, Filipe Manana wrote: On Wed, Sep 30, 2015 at 8:23 PM, Robin Ruede wrote: This fixes a regression introduced by 37b8d27d between v4.1 and v4.2. When a snapshot is received, its received_uuid is set to the original uuid of the

Re: [PATCH v5 9/9] btrfs: btrfs_copy_file_range() only supports reflinks

On Mon, Oct 12, 2015 at 07:34:44AM -0700, Christoph Hellwig wrote: > On Mon, Oct 12, 2015 at 11:23:05AM +0100, P??draig Brady wrote: > > You're right that if the user doesn't notice, then there is no > > point exposing this. However I think the user does notice as > > there is a difference in the

Re: BTRFS with 8TB SMR drives

Sounds to me like this: https://bugzilla.kernel.org/show_bug.cgi?id=93581 On Mon, Oct 12, 2015 at 11:37 AM, Chris Murphy wrote: > I get a lot of these from both sdb and sdc > > Oct 11 23:00:03 cloud.warrenhughes.net kernel: sd 0:0:1:0: [sdb] > UNKNOWN(0x2003) Result:

Re: BTRFS with 8TB SMR drives

yes indeed - referenced it in my update here https://mail-archive.com/linux-btrfs@vger.kernel.org/msg47380.html On 13 October 2015 at 13:04, Justin Maggard wrote: > Sounds to me like this: https://bugzilla.kernel.org/show_bug.cgi?id=93581 > > On Mon, Oct 12, 2015 at 11:37

[PATCH v3 00/21] Rework btrfs qgroup reserved space framework

In previous rework of qgroup, we succeeded in fixing qgroup accounting part, making the rfer/excl numbers accurate. But that's just part of qgroup work, another part of qgroup still has quite a lot problem, that's qgroup reserve space part which will lead to EQUOT even we are far from the limit.

[PATCH v3 12/21] btrfs: extent-tree: Add new version of btrfs_delalloc_reserve/release_space

Add new version of btrfs_delalloc_reserve_space() and btrfs_delalloc_release_space() functions, which supports accurate qgroup reserve. Signed-off-by: Qu Wenruo --- v2: Add new function btrfs_delalloc_release_space() to handle error case. v3: None ---

[PATCH v3 20/21] btrfs: qgroup: Avoid calling btrfs_free_reserved_data_space in clear_bit_hook

In clear_bit_hook, qgroup reserved data is already handled quite well, either released by finish_ordered_io or invalidatepage. So calling btrfs_qgroup_free_data() here is completely meaningless, and since btrfs_qgroup_free_data() will lock io_tree, so it can't be called with io_tree lock hold.

[PATCH v3 13/21] btrfs: extent-tree: Switch to new delalloc space reserve and release

Use new __btrfs_delalloc_reserve_space() and __btrfs_delalloc_release_space() to reserve and release space for delalloc. Signed-off-by: Qu Wenruo --- v2: Also use __btrfs_delalloc_release_space() function. v3: None --- fs/btrfs/file.c | 5 +++--

[PATCH v3 02/21] btrfs: extent_io: Introduce new function set_record_extent_bits

Introduce new function set_record_extent_bits(), which will not only set given bits, but also record how many bytes are changed, and detailed range info. This is quite important for later qgroup reserve framework. The number of bytes will be used to do qgroup reserve, and detailed range info will

[PATCH v3 16/21] btrfs: Add handler for invalidate page

For btrfs_invalidatepage() and its variant evict_inode_truncate_page(), there will be pages don't reach disk. In that case, their reserved space won't be release nor freed by finish_ordered_io() nor delayed_ref handler. So we must free their qgroup reserved space, or we will leaking reserved

[PATCH v3 15/21] btrfs: qgroup: Add handler for NOCOW and inline

For NOCOW and inline case, there will be no delayed_ref created for them, so we should free their reserved data space at proper time(finish_ordered_io for NOCOW and cow_file_inline for inline). Signed-off-by: Qu Wenruo --- v2: Newly introduced v3: None ---

[PATCH v3 08/21] btrfs: qgroup: Introduce new functions to reserve/free metadata

Introduce new functions btrfs_qgroup_reserve/free_meta() to reserve/free metadata reserved space. Signed-off-by: Qu Wenruo --- v2: None v3: None --- fs/btrfs/ctree.h | 3 +++ fs/btrfs/disk-io.c | 1 + fs/btrfs/qgroup.c | 40

[PATCH v3 18/21] btrfs: fallocate: Add support to accurate qgroup reserve

Now fallocate will do accurate qgroup reserve space check, unlike old method, which will always reserve the whole length of the range. With this patch, fallocate will: 1) Iterate the desired range and mark in data rsv map Only range which is going to be allocated will be recorded in data

[PATCH v3 21/21] btrfs: qgroup: Check if qgroup reserved space leaked

Add check at btrfs_destroy_inode() time to detect qgroup reserved space leak. Signed-off-by: Qu Wenruo --- v3: Separate from old btrfs_qgroup_free_data_rsv_map(). --- fs/btrfs/inode.c | 1 + fs/btrfs/qgroup.c | 32 fs/btrfs/qgroup.h

[PATCH v3 17/21] btrfs: qgroup: Add new trace point for qgroup data reserve

Now each qgroup reserve for data will has its ftrace event for better debugging. Signed-off-by: Qu Wenruo --- v2: Newly introduced v3: None --- fs/btrfs/qgroup.c| 11 - fs/btrfs/qgroup.h| 8 +++ include/trace/events/btrfs.h | 113

[PATCH v3 09/21] btrfs: qgroup: Use new metadata reservation.

As we have the new metadata reservation functions, use them to replace the old btrfs_qgroup_reserve() call for metadata. Signed-off-by: Qu Wenruo --- v2: None v3: None --- fs/btrfs/extent-tree.c | 14 ++ fs/btrfs/transaction.c | 34

[PATCH v3 11/21] btrfs: extent-tree: Switch to new check_data_free_space and free_reserved_data_space

Use new reserve/free for buffered write and inode cache. For buffered write case, as nodatacow write won't increase quota account, so unlike old behavior which does reserve before check nocow, now we check nocow first and then only reserve data if we can't do nocow write. Signed-off-by: Qu

[PATCH v3 14/21] btrfs: qgroup: Cleanup old inaccurate facilities

Cleanup the old facilities which use old btrfs_qgroup_reserve() function call, replace them with the newer version, and remove the "__" prefix in them. Also, make btrfs_qgroup_reserve/free() functions private, as they are now only used inside qgroup codes. Now, the whole btrfs qgroup is swithed

[PATCH v3 10/21] btrfs: extent-tree: Add new version of btrfs_check_data_free_space and btrfs_free_reserved_data_space.

Add new functions __btrfs_check_data_free_space() and __btrfs_free_reserved_data_space() to work with new accurate qgroup reserved space framework. The new function will replace old btrfs_check_data_free_space() and btrfs_free_reserved_data_space() respectively, but until all the change is done,

[PATCH v3 03/21] btrfs: extent_io: Introduce new function clear_record_extent_bits()

Introduce new function clear_record_extent_bits(), which will clear bits for given range and record the details about which ranges are cleared and how many bytes in total it changes. This provides the basis for later qgroup reserve codes. Signed-off-by: Qu Wenruo ---

[PATCH v3 06/21] btrfs: delayed_ref: Add new function to record reserved space into delayed ref

Add new function btrfs_add_delayed_qgroup_reserve() function to record how much space is reserved for that extent. As btrfs only accounts qgroup at run_delayed_refs() time, so newly allocated extent should keep the reserved space until then. So add needed function with related members to do it.

[PATCH v3 01/21] btrfs: extent_io: Introduce needed structure for recoding set/clear bits

Add a new structure, extent_change_set, to record how many bytes are changed in one set/clear_extent_bits() operation, with detailed changed ranges info. This provides the needed facilities for later qgroup reserve framework. Signed-off-by: Qu Wenruo --- v3: Newly

[PATCH v3 07/21] btrfs: delayed_ref: release and free qgroup reserved at proper timing

Qgroup reserved space needs to be released from inode dirty map and get freed at different timing: 1) Release when the metadata is written into tree After corresponding metadata is written into tree, any newer write will be COWed(don't include NOCOW case yet). So we must release its range from

[PATCH v3 04/21] btrfs: qgroup: Introduce btrfs_qgroup_reserve_data function

Introduce a new function, btrfs_qgroup_reserve_data(), which will use io_tree to accurate qgroup reserve, to avoid reserved space leaking. Signed-off-by: Qu Wenruo --- v2: Add needed parameter for later trace functions v3: Use io_tree facilities instead of

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On Mon, Oct 12, 2015 at 7:17 PM, Darrick J. Wong wrote: > On Sun, Oct 11, 2015 at 07:22:03AM -0700, Christoph Hellwig wrote: >> On Wed, Sep 30, 2015 at 01:26:52PM -0400, Anna Schumaker wrote: >> > This allows us to have an in-kernel copy mechanism that avoids frequent >>