Re: About hung task on generic/041

2018-07-13 Thread Lu Fengqi
On Fri, Jul 13, 2018 at 11:33:39AM +0100, Filipe Manana wrote: >On Fri, Jul 13, 2018 at 9:44 AM, Lu Fengqi wrote: >> On Thu, Jul 12, 2018 at 08:33:59PM +0800, Lu Fengqi wrote: >>>On Thu, Jul 12, 2018 at 11:40:54AM +0100, Filipe Manana wrote: On Wed, Jul 11, 2018 at 10:02 AM, Lu Fengqi

Re: [PATCH 2/4] btrfs: add support for 3-copy replication (raid1c3)

2018-07-13 Thread Goffredo Baroncelli
As general comment, good to hear that something is moving around raid5/6 + write hole and multiple mirroring. However I am guessing if this is time to simplify the RAID code. There are a lot of "if" which could be avoided using the values stored in the array "btrfs_raid_array[]". Below some

[PATCH 4/4] btrfs: add incompatibility bit for extended raid features

2018-07-13 Thread David Sterba
This bit will cover all newly added features of RAID: - 3 copy replication - 4 copy replication - configurable stripe length - triple parity - raid56 log Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 1 + fs/btrfs/sysfs.c | 2 ++ fs/btrfs/volumes.c | 9

[PATCH 2/4] btrfs: add support for 3-copy replication (raid1c3)

2018-07-13 Thread David Sterba
Add new block group profile to store 3 copies in a simliar way that current RAID1 does. The profile name is temporary and may change in the future. Signed-off-by: David Sterba --- fs/btrfs/extent-tree.c | 6 + fs/btrfs/relocation.c | 1 + fs/btrfs/scrub.c

[PATCH 3/4] btrfs: add support for 4-copy replication (raid1c4)

2018-07-13 Thread David Sterba
Add new block group profile to store 4 copies in a simliar way that current RAID1 does. The profile name is temporary and may change in the future. Signed-off-by: David Sterba --- fs/btrfs/extent-tree.c | 3 +++ fs/btrfs/scrub.c| 3 ++- fs/btrfs/super.c

[PATCH 1/4] btrfs: refactor block group replication factor calculation to a helper

2018-07-13 Thread David Sterba
There are many places that open code the duplicity factor of the block group profiles, create a common helper. This can be easily extended for more copies. Signed-off-by: David Sterba --- fs/btrfs/extent-tree.c | 36 fs/btrfs/super.c | 11 +++

[PATCH] btrfs-progs: add support for raid1c3 and raid1c4

2018-07-13 Thread David Sterba
$ ./mkfs.btrfs -m raid1c4 -d raid1c3 /dev/sd[abcd] Label: (null) UUID: f1f988ab-6750-4bc2-957b-98a4ebe98631 Node size: 16384 Sector size:4096 Filesystem size:8.00GiB Block group profiles: Data: RAID1C3 273.06MiB Metadata:

[PATCH 0/4] 3- and 4- copy RAID1

2018-07-13 Thread David Sterba
Hi, I have some goodies that go into the RAID56 problem, although not implementing all the remaining features, it can be useful independently. This time my hackweek project https://hackweek.suse.com/17/projects/do-something-about-btrfs-and-raid56 aimed to implement the fix for the write hole

Re: [PATCH v3] btrfs: use customized batch size for total_bytes_pinned

2018-07-13 Thread David Sterba
On Fri, Jul 13, 2018 at 04:50:42PM +0800, Ethan Lien wrote: > In commit b150a4f10d878 ("Btrfs: use a percpu to keep track of possibly > pinned bytes") we use total_bytes_pinned to track how many bytes we are > going to free in this transaction. When we are close to ENOSPC, we check it > and know

Re: [PATCH] btrfs: fix use-after-free of cmp workspace pages

2018-07-13 Thread David Sterba
On Fri, Jul 13, 2018 at 11:07:20PM +0900, Naohiro Aota wrote: > btrfs_cmp_data_free() puts cmp's src_pages and dst_pages, but leaves > their page address intact. Now, if you hit "goto again" in > btrfs_extent_same_range() and hit some error in > btrfs_cmp_data_prepare(), you'll try to unlock/put

Re: [PATCH] btrfs: Introduce compile time structure size check

2018-07-13 Thread David Sterba
On Fri, Jul 13, 2018 at 10:57:07PM +0800, Qu Wenruo wrote: > > > On 2018年07月13日 22:46, David Sterba wrote: > > On Fri, Jul 13, 2018 at 10:36:42PM +0800, Qu Wenruo wrote: > >> > >> > >> On 2018年07月13日 22:34, David Sterba wrote: > >>> On Thu, Jul 12, 2018 at 02:19:07PM +0800, Qu Wenruo wrote: >

Re: [PATCH v4 1/2] btrfs: make fs_devices to be a local variable

2018-07-13 Thread David Sterba
On Thu, Jul 12, 2018 at 02:23:15PM +0800, Gu Jinxiang wrote: > fs_devices is always passed to btrfs_scan_one_device which > overrides it. And in the call stack below fs_devices is passed to > btrfs_scan_one_device from btrfs_mount_root. > And in btrfs_mount_root the output fs_devices of this call

Re: [PATCH] btrfs: Introduce compile time structure size check

2018-07-13 Thread Qu Wenruo
On 2018年07月13日 22:46, David Sterba wrote: > On Fri, Jul 13, 2018 at 10:36:42PM +0800, Qu Wenruo wrote: >> >> >> On 2018年07月13日 22:34, David Sterba wrote: >>> On Thu, Jul 12, 2018 at 02:19:07PM +0800, Qu Wenruo wrote: Introduce a new macro based compile time check for ioctl structures.

Re: [PATCH] btrfs: Introduce compile time structure size check

2018-07-13 Thread David Sterba
On Fri, Jul 13, 2018 at 10:36:42PM +0800, Qu Wenruo wrote: > > > On 2018年07月13日 22:34, David Sterba wrote: > > On Thu, Jul 12, 2018 at 02:19:07PM +0800, Qu Wenruo wrote: > >> Introduce a new macro based compile time check for ioctl structures. > >> > >> The new macro is BTRFS_ASSERT_SIZE(),

Re: [PATCH] btrfs: Introduce compile time structure size check

2018-07-13 Thread Qu Wenruo
On 2018年07月13日 22:34, David Sterba wrote: > On Thu, Jul 12, 2018 at 02:19:07PM +0800, Qu Wenruo wrote: >> Introduce a new macro based compile time check for ioctl structures. >> >> The new macro is BTRFS_ASSERT_SIZE(), which is mostly copied from >> VMMDEV_ASSERT_SIZE(). > > The macro should be

Re: [PATCH] btrfs: Introduce compile time structure size check

2018-07-13 Thread David Sterba
On Thu, Jul 12, 2018 at 02:56:55PM +0800, Qu Wenruo wrote: > > > On 2018年07月12日 14:27, Nikolay Borisov wrote: > > > > > > On 12.07.2018 09:19, Qu Wenruo wrote: > >> Introduce a new macro based compile time check for ioctl structures. > >> > >> The new macro is BTRFS_ASSERT_SIZE(), which is

Re: [PATCH] btrfs: Introduce compile time structure size check

2018-07-13 Thread David Sterba
On Thu, Jul 12, 2018 at 02:19:07PM +0800, Qu Wenruo wrote: > Introduce a new macro based compile time check for ioctl structures. > > The new macro is BTRFS_ASSERT_SIZE(), which is mostly copied from > VMMDEV_ASSERT_SIZE(). The macro should be generic, there's nothing specific to btrfs. There's

[PATCH] btrfs: fix use-after-free of cmp workspace pages

2018-07-13 Thread Naohiro Aota
btrfs_cmp_data_free() puts cmp's src_pages and dst_pages, but leaves their page address intact. Now, if you hit "goto again" in btrfs_extent_same_range() and hit some error in btrfs_cmp_data_prepare(), you'll try to unlock/put already put pages. This is simple fix to reset the address to avoid

Re: [PULL] volume and uuid_mutex cleanups

2018-07-13 Thread David Sterba
On Wed, Jul 11, 2018 at 05:22:06PM +0800, Anand Jain wrote: > > Hi David, > > Here I have put together a set of volume related patches which were > sent to the ML as independent patches earlier. These have been > reviewed and tested. Please pull. > > g...@github.com:asj/btrfs-devel.git

Re: [PATCH 2/7] btrfs: extend critical section when scanning a new device

2018-07-13 Thread David Sterba
On Wed, Jul 04, 2018 at 04:18:09PM +0800, Anand Jain wrote: > > > On 06/21/2018 01:51 AM, David Sterba wrote: > > The stale device list removal needs to be protected by device_list_mutex > > too as this could delete from the list and could race with another list > > modification and cause crash.

Re: [PATCH 1/7] btrfs: restore uuid_mutex in btrfs_open_devices

2018-07-13 Thread David Sterba
On Wed, Jul 04, 2018 at 04:09:50PM +0800, Anand Jain wrote: > On 06/21/2018 01:51 AM, David Sterba wrote: > > Commit 542c5908abfe84f7b4c1 ("btrfs: replace uuid_mutex by > > device_list_mutex in btrfs_open_devices") switched to device_list_mutex > > as we need that for the device list traversal,

Re: [PATCH 3/3] btrfs: add helper function check device delete able

2018-07-13 Thread Nikolay Borisov
On 13.07.2018 14:27, Anand Jain wrote: > > > On 07/12/2018 03:43 PM, Nikolay Borisov wrote: >> >> >> On 10.07.2018 21:22, Anand Jain wrote: >>> Move the section of the code which performs the check if the device is >>> indelible, move that into a helper function. >>> >>> Signed-off-by: Anand

Re: [PATCH 3/3] btrfs: add helper function check device delete able

2018-07-13 Thread Anand Jain
On 07/12/2018 03:43 PM, Nikolay Borisov wrote: On 10.07.2018 21:22, Anand Jain wrote: Move the section of the code which performs the check if the device is indelible, move that into a helper function. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 49

Re: [PATCH 2/3] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-07-13 Thread Nikolay Borisov
On 13.07.2018 14:17, Anand Jain wrote: > > > On 07/12/2018 03:31 PM, Nikolay Borisov wrote: >> >> >> On 10.07.2018 21:22, Anand Jain wrote: >>> When the replace is running the fs_devices::num_devices also includes >>> the replace device, however in some operations like device delete and >>>

Re: [PATCH 2/3] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-07-13 Thread Anand Jain
On 07/12/2018 03:31 PM, Nikolay Borisov wrote: On 10.07.2018 21:22, Anand Jain wrote: When the replace is running the fs_devices::num_devices also includes the replace device, however in some operations like device delete and balance it needs the actual num_devices without the repalce

Re: [PATCH 1/3] btrfs: warn for num_devices below 0

2018-07-13 Thread Anand Jain
On 07/12/2018 03:13 PM, Nikolay Borisov wrote: On 10.07.2018 21:22, Anand Jain wrote: In preparation to de-duplicate a section of code where we deduce the num_devices, use warn instead of bug. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+), 1

[lufq.f...@cn.fujitsu.com: Re: About hung task on generic/041]

2018-07-13 Thread Lu Fengqi
Hi, Sorry for the duplicate mail. The previous one seems to be due to the attachment is too large to be sent to the mailing list. On Thu, Jul 12, 2018 at 08:33:59PM +0800, Lu Fengqi wrote: >On Thu, Jul 12, 2018 at 11:40:54AM +0100, Filipe Manana wrote: >>On Wed, Jul 11, 2018 at 10:02 AM, Lu

[PATCH v3] btrfs: use customized batch size for total_bytes_pinned

2018-07-13 Thread Ethan Lien
In commit b150a4f10d878 ("Btrfs: use a percpu to keep track of possibly pinned bytes") we use total_bytes_pinned to track how many bytes we are going to free in this transaction. When we are close to ENOSPC, we check it and know if we can make the allocation by commit the current transaction. For

Re: [DOC] BTRFS Volume operations, Device Lists and Locks all in one page

2018-07-13 Thread Qu Wenruo
On 2018年07月13日 15:24, Anand Jain wrote: > > > On 07/13/2018 01:39 PM, Qu Wenruo wrote: >> >> >> On 2018年07月13日 13:32, Anand Jain wrote: >>> >>> >>> >>> >>> > But if you are planning to >     record and start at transaction [14] then its an overkill because >     transaction [19 and

Re: Transaction aborted (error -28) btrfs_run_delayed_refs*0x163/0x190

2018-07-13 Thread Pete
On 07/12/2018 11:12 PM, Pete wrote: > Nothing seen, though I recently had the disks go read-only. I'll wait > and see what happens. OK, it went read only - here is the relevent section of the logs. BTRFS: block rsv returned -28 Jul 12 06:10:09 phoenix kernel: [30637.427155] WARNING: CPU: 3

Re: [DOC] BTRFS Volume operations, Device Lists and Locks all in one page

2018-07-13 Thread Anand Jain
On 07/13/2018 01:39 PM, Qu Wenruo wrote: On 2018年07月13日 13:32, Anand Jain wrote: But if you are planning to    record and start at transaction [14] then its an overkill because    transaction [19 and [20] are already in the disk. Yes, I'm doing it overkilled.  Ah. Ok. But