Re: [PATCH V7] Btrfs: enhance raid1/10 balance heuristic

2018-11-13 Thread Goffredo Baroncelli
ripes = map->num_stripes; > > - preferred_mirror = first + current->pid % num_stripes; > + preferred_mirror = first + guess_optimal(map, num_stripes, > + current->pid % num_stripes); > > if (dev_replace_is_ongoing && > fs_info->dev_replace.cont_reading_from_srcdev_mode == > -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5

[PATCH 8/9] btrfs: Make more generic the code for RAID 6 rebuilding

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli The original code which handles the recovery of a RAID 6 disks array assumes that all reads are multiple of 1 << GRUB_DISK_SECTOR_BITS and it assumes that all the I/O is done via the struct grub_diskfilter_segment. This is not true for the btrfs code. In order to

[PATCH 9/9] btrfs: Add RAID 6 recovery for a btrfs filesystem.

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Add the RAID 6 recovery, in order to use a RAID 6 filesystem even if some disks (up to two) are missing. This code use the md RAID 6 code already present in grub. Signed-off-by: Goffredo Baroncelli --- grub-core/fs/btrfs.c | 63

[PATCH 6/9] btrfs: Refactor the code that read from disk

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Move the code in charge to read the data from disk into a separate function. This helps to separate the error handling logic (which depends on the different raid profiles) from the read from disk logic. Refactoring this code increases the general readability too

[PATCH 3/9] btrfs: Move the error logging from find_device() to its caller.

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli The caller knows better if this error is fatal or not, i.e. another disk is available or not. This is a preparatory patch. Signed-off-by: Goffredo Baroncelli Reviewed-by: Daniel Kiper --- grub-core/fs/btrfs.c | 10 -- 1 file changed, 4 insertions(+), 6

[PATCH 5/9] btrfs: Move logging code in grub_btrfs_read_logical()

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli A portion of the logging code is moved outside of internal for(;;). The part that is left inside is the one which depends on the internal for(;;) index. This is a preparatory patch. The next one will refactor the code inside the for(;;) into an another function

[PATCH 4/9] btrfs: Avoid a rescan for a device which was already not found.

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Change the behavior of find_device(): before the patch, a read of a missed device may trigger a rescan. However, it is never recorded that a device is missed, so each single read of a missed device may triggers a rescan. It is the caller who decides if a rescan

[PATCH 7/9] btrfs: Add support for recovery for a RAID 5 btrfs profiles.

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Add support for recovery for a RAID 5 btrfs profile. In addition it is added some code as preparatory work for RAID 6 recovery code. Signed-off-by: Goffredo Baroncelli --- grub-core/fs/btrfs.c | 160 +-- 1 file changed, 155

[PATCH 2/9] btrfs: Add helper to check the btrfs header.

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli This helper is used in a few places to help the debugging. As conservative approach the error is only logged. This does not impact the error handling. Signed-off-by: Goffredo Baroncelli Reviewed-by: Daniel Kiper --- grub-core/fs/btrfs.c | 24

[PATCH V8] Add support for BTRFS raid5/6 to GRUB

2018-09-27 Thread Goffredo Baroncelli
ved an update to the comment BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5

[PATCH 1/9] btrfs: Add support for reading a filesystem with a RAID 5 or RAID 6 profile.

2018-09-27 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Signed-off-by: Goffredo Baroncelli --- grub-core/fs/btrfs.c | 74 1 file changed, 74 insertions(+) diff --git a/grub-core/fs/btrfs.c b/grub-core/fs/btrfs.c index be195448d..9bc6d399d 100644 --- a/grub-core/fs/btrfs.c +++ b

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-19 Thread Goffredo Baroncelli
On 18/09/2018 19.15, Goffredo Baroncelli wrote: >> b. The bootloader code, would have to have sophisticated enough Btrfs >> knowledge to know if the grubenv has been reflinked or snapshot, >> because even if +C, it may not be valid to overwrite, and COW must >> still ha

[PATCH 8/9] btrfs: Make more generic the code for RAID 6 rebuilding

2018-09-19 Thread Goffredo Baroncelli
From: Goffredo Baroncelli The original code which handles the recovery of a RAID 6 disks array assumes that all reads are multiple of 1 << GRUB_DISK_SECTOR_BITS and it assumes that all the I/O is done via the struct grub_diskfilter_segment. This is not true for the btrfs code. In order to

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Goffredo Baroncelli
On 18/09/2018 20.52, Chris Murphy wrote: > On Tue, Sep 18, 2018 at 11:15 AM, Goffredo Baroncelli > wrote: >> On 18/09/2018 06.21, Chris Murphy wrote: >>> b. The bootloader code, would have to have sophisticated enough Btrfs >>> knowledge to know if the grubenv

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Goffredo Baroncelli
BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-20 Thread Goffredo Baroncelli
On 07/20/2018 07:17 AM, Andrei Borzenkov wrote: > 18.07.2018 22:42, Goffredo Baroncelli пишет: >> On 07/18/2018 09:20 AM, Duncan wrote: >>> Goffredo Baroncelli posted on Wed, 18 Jul 2018 07:59:52 +0200 as >>> excerpted: >>> >>>> On 07/17/2018 1

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-20 Thread Goffredo Baroncelli
On 07/19/2018 09:10 PM, Austin S. Hemmelgarn wrote: > On 2018-07-19 13:29, Goffredo Baroncelli wrote: [...] >> >> So until now you are repeating what I told: the only useful raid profile are >> - striping >> - mirroring >> - striping+paring (even limiting the numbe

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-18 Thread Goffredo Baroncelli
On 07/18/2018 09:20 AM, Duncan wrote: > Goffredo Baroncelli posted on Wed, 18 Jul 2018 07:59:52 +0200 as > excerpted: > >> On 07/17/2018 11:12 PM, Duncan wrote: >>> Goffredo Baroncelli posted on Mon, 16 Jul 2018 20:29:46 +0200 as >>> excerpted: >>>

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-18 Thread Goffredo Baroncelli
On 07/17/2018 11:12 PM, Duncan wrote: > Goffredo Baroncelli posted on Mon, 16 Jul 2018 20:29:46 +0200 as > excerpted: > >> On 07/15/2018 04:37 PM, waxhead wrote: > >> Striping and mirroring/pairing are orthogonal properties; mirror and >> parity are mutually exc

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-16 Thread Goffredo Baroncelli
ROR3 > RAID10  / R1.Sm.P0 / STRIPE.MIRROR1 > RAID5   / R1.Sm.P1 / STRIPE.PARITY1 > RAID6   / R1.Sm.P2 / STRIPE.PARITY2 > > And i think this is much more readable, but others may disagree. And as a > side note... from a (hobby) coders perspective this is probably simpler to > parse as wel

Re: [PATCH 2/4] btrfs: add support for 3-copy replication (raid1c3)

2018-07-13 Thread Goffredo Baroncelli
SNAP_CREATE _IOW(BTRFS_IOCTL_MAGIC, 1, \ > diff --git a/include/uapi/linux/btrfs_tree.h b/include/uapi/linux/btrfs_tree.h > index aff1356c2bb8..fa75b63dd928 100644 > --- a/include/uapi/linux/btrfs_tree.h > +++ b/include/uapi/linux/btrfs_tree.h > @@ -836,6 +836,7 @@ struct btrfs_dev_replace_item { > #define BTRFS_BLOCK_GROUP_RAID10

Re: Major design flaw with BTRFS Raid, temporary device drop will corrupt nodatacow files

2018-06-28 Thread Goffredo Baroncelli
th cow enabled of course !) I think that this is a good example of why a HW Raid controller battery backed could be better than a SW raid. Of course the likelihood of a lot of problems could be reduced using a power supply. BR -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint

Re: unsolvable technical issues?

2018-06-24 Thread Goffredo Baroncelli
specifically a > corporation making it a/the main money product, whereas btrfs was always > something the btrfs devs used at their employers (oracle, facebook), who > have other things as their main product. As such, stratis is much more > likely to prioritize things like raid status

Re: [PATCH 00/14 RFC] Btrfs: Add journal for raid5/6 writes

2018-05-03 Thread Goffredo Baroncelli
dary. For a filesystem this is complicate to grant, however for a journal it would be more simple to do; BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe l

Re: RAID56 - 6 parity raid

2018-05-03 Thread Goffredo Baroncelli
omo-info.html > > His patch was about a generic library to do RAID6, it wasn't directly > for btrfs. > > To actually use that for btrfs someone would have to actually port btrfs > to that library. In the past Andrea ported this library to btrfs too https://lwn.net/Articles/588

Re: RAID56 - 6 parity raid

2018-05-03 Thread Goffredo Baroncelli
start a scrub once per week, the fact that grub requires 1 hr, or 1 day doesn't impact the reliability, because in any case you have 1 week of un-scrubbed data. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
r is it calculated > on the data+checksum ?! > Data is checksummed per block. The parity is invisible to the checksum. The parity are allocated in an "address space" parallel to the data "address space" exposed by the BG. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffred

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
On 05/02/2018 09:29 PM, Austin S. Hemmelgarn wrote: > On 2018-05-02 13:25, Goffredo Baroncelli wrote: >> On 05/02/2018 06:55 PM, waxhead wrote: >>>> >>>> So again, which problem would solve having the parity checksummed ? On the >>>> best

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
On 05/02/2018 08:17 PM, waxhead wrote: > Goffredo Baroncelli wrote: >> On 05/02/2018 06:55 PM, waxhead wrote: >>>> >>>> So again, which problem would solve having the parity checksummed ? On the >>>> best of my knowledge nothing. In any case the

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
igher than the gain, and the likelihood the gain is very lower compared to the likelihood (=100% or always) of the cost. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the lin

Re: RAID56 - 6 parity raid

2018-05-02 Thread Goffredo Baroncelli
de complexity and the write amplification, because every time a part of the stripe is touched not only the parity has to be updated, but also the checksum has too.. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E

Re: [RFC] Add support for BTRFS raid5/6 to GRUB

2018-04-23 Thread Goffredo Baroncelli
On 04/23/2018 01:50 PM, Daniel Kiper wrote: > On Tue, Apr 17, 2018 at 09:57:40PM +0200, Goffredo Baroncelli wrote: >> Hi All, >> >> Below you can find a patch to add support for accessing files from >> grub in a RAID5/6 btrfs filesystem. This is a RFC because i

[RFC] Add support for BTRFS raid5/6 to GRUB

2018-04-17 Thread Goffredo Baroncelli
. Comments are welcome. BR G.Baroncelli --- commit 8c80a1b7c913faf50f95c5c76b4666ed17685666 Author: Goffredo Baroncelli <kreij...@inwind.it> Date: Tue Apr 17 21:40:31 2018 +0200 Add initial support for btrfs raid5/6 chunk diff --git a/grub-core/fs/btrfs.c b/grub-core/fs/btrfs.c index be1

Re: How to dump/find parity of RAID-5 file?

2018-04-17 Thread Goffredo Baroncelli
On 02/06/2017 09:40 PM, Goffredo Baroncelli wrote: > On 2017-02-03 11:44, Lakshmipathi.G wrote: >> Hi. >> >> Came across this thread >> https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg55161.html >> Exploring possibility of adding test-scripts arou

Re: [PATCH v2] btrfs-progs: dump-super: Refactor print function and add extra check

2018-04-11 Thread Goffredo Baroncelli
erbose... however if this is the constraint, why not print it > > 3) C macro usage >Just a minor problem, macro usage such as SZ_4K/SZ_64K may not looks >good for new developers. I find it quite clear and intuitive even if this is the first time that I notice these macr

Re: [PATCH v2] btrfs-progs: dump-super: Refactor print function and add extra check

2018-04-09 Thread Goffredo Baroncelli
btrfs_stack_device_bytes_used(>dev_item)); > - printf("dev_item.io_align\t%u\n", (unsigned int) > -btrfs_stack_device_io_align(>dev_item)); > - printf("dev_item.io_width\t%u\n", (unsigned int) > -btrfs_stack_device_io

Re: Status of RAID5/6

2018-04-04 Thread Goffredo Baroncelli
On 04/04/2018 08:01 AM, Zygo Blaxell wrote: > On Wed, Apr 04, 2018 at 07:15:54AM +0200, Goffredo Baroncelli wrote: >> On 04/04/2018 12:57 AM, Zygo Blaxell wrote: [...] >> Before you pointed out that the non-contiguous block written has >> an impact on perfor

Re: Status of RAID5/6

2018-04-03 Thread Goffredo Baroncelli
he end of an extent without further data (e.g. the gap to the stripe has to be filled in the same transaction) BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Status of RAID5/6

2018-04-03 Thread Goffredo Baroncelli
On 04/03/2018 02:31 AM, Zygo Blaxell wrote: > On Mon, Apr 02, 2018 at 06:23:34PM -0400, Zygo Blaxell wrote: >> On Mon, Apr 02, 2018 at 11:49:42AM -0400, Austin S. Hemmelgarn wrote: >>> On 2018-04-02 11:18, Goffredo Baroncelli wrote: >>>> I thought that a pos

Re: Status of RAID5/6

2018-04-02 Thread Goffredo Baroncelli
o empty the BG #1, and #2. Otherwise a new BG should be allocated. The cost should be comparable to the logging/journaling (each data shorter than a full-stripe, has to be written two times); the implementation should be quite easy, because already NOW btrfs support BG with different set of disks. BR G.

Re: Status of RAID5/6

2018-03-31 Thread Goffredo Baroncelli
e "old" data should be untouched; or you are saying that the "old" data is rewritten with the same data ? BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send

Re: Status of RAID5/6

2018-03-31 Thread Goffredo Baroncelli
failure+device missing case. However in this case even logging the "new data" would be not sufficient. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubs

Re: [RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

2018-03-30 Thread Goffredo Baroncelli
On 03/30/2018 06:46 AM, Misono Tomohiro wrote: > On 2018/03/30 2:35, Goffredo Baroncelli wrote: >> Hi Misono, > > Hello, [...] >> My conclusion, is that your work is not consistent with the current >> implementation; so I am suggesting to create a new subcommand (&q

Re: [RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

2018-03-30 Thread Goffredo Baroncelli
On 03/30/2018 06:46 AM, Misono Tomohiro wrote: > [Un]fortunately, the stock "btrfs sub list" has the capability to show all > the subvolumes, even the ones not mounted, so this can be entirely replaced > by your API. s/can be entirely/can't be entirely/ -- gpg @keyserver

Re: Status of RAID5/6

2018-03-30 Thread Goffredo Baroncelli
ormat, much less implemented. It could go even further; it would be sufficient to track which *partial* stripes update will be performed before a commit, in one of the btrfs logs. Then in case of a mount of an unclean filesystem, a scrub on these stripes would be sufficient. BR G.Baroncelli [...] -

Re: [RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

2018-03-29 Thread Goffredo Baroncelli
normal user > > btrfs-list.c | 376 > +++-- > btrfs-list.h | 7 +- > cmds-subvolume.c | 14 +- > ioctl.h | 86 +++ > tests/cli-tests/009-subvolume-list/test.

Re: [PATCH v2 0/3] Allow rmdir(2) to delete a subvolume

2018-03-26 Thread Goffredo Baroncelli
() by using btrfs_delete_subvolume() > > fs/btrfs/ctree.h | 5 +- > fs/btrfs/inode.c | 199 > ++- > fs/btrfs/ioctl.c | 185 +-- > 3 files changed, 200 insertions(+), 189 dele

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

2018-03-22 Thread Goffredo Baroncelli
On 03/22/2018 01:15 PM, Austin S. Hemmelgarn wrote: > On 2018-03-21 16:38, Goffredo Baroncelli wrote: >> On 03/21/2018 12:47 PM, Austin S. Hemmelgarn wrote: >>> I agree as well, with the addendum that I'd love to see a new ioctl that >>> does proper permissions che

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

2018-03-21 Thread Goffredo Baroncelli
missions, and only if all the permissions are OK, you can delete the subvolume. However this cannot be a "quick" subvolume delete BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe f

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

2018-03-20 Thread Goffredo Baroncelli
goto out_dput; > > - err = inode_permission(inode, MAY_WRITE | MAY_EXEC); > - if (err) > - goto out_dput; > + err = inode_permission(inode, MAY_WRITE | MAY_EXEC); > + if (err) > +

Re: Crashes running btrfs scrub

2018-03-20 Thread Goffredo Baroncelli
> David, any idea? > > thanks, > liubo > -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.ke

Re: [RFC PATCH v3 5/7] btrfs-progs: sub list: Allow normal user to call "subvolume list/show"

2018-03-19 Thread Goffredo Baroncelli
for the old kernel > } > > - /* > - * now we have an rbtree full of root_info objects, but we need to fill > - * in their path names within the subvol that is referencing each one. > - */ > - ret = list_subvol_fill_paths(fd, root_lookup); >

Re: Crashes running btrfs scrub

2018-03-18 Thread Goffredo Baroncelli
On 03/18/2018 08:57 AM, Goffredo Baroncelli wrote: > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE = 2048 > sizeof(struct btrfs_chunk)) = 48 > sizeof(struct btrfs_stripe) = 32 > > So > > (2048/2-48)/32+1 = 31 > > If my math is correct my math was wr

Re: Crashes running btrfs scrub

2018-03-18 Thread Goffredo Baroncelli
48)/32+1 = 31 If my math is correct gb -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org Mor

Re: [RFC PATCH v2 6/8] btrfs-progs: sub list: Allow normal user to call "subvolume list/show"

2018-03-17 Thread Goffredo Baroncelli
atic int cmd_subvol_list(int argc, char **argv) > goto out; > } > > + ret = check_perm_for_tree_search(fd); > + if (ret < 0) { > + ret = -1; > + error("can't check the permission for tree search: %s", > +

Re: [RFC PATCH v2 3/8] btrfs-progs: sub list: Add helper function which checks the permission for tree search ioctl

2018-03-17 Thread Goffredo Baroncelli
c778..6225311d 100644 > --- a/btrfs-list.h > +++ b/btrfs-list.h > @@ -176,5 +176,6 @@ char *btrfs_list_path_for_root(int fd, u64 root); > int btrfs_list_get_path_rootid(int fd, u64 *treeid); > int btrfs_get_subvol(int fd, struct root_info *the_ri); > int btrfs_get_toplevel_subv

Re: Ongoing Btrfs stability issues

2018-03-14 Thread Goffredo Baroncelli
On 03/14/2018 08:27 PM, Austin S. Hemmelgarn wrote: > On 2018-03-14 14:39, Goffredo Baroncelli wrote: >> On 03/14/2018 01:02 PM, Austin S. Hemmelgarn wrote: >> [...] >>>> >>>> In btrfs, a checksum mismatch creates an -EIO error during the reading. In >

Re: Ongoing Btrfs stability issues

2018-03-14 Thread Goffredo Baroncelli
sole this issue. > dm-integrity uses journaling, and actually based on the testing I've done, > will typically have much worse performance than the overhead of just enabling > COW on files on BTRFS and manually defragmenting them on a regular basis. Good to know > -- gpg @keyserver.linux

Re: Ongoing Btrfs stability issues

2018-03-13 Thread Goffredo Baroncelli
On 03/12/2018 10:48 PM, Christoph Anton Mitterer wrote: > On Mon, 2018-03-12 at 22:22 +0100, Goffredo Baroncelli wrote: >> Unfortunately no, the likelihood might be 100%: there are some >> patterns which trigger this problem quite easily. See The link which >> I posted

Re: Ongoing Btrfs stability issues

2018-03-12 Thread Goffredo Baroncelli
On 03/11/2018 11:37 PM, Christoph Anton Mitterer wrote: > On Sun, 2018-03-11 at 18:51 +0100, Goffredo Baroncelli wrote: >> >> COW is needed to properly checksum the data. Otherwise is not >> possible to ensure the coherency between data and checksum (however I >> have to

Re: Ongoing Btrfs stability issues

2018-03-11 Thread Goffredo Baroncelli
majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > [*] https://www.spinics.net/lists/linux-btrfs/msg69185.html -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe

Re: [PATCH 1/2] btrfs: Add unprivileged subvolume search ioctl

2018-03-07 Thread Goffredo Baroncelli
On 03/07/2018 01:40 AM, Misono, Tomohiro wrote: > On 2018/03/07 5:29, Goffredo Baroncelli wrote: >> On 03/06/2018 09:30 AM, Misono, Tomohiro wrote: >>> Add new unprivileged ioctl (BTRFS_IOC_GET_SUBVOL_INFO) which searches >>> and returns only subvolume related item (ROOT

Re: [PATCH 1/2] btrfs: Add unprivileged subvolume search ioctl

2018-03-06 Thread Goffredo Baroncelli
t; + if (ret == 0 && copy_to_user(>key, , sizeof(sk))) > + ret = -EFAULT; > + return ret; > +} > + > static noinline int btrfs_ioctl_snap_destroy(struct file *file, >void __user *arg) > { > @@ -5663,6 +5915

Re: [PATCH RFC] Btrfs: expose bad chunks in sysfs

2018-02-09 Thread Goffredo Baroncelli
On 02/08/2018 08:07 PM, Liu Bo wrote: > On Thu, Feb 08, 2018 at 07:52:05PM +0100, Goffredo Baroncelli wrote: >> On 02/06/2018 12:15 AM, Liu Bo wrote: >> [...] >>> One way to mitigate the data loss pain is to expose 'bad chunks', >>> i.e. degraded chunks, to u

Re: [PATCH RFC] Btrfs: expose bad chunks in sysfs

2018-02-08 Thread Goffredo Baroncelli
ould be easy understand if the chunk is in "degraded" mode or not. However I have to admit that I don't know how feasible is iterate over a sysfs directory which is a map of a kernel objects list. I think that if these interface would be good enough, we could get rid of a lot of ioctl(TREE_

Re: degraded permanent mount option

2018-01-27 Thread Goffredo Baroncelli
be done sanely from outside the kernel. Adding finer control > would be reasonable ("wait and block" vs "try and return immediately") but > that's about all. It's be also wrong to have a different interface for > daemon X than for humans. > > Ie, the thing syst

Re: [PATCH 11/26] libbtrfsutil: add btrfs_util_create_snapshot()

2018-01-27 Thread Goffredo Baroncelli
On 01/27/2018 06:45 AM, Omar Sandoval wrote: > On Sat, Jan 27, 2018 at 01:00:58PM +0800, Qu Wenruo wrote: >> >> >> On 2018年01月27日 03:46, Omar Sandoval wrote: >>> On Fri, Jan 26, 2018 at 08:31:06PM +0100, Goffredo Baroncelli wrote: >>>> On 01/26/2018 07:40

Re: [PATCH 11/26] libbtrfsutil: add btrfs_util_create_snapshot()

2018-01-26 Thread Goffredo Baroncelli
L_CREATE_SNAPSHOT_RECURSIVE))) { > + errno = EINVAL; > + return BTRFS_UTIL_ERROR_INVALID_ARGUMENT; > + } > + > + if (flags & BTRFS_UTIL_CREATE_SNAPSHOT_READ_ONLY) > + args.flags |= BTRFS_SUBVOL_RDONLY; > + if (async_transid) > +

Re: [RFC] Improve subvolume usability for a normal user

2017-12-05 Thread Goffredo Baroncelli
On 12/05/2017 09:17 PM, Austin S. Hemmelgarn wrote: > On 2017-12-05 14:09, Goffredo Baroncelli wrote: >> On 12/05/2017 07:46 PM, Graham Cobb wrote: >>> On 05/12/17 18:01, Goffredo Baroncelli wrote: >>>> On 12/05/2017 04:42 PM, Graham Cobb wrote: >> [] &

Re: [RFC] Improve subvolume usability for a normal user

2017-12-05 Thread Goffredo Baroncelli
On 12/05/2017 07:46 PM, Graham Cobb wrote: > On 05/12/17 18:01, Goffredo Baroncelli wrote: >> On 12/05/2017 04:42 PM, Graham Cobb wrote: [] >>>>> Then no impact to kernel, all complex work is done in user space. >>>> Exactly how hard is it to ju

Re: [RFC] Improve subvolume usability for a normal user

2017-12-05 Thread Goffredo Baroncelli
/quota info for accessible > subvolumes > == > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > [1] h

Re: [RFC] Improve subvolume usability for a normal user

2017-12-05 Thread Goffredo Baroncelli
't have write access to the parent directory. I assume that this is a POSIX requirement; and if so this should be true even for BTRFS. This means that in order to remove a subvolume and you are not root, you should check all the contained directories. It is not sufficient to check one inode. In the

Re: [PATCH v2] btrfs: handle dynamically reappearing missing device

2017-11-20 Thread Goffredo Baroncelli
On 11/20/2017 09:19 AM, Anand Jain wrote: > > > On 11/18/2017 09:52 PM, Goffredo Baroncelli wrote: >> On 11/17/2017 01:36 PM, Anand Jain wrote: >>> If the device is not present at the time of (-o degrade) mount, >>> the mount context will create a dummy missi

Re: [PATCH v2] btrfs: handle dynamically reappearing missing device

2017-11-18 Thread Goffredo Baroncelli
Because the filesystem is a raid1, btrfs may read from /dev/sdb (updated data) or /dev/sda (old data), or worse read something from the former and something from the later (metadata from sda and data from sdb ) ??? Am I missing something ? BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo B

Re: [PATCH] btrfs: avoid overflow when sector_t is 32 bit

2017-10-04 Thread Goffredo Baroncelli
On 10/04/2017 07:13 PM, Liu Bo wrote: > On Wed, Oct 04, 2017 at 04:22:28PM +0200, David Sterba wrote: >> On Tue, Oct 03, 2017 at 07:31:10PM +0200, Goffredo Baroncelli wrote: >>> From: Goffredo Baroncelli <kreij...@inwind.it> >>> >>> Jean-Denis

[PATCH] btrfs: avoid overflow when sector_t is 32 bit

2017-10-03 Thread Goffredo Baroncelli
From: Goffredo Baroncelli <kreij...@inwind.it> Jean-Denis Girard noticed commit c821e7f3 "pass bytes to btrfs_bio_alloc" (https://patchwork.kernel.org/patch/9763081/) introduces a regression on 32 bit machines. When CONFIG_LBDAF is _not_ defined (CONFIG_LBDAF == Support for la

Re: Mount error on 32 bits, ok on 64 bits

2017-10-02 Thread Goffredo Baroncelli
On 10/02/2017 11:46 PM, Goffredo Baroncelli wrote: > which on 32bit is 32 bit if CONFIG_LBDAF is _not_ defined (CONFIG_LBDAF == > Support for large (2TB+) block devices and files) Only now I noticed that in the first email you attached the config; in fact in your config CONFIG_LBDAF is n

Re: Mount error on 32 bits, ok on 64 bits

2017-10-02 Thread Goffredo Baroncelli
bio = btrfs_bio_alloc(bdev, sector << 9); with bio = btrfs_bio_alloc(bdev, (u64)sector << 9); [1] http://elixir.free-electrons.com/linux/latest/source/fs/btrfs/extent_io.c#L2762 [2] http://elixir.free-electrons.com/linux/latest/source/include/linux/types.h#L133 > >

Re: Why do full balance and deduplication reduce available free space?

2017-10-02 Thread Goffredo Baroncelli
a lot (about 24+7+4 = 35 times) > > Thanks, > Niccolò BR G.Baroncelli > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at  http://vger.kernel.org/majordomo-

Re: What means "top level" in "btrfs subvolume list" ?

2017-10-02 Thread Goffredo Baroncelli
earching a way to handle the snapshots which fully satisfy me (putting them under /, is not the best) BR -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-

Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

2017-10-02 Thread Goffredo Baroncelli
make a usable filesystem ERROR: minimum size for each btrfs device is 41943040 you have to create a big enough ghigo@venice:/tmp$ truncate -s 10G disk.img ghigo@venice:/tmp$ mkfs.btrfs --root test disk.img btrfs-progs v4.12 See http://btrfs.wiki.kernel.org for more information. [...] BR -- gpg @keys

What means "top level" in "btrfs subvolume list" ?

2017-09-30 Thread Goffredo Baroncelli
/tmp/test1/test3 ID = 535 /debian/tmp/test1.snap2 ID = 536 /test ID = 537 /test/sub1 ID = 538 BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To

What means "top id" in "btrfs subvolume list" ?

2017-09-30 Thread Goffredo Baroncelli
sub1 ID = 538 BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Can't remove device -> I/O error

2017-09-30 Thread Goffredo Baroncelli
environment, but I suppose that also on the bare metal it will be quite quick. BR G.Baroncelli > > > > Am 30.09.2017 um 09:16 schrieb Goffredo Baroncelli: >> On 09/30/2017 01:06 AM, DocMAX wrote: >>>>> Did you removed the disk before mounting (physically or doing echo

Re: Can't remove device -> I/O error

2017-09-30 Thread Goffredo Baroncelli
> same i/o error > Did you switch off the machine ? If not, before mounting in degraded mode, do "echo 1 >/sys/block/xxx/device/delete". After the monting do a btrfs dev del missing > -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B

Re: Can't remove device -> I/O error

2017-09-29 Thread Goffredo Baroncelli
re-format. No, it should not be necessary. > > > > Am 29.09.2017 um 23:04 schrieb Goffredo Baroncelli: >> On 09/29/2017 10:00 PM, Dirk Diggler wrote: >>> Hi, >>> >>> is there any chance to get my device removed? >> I simulated a device removing in

Re: Can't remove device -> I/O error

2017-09-29 Thread Goffredo Baroncelli
1.82TiB >Device slack: 0.00B >Data,RAID5:333.00GiB >Data,RAID5:955.00GiB >Data,RAID5: 5.57GiB >Metadata,RAID1: 3.00GiB > Unallocated: 566.44GiB > > Thanks, > DocMAX >

Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

2017-09-29 Thread Goffredo Baroncelli
On 09/28/2017 02:00 AM, Qu Wenruo wrote: > > > On 2017年09月28日 00:20, David Sterba wrote: >> On Mon, Sep 25, 2017 at 07:15:30AM -0400, Austin S. Hemmelgarn wrote: >>> On 2017-09-24 10:08, Goffredo Baroncelli wrote: >>>> On 09/24/2017 12:10 PM, Ana

[PATCH] New ioctl BTRFS_IOC_GET_CHUNK_INFO

2017-09-25 Thread Goffredo Baroncelli
From: Goffredo Baroncelli <kreij...@inwind.it> Add a new ioctl to get info about chunk without requiring the root privileges. This allow to a non root user to know how the space of the filesystem is allocated. Signed-off-by: Goffredo Baroncelli <kreij...@inwind.it> --- fs/b

[RFC] btrfs: new ioctl BTRFS_IOC_GET_CHUNK_INFO

2017-09-25 Thread Goffredo Baroncelli
he api is a good api. In particular I am not sure that returning -EAGAIN is the right thing to do when further data is available but the buffer is not enough. Comments are welcome BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2

[PATCH 2/2] Use the new ioctl BTRFS_IOC_GET_CHUNK_INFO for load_chunk_info()

2017-09-25 Thread Goffredo Baroncelli
From: Goffredo Baroncelli <kreij...@inwind.it> Use the new ioctl BTRFS_IOC_GET_CHUNK_INFO in load_chunk_info() instead of the BTRFS_IOC_TREE_SEARCH. The old method is still present as fallback for compatibility reason. The goal is to avoid BTRFS_IOC_GET_CHUNK_INFO because it require

[RFC] btrfs-progs: use the new ioctl BTRFS_IOC_GET_CHUNK_INFO

2017-09-25 Thread Goffredo Baroncelli
li -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.

[PATCH 1/2] get_partition_size() doens't require root privileges

2017-09-25 Thread Goffredo Baroncelli
From: Goffredo Baroncelli <kreij...@inwind.it> Allow the get_partition_size() to be called without root privileges. Signed-off-by: Goffredo baroncelli <kreij...@inwind.it> --- utils.c | 27 +++ 1 file changed, 19 insertions(+), 8 deletions(-) diff --git

Re: [PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

2017-09-24 Thread Goffredo Baroncelli
remove this feature, it needs fix such as #2 (above) to improve on > #1 (above) as in your list. > > Thanks, Anand > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

2017-09-18 Thread Goffredo Baroncelli
rted in kernel log > 2) read(2) returns successfully with some content being 0x01. > > In order to fix it, we need to report its checksum mismatch error to > the upper layer (dio layer in this case) as well. > > Signed-off-by: Liu Bo <bo.li@oracle.com> > Reporte

[PATCH] btrfs-progs: load_chunk_info() wrong set search key 'type'

2017-09-17 Thread Goffredo Baroncelli
The function load_chunk_info() doesn't initialize correctly the sk->min/max_type when it calls the TREE_SEARCH ioctl: these keys are swapped. Moreover this function assumes that all the items contained in the tree BTRFS_CHUNK_TREE_OBJECTID are of type BTRFS_CHUNK_ITEM_KEY, however some items are

[PATCH] load_chunk_info() : wrong search key 'type'

2017-09-17 Thread Goffredo Baroncelli
From: Goffredo Baroncelli <kreij...@inwind.it> The function load_chunk_info() doesn't initialize correctly the sk->min/max_type when it calls the TREE_SEARCH ioctl: these keys are swapped. Moreover this function assumes that all the items contained in the tree BTRFS_CHUNK_TREE

Re: subvolume list as user?

2017-09-16 Thread Goffredo Baroncelli
> > This is the explantion why it does not work, but it does not help me. I > still have the problem: how can a user get a list of his subvolumes? He > may created them some time ago and forget it. He now wants to have a list > of them. Far to be perfect, but this could help find

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

2017-09-16 Thread Goffredo Baroncelli
in 0+0 records out 0 bytes copied, 0.000404156 s, 0.0 kB/s ghigo@venice:~/btrfs/crash-o-direct/t$ dmesg -T | tail -1 [Sat Sep 16 13:56:41 2017] BTRFS warning (device sdd5): csum failed root 5 ino 257 off 0 csum 0x98f94189 expected csum 0x0ab6be80 mirror 1 > > Signed-off-by: Liu Bo <bo.

Re: A user cannot remove his readonly snapshots?!

2017-09-16 Thread Goffredo Baroncelli
check around "btrfs sub del", so an user can remove an _empty_ subvolume https://www.spinics.net/lists/linux-btrfs/msg06522.html > > BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe f

  1   2   3   4   5   6   7   8   9   10   >