Re: IO errors when building RAID1.... ?
On Sat, Sep 1, 2018 at 1:03 AM, Pierre Couderc wrote: > > > On 08/31/2018 08:52 PM, Chris Murphy wrote: >> >> >> Bad sector which is failing write. This is fatal, there isn't anything >> the block layer or Btrfs (or ext4 or XFS) can do about it. Well, >> ext234 do have an option to scan for bad sectors and create a bad >> sector map which then can be used at mkfs time, and ext234 will avoid >> using those sectors. And also the md driver has a bad sector option >> for the same, and does remapping. But XFS and Btrfs don't do that. >> >> If the drive is under warranty, get it swapped out, this is definitely >> a warranty covered problem. >> >> >> >> > Thank you very much. > > Once upon a time...(I am old), there were lists of bad sectors, and the > software did avoid wrting in them. It seems to have disappeared. For which > reason ? Maybe because these errors occur so rarely, that it is not worth > the trouble ? For > 10 years drive firmware handles bad sector remapping internally. It remaps the sector logical address to a reserve physical sector. NTFS and ext[234] have a means of accepting a list of bad sectors, and will avoid using them. Btrfs doesn't. But also ZFS, XFS, APFS, HFS+ and I think even FAT, lack this capability. I'm not aware of any file system that once had bad sector tracking, that has since dropped the capability. -- Chris Murphy
Re: bug: btrfs-progs scrub -R flag doesn't show per device stats
On 09/01/2018 02:43 AM, Chris Murphy wrote: btrfs-progs v4.17.1 man btrfs-scrub: -R print raw statistics per-device instead of a summary However, on a two device Btrfs volume, -R does not show per device statistics. See screenshot: https://drive.google.com/open?id=1xmt_NHGlNJPc8I0F4_OZxgGe9b3quCnD Additionally, the description of -d and -R doesn't help me distinquish between the two. -R says "instead of a summary" so that suggests -d will summarize but isn't explicitly stated. The description of '-R' is incorrect, it shoudle be 'raw print mode, print full data instead of summary'. The option '-d' do print separate statistics for each device indded, not the '-R'. A patch has been sent(forgot to CC you). Thanks, Su
[PATCH] btrfs-progs: docs: update document about option -R of btrfs-scrub
The option '-R' of btrfs-scrub was documented by mistake as
'print raw statistics per-device instead of a summary'.
Here change it to 'raw print mode, print full data instead of
summary' which it works actually.
Fixes: 162257574a56 ("btrfs-progs: docs: update btrfs-scrub")
Reported-by: Chris Murphy
Signed-off-by: Su Yue
---
Documentation/btrfs-scrub.asciidoc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/btrfs-scrub.asciidoc
b/Documentation/btrfs-scrub.asciidoc
index d2d206275c36..4c49269e2f36 100644
--- a/Documentation/btrfs-scrub.asciidoc
+++ b/Documentation/btrfs-scrub.asciidoc
@@ -69,7 +69,7 @@ be quiet, omit error messages and statistics
run in read-only mode, do not attempt to correct anything, can be run on a
read-only
filesystem
-R
-print raw statistics per-device instead of a summary
+raw print mode, print full data instead of summary
-c
set IO priority class (see `ionice`(1) manpage)
-n
--
2.18.0
Re: [PATCH 0/4] Userspace support for FSID change
On 2018/9/2 下午5:56, Nikolay Borisov wrote: > > > On 29.08.2018 15:47, Qu Wenruo wrote: >> >> >> On 2018/8/29 下午8:33, Nikolay Borisov wrote: >>> >>> >>> On 29.08.2018 15:09, Qu Wenruo wrote: On 2018/8/29 下午4:35, Nikolay Borisov wrote: > Here is the userspace tooling support for utilising the new metadata_uuid > field, > enabling the change of fsid without having to rewrite every metadata > block. This > patchset consists of adding support for the new field to various tools > and > files (Patch 1). The actual implementation of the new -m|-M options > (which are > described in more detail in Patch 2). A new misc-tests testcasei (Patch > 3) which > exercises the new options and verifies certain invariants hold (these are > also > described in Patch2). Patch 4 is more or less copy of the kernel > conuterpart > just reducing some duplication between btrfs_fs_info and btrfs_fs_devices > structures. So to my understand, now we have another layer of UUID. Before we have one fsid, both used in superblock and tree blocks. Now we have 2 fsid, the one used in tree blocks are kept the same, but changed its name to metadata_uuid in superblock. And superblock::fsid will become a new field, and although they are the same at mkfs time, they could change several times during its operation. This indeed makes uuid change super fast, only needs to update all superblocks of the fs, instead of all tree blocks. However I have one nitpick of the design. Unlike XFS, btrfs supports multiple devices. If we have a raid10 fs with 4 devices, and it has already gone through several UUID change (so its metadata uuid is already different from fsid). And during another UUID change procedure, we lost power while only updated 2 super blocks, what will happen for kernel device assembly? (Although considering how fast the UUID change would happen, such case should be super niche) >>> >>> Then I guess you will be fucked. I'm all ears for suggestion how to >>> rectify this without skyrocketing the complexity. The current UUID >>> rewrite method sets a flag int he superblock that FSID change is in >>> progress and clears it once every metadatablock has been rewritten. I >>> can piggyback on this mechanism but I'm not sure it provides 100% >>> guarantee. Because by the some token you can set this flag, start >>> writing the super blocks then lose power and then only some of the >>> superblocks could have this flag set so we back at square 1. >> >> Well, forget it, considering how fast the new method is, such case >> should be really rare. >> >>> > > The intended usecase of this feature is to give the sysadmin the ability > to > create copies of filesystesm, change their uuid quickly and mount them > alongside > the original filesystem for, say, forensic purposes. > > One thing which still hasn't been set in stone is whether the new options > will remain as -m|-M or whether they should subsume the current -u|-U - > from > the point of view of users nothing should change. Well, user would be surprised by how fast the new -m is, thus there is still something changed :) I prefer to subsume current -u/-U, and use the new one if the incompat feature is already set. Or fall back to original behavior. But I'm not a fan of using INCOMPAT flags as an indicator of changed fsid/metadata uuid. INCOMPAT feature should not change so easily nor acts as an indicator. That's to say, the flag should only be set at mkfs time, and then never change unlike the 2nd patch (I don't even like btrfstune to change incompat flags). E.g. mkfs.btrfs -O metadata_uuid , then we could use the new way to change fsid without touching metadata uuid. Or we could only use the old method. >>> >>> I disagree, I don't see any benefit in this but only added complexity. >>> Can you elaborate more ? >> >> Well, the incompat feature is really introducing some incompatible >> on-disk format change. >> So if you're introducing the new metadata_uuid field, no matter if it >> differs from fsid or not, it's a new field, and the incompat flag should >> be set. >> >> To me, older kernel could recognize the new format when fsid matches >> metadata uuid (since there in your current patchset, such case will not >> have incompat flag set) is a little dangerous. >> >> What will happen if such old kernel/btrfs-progs tries to change fsid? >> Older btrfs-progs doesn't know there is a new field, it will not touch >> the metadata uuid field, just changing the fsid field along with all >> tree blocks. >> >> This will cause a fs whose fsid doesn't match metadata uuid and has no >> incompat flag set. This is definitely leading to compatibility problem. > >
Re: [PATCH 0/4] Userspace support for FSID change
On 29.08.2018 15:47, Qu Wenruo wrote: > > > On 2018/8/29 下午8:33, Nikolay Borisov wrote: >> >> >> On 29.08.2018 15:09, Qu Wenruo wrote: >>> >>> >>> On 2018/8/29 下午4:35, Nikolay Borisov wrote: Here is the userspace tooling support for utilising the new metadata_uuid field, enabling the change of fsid without having to rewrite every metadata block. This patchset consists of adding support for the new field to various tools and files (Patch 1). The actual implementation of the new -m|-M options (which are described in more detail in Patch 2). A new misc-tests testcasei (Patch 3) which exercises the new options and verifies certain invariants hold (these are also described in Patch2). Patch 4 is more or less copy of the kernel conuterpart just reducing some duplication between btrfs_fs_info and btrfs_fs_devices structures. >>> >>> So to my understand, now we have another layer of UUID. >>> >>> Before we have one fsid, both used in superblock and tree blocks. >>> >>> Now we have 2 fsid, the one used in tree blocks are kept the same, but >>> changed its name to metadata_uuid in superblock. >>> And superblock::fsid will become a new field, and although they are the >>> same at mkfs time, they could change several times during its operation. >>> >>> This indeed makes uuid change super fast, only needs to update all >>> superblocks of the fs, instead of all tree blocks. >>> >>> However I have one nitpick of the design. Unlike XFS, btrfs supports >>> multiple devices. >>> If we have a raid10 fs with 4 devices, and it has already gone through >>> several UUID change (so its metadata uuid is already different from fsid). >>> >>> And during another UUID change procedure, we lost power while only >>> updated 2 super blocks, what will happen for kernel device assembly? >>> >>> (Although considering how fast the UUID change would happen, such case >>> should be super niche) >> >> Then I guess you will be fucked. I'm all ears for suggestion how to >> rectify this without skyrocketing the complexity. The current UUID >> rewrite method sets a flag int he superblock that FSID change is in >> progress and clears it once every metadatablock has been rewritten. I >> can piggyback on this mechanism but I'm not sure it provides 100% >> guarantee. Because by the some token you can set this flag, start >> writing the super blocks then lose power and then only some of the >> superblocks could have this flag set so we back at square 1. > > Well, forget it, considering how fast the new method is, such case > should be really rare. > >> >>> The intended usecase of this feature is to give the sysadmin the ability to create copies of filesystesm, change their uuid quickly and mount them alongside the original filesystem for, say, forensic purposes. One thing which still hasn't been set in stone is whether the new options will remain as -m|-M or whether they should subsume the current -u|-U - from the point of view of users nothing should change. >>> >>> Well, user would be surprised by how fast the new -m is, thus there is >>> still something changed :) >>> >>> I prefer to subsume current -u/-U, and use the new one if the incompat >>> feature is already set. Or fall back to original behavior. >>> >>> But I'm not a fan of using INCOMPAT flags as an indicator of changed >>> fsid/metadata uuid. >>> INCOMPAT feature should not change so easily nor acts as an indicator. >>> >>> That's to say, the flag should only be set at mkfs time, and then never >>> change unlike the 2nd patch (I don't even like btrfstune to change >>> incompat flags). >>> >>> E.g. >>> mkfs.btrfs -O metadata_uuid , then we could use the new way to >>> change fsid without touching metadata uuid. >>> Or we could only use the old method. >> >> I disagree, I don't see any benefit in this but only added complexity. >> Can you elaborate more ? > > Well, the incompat feature is really introducing some incompatible > on-disk format change. > So if you're introducing the new metadata_uuid field, no matter if it > differs from fsid or not, it's a new field, and the incompat flag should > be set. > > To me, older kernel could recognize the new format when fsid matches > metadata uuid (since there in your current patchset, such case will not > have incompat flag set) is a little dangerous. > > What will happen if such old kernel/btrfs-progs tries to change fsid? > Older btrfs-progs doesn't know there is a new field, it will not touch > the metadata uuid field, just changing the fsid field along with all > tree blocks. > > This will cause a fs whose fsid doesn't match metadata uuid and has no > incompat flag set. This is definitely leading to compatibility problem. Nope, when both fsid/metadata_uuid match and the INCOMPAT flag is not set then on-disk we never save the metadata_uuid value and this value is set only in-memory, the
