Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Thu, May 03, 2012 at 03:11:45PM +0200, Hubert Kario wrote: nice, didn't know about this. Such functionality would be nice to have. But then I don't think that a recreate the array if the parameters are the same is actually a good idea, lots of space for error. A pair of functions: btrfs dev zero-superblock btrfs dev restore-superblock As a user, I'm not sure what can I expect from the restore command. From where does it restore? Eg. a file? As a tester I have use for a temporary clearing of a superblock on a device, then mount it with -o degraded, work work, and then undo clearing. So, my idea is like btrfs device zero-superblock --undo with the obvious sanity checks. A regular user would never need to call this. david -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 09 of May 2012 19:18:07 David Sterba wrote: On Thu, May 03, 2012 at 03:11:45PM +0200, Hubert Kario wrote: nice, didn't know about this. Such functionality would be nice to have. But then I don't think that a recreate the array if the parameters are the same is actually a good idea, lots of space for error. A pair of functions: btrfs dev zero-superblock btrfs dev restore-superblock As a user, I'm not sure what can I expect from the restore command. From where does it restore? Eg. a file? As a tester I have use for a temporary clearing of a superblock on a device, then mount it with -o degraded, work work, and then undo clearing. So, my idea is like btrfs device zero-superblock --undo with the obvious sanity checks. A regular user would never need to call this. Yes, that's a better idea. -- Hubert Kario QBS - Quality Business Software 02-656 Warszawa, ul. Ksawerów 30/85 tel. +48 (22) 646-61-51, 646-74-24 www.qbs.com.pl -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 02 of May 2012 19:36:29 David Sterba wrote:
On Wed, May 02, 2012 at 06:42:16PM +0200, Hubert Kario wrote:
I'm not sure if this is useful and sensible usecase, clearing superblock
is a one-time action anyway, so it's more for the sake of tool
flexibility.
Clearing superblock is not a light decision and should generally be
performed just before formatting the partition with some other fs or
physical volume for LVM. IMHO recoverability of cleared superblock is a
function hardly anyone would use.
googled, a few users asking about recovering from md zero-superblock, and
the solution was to recreate the array, md is said to be smart and
recognize traces of previous array and will not destroy it if the
parameters are same. Point for md, btrfs does not do this.
nice, didn't know about this. Such functionality would be nice to have.
But then I don't think that a recreate the array if the parameters are the
same is actually a good idea, lots of space for error. A pair of functions:
btrfs dev zero-superblock
btrfs dev restore-superblock
would be a better solution IMO
To your implementation: I think adding a function doing the superblock
reset would be enough here. Something like this (in pseudocode):
for (i = 0 ; i BTRFS_SUPER_MIRROR_MAX; i++) {
bytenr = btrfs_sb_offset(i);
break if bytenr device size
memset(superblock buffer, CLEARPATTERN, sizeof(...))
}
write_all_supers(root);
That's exactly what btrfs_prepare_device does. And it's a function run by
btfs just before btrfs dev add and by mkfs. Duplicating its code would be
a bad idea.
Not 'exactly' IMO:
* calls TRIM/discard on the device
* zeroes first 2 megabytes
* zeroes all reachable superblocks
* zeroes last 2 megabytes
Too many undocumented and unobvious side-efects.
True. But close enough ;)
Code duplication can be avoided by factoring the 'zero superblock' into
a function and calling it from btrfs_prepare_device().
Then there's also the actually zero vs reversibly destroy difference but
it's trivial to fix using a single option.
Regards
--
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawerów 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Tue, May 01, 2012 at 02:40:29PM +0200, Hubert Kario wrote:
+static const char * const cmd_zero_dev_usage[] = {
+ btrfs device zero-superblock device [device ...],
FYI, this step is named 'clear superblock' in kernel code as done after the
device is removed, and I suggest to consider to name the command like
'clear-superblock' or 'clear-sb'.
Also, kernel clears only the superblock magic string, ie. the
_BHRfS_M bytes. This leaves the rest of the superblock intact, for
possible recovery when it's cleared accidentally.
I had prototyped a similar utility (in perl, so nothing for progs
inclusion for now) and rewrote the magic string with _BHRf$_M ie. the
S - $ for visual similarity with the action performed. This allows to
detect cleared superblocks and activate them back eventually.
I'm not sure if this is useful and sensible usecase, clearing superblock
is a one-time action anyway, so it's more for the sake of tool
flexibility.
To your implementation: I think adding a function doing the superblock
reset would be enough here. Something like this (in pseudocode):
for (i = 0 ; i BTRFS_SUPER_MIRROR_MAX; i++) {
bytenr = btrfs_sb_offset(i);
break if bytenr device size
memset(superblock buffer, CLEARPATTERN, sizeof(...))
}
write_all_supers(root);
david
+ Remove btrfs filesystem superblock from devices.,
+ WARNING! This command will make filesystem residing on the devices,
+ completely unmountable!,
+ NULL
+};
+
+static int cmd_zero_dev(int argc, char **argv)
+{
+ int fd;
+ char *file;
+ int arg_processed;
+ int ret = 0;
+ int n;
+ u64 device_len;
+ int mixed_mode_needed = 1; /* keep btrfs_prepare_device() quiet */
+ const int ZERO_END = 1;
+
+ if( argc 2 ) {
+ usage(cmd_zero_dev_usage);
+ }
+
+ for(arg_processed = 1; arg_processed argc; arg_processed++) {
+ file = argv[arg_processed];
+
+ fd = open(file, O_RDWR);
+ if (fd 0) {
+ fprintf(stderr, Unable to open %s\n, file);
+ ret |= 1;
+ continue;
+ }
+
+ n = btrfs_prepare_device(fd, file, ZERO_END, device_len,
+ mixed_mode_needed);
+ if (n) {
+ fprintf(stderr, Error when zeroing out %s\n, file);
+ ret |= n;
+ }
+
+ close(fd);
+ }
+
+ return ret;
+}
+
const struct cmd_group device_cmd_group = {
device_cmd_group_usage, NULL, {
{ add, cmd_add_dev, cmd_add_dev_usage, NULL, 0 },
{ delete, cmd_rm_dev, cmd_rm_dev_usage, NULL, 0 },
{ scan, cmd_scan_dev, cmd_scan_dev_usage, NULL, 0 },
+ { zero-superblock, cmd_zero_dev, cmd_zero_dev_usage, NULL, 0
},
{ 0, 0, 0, 0, 0 }
}
};
diff --git a/man/btrfs.8.in b/man/btrfs.8.in
index be478e0..a840f7e 100644
--- a/man/btrfs.8.in
+++ b/man/btrfs.8.in
@@ -39,6 +39,8 @@ btrfs \- control a btrfs filesystem
.PP
\fBbtrfs\fP \fBdevice delete\fP\fI device [device...] path \fP
.PP
+\fBbtrfs\fP \fBdevice zero-superblock\fP\fI device [device...] \fP
+.PP
\fBbtrfs\fP \fBscrub start\fP [-Bdqru] {\fIpath\fP|\fIdevice\fP}
.PP
\fBbtrfs\fP \fBscrub cancel\fP {\fIpath\fP|\fIdevice\fP}
@@ -230,6 +232,11 @@ Finally, if \fB--all-devices\fP is passed, all the
devices under /dev are
scanned.
.TP
+\fBdevice zero-superblock\fR\fI dev [dev..]\fR
+The space on the disk where btrfs metadata can reside is overwritten with
+zeros.
+.TP
+
\fBscrub start\fP [-Bdqru] {\fIpath\fP|\fIdevice\fP}
Start a scrub on all devices of the filesystem identified by \fIpath\fR or
on
a single \fIdevice\fR. Without options, scrub is started as a background
--
1.7.10
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wed, May 02, 2012 at 04:28:43PM +0200, David Sterba wrote:
I had prototyped a similar utility (in perl, so nothing for progs
inclusion for now)
attached.
david
#!/usr/bin/perl
# clear btrfs signature from a device
use Fcntl;
use constant BTRFS_SUPER_INFO_OFFSET = 64 * 1024;
use constant BTRFS_SUPER_INFO_SIZE = 4096;
use constant BTRFS_SUPER_MIRROR_MAX = 3;
use constant BTRFS_SUPER_MIRROR_SHIFT = 12;
use constant BTRFS_MAGIC = _BHRfS_M;
use constant BTRFS_DEAD = '_BHRf$_M';
sub btrfs_sb_offset($) {
my $mirror =$_[0];
my $start = 16 * 1024;
if ($mirror0) {
return $start (BTRFS_SUPER_MIRROR_SHIFT * $mirror);
}
return BTRFS_SUPER_INFO_OFFSET;
}
my $dbg=1;
my $savesb=0;
# main
my $dev=$ARGV[0];
my $size;
if(!-b $dev) {
print(Not a block device: $dev\n);
$size=(stat($dev))[7];
} else {
$size=`blockdev --getsize64 $dev`;
}
sysopen(F, $dev, O_EXCL | O_RDWR) or die(Cannot open $dev exclusively: $!);
print(Device size: $size\n) if($dbg);
for(my $i=0;$i6;$i++) {
my $off=btrfs_sb_offset($i);
if($off $size) {
print(Offset for SB $i beyond EOF\n) if($dbg);
last;
}
print(Offset $i is $off\n) if($dbg);
sysseek(F, $off, 0);
sysread(F, $buf, BTRFS_SUPER_INFO_SIZE);
if($savesb) {
open(Q,SB$i);
print Q ($buf);
close(Q);
}
my $sbmagic=substr($buf, 0x40, length(BTRFS_MAGIC));
print(SB magic: $sbmagic\n) if($dbg);
if(BTRFS_MAGIC eq $sbmagic) {
print(Found a valid signature of superblock $i\n);
sysseek(F, $off + 0x40, 0);
print(Clearing...\n);
syswrite(F, BTRFS_DEAD, length(BTRFS_DEAD));
} elsif(BTRFS_DEAD eq $sbmagic) {
print(Found a signature of a dead superblock $i\n);
} else {
print(Superblock $i does not look like a btrfs one\n);
}
}
close(F);
print(Syncing dev\n);
if (!-b $dev) {
system(fsync \'$dev\');
} else {
system(blockdev --flushbufs \'$dev\');
}
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 02 of May 2012 16:28:43 David Sterba wrote:
On Tue, May 01, 2012 at 02:40:29PM +0200, Hubert Kario wrote:
+static const char * const cmd_zero_dev_usage[] = {
+ btrfs device zero-superblock device [device ...],
FYI, this step is named 'clear superblock' in kernel code as done after the
device is removed, and I suggest to consider to name the command like
'clear-superblock' or 'clear-sb'.
A similar function in mdadm is called zero superblock so I just re used the
name (according to the principle of least surprise). Users, even admins,
generally don't read kernel code...
Also, kernel clears only the superblock magic string, ie. the
_BHRfS_M bytes. This leaves the rest of the superblock intact, for
possible recovery when it's cleared accidentally.
That's when a device is removed from the filesystem, not when a filesystem is
just not used any more and you want to re-purpose the devices.
I had prototyped a similar utility (in perl, so nothing for progs
inclusion for now) and rewrote the magic string with _BHRf$_M ie. the
S - $ for visual similarity with the action performed. This allows to
detect cleared superblocks and activate them back eventually.
I'm not sure if this is useful and sensible usecase, clearing superblock
is a one-time action anyway, so it's more for the sake of tool
flexibility.
Clearing superblock is not a light decision and should generally be performed
just before formatting the partition with some other fs or physical volume for
LVM. IMHO recoverability of cleared superblock is a function hardly anyone
would use.
To your implementation: I think adding a function doing the superblock
reset would be enough here. Something like this (in pseudocode):
for (i = 0 ; i BTRFS_SUPER_MIRROR_MAX; i++) {
bytenr = btrfs_sb_offset(i);
break if bytenr device size
memset(superblock buffer, CLEARPATTERN, sizeof(...))
}
write_all_supers(root);
That's exactly what btrfs_prepare_device does. And it's a function run by btfs
just before btrfs dev add and by mkfs. Duplicating its code would be a bad
idea.
Regards,
--
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawerów 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wed, May 02, 2012 at 06:42:16PM +0200, Hubert Kario wrote:
A similar function in mdadm is called zero superblock so I just re used the
name (according to the principle of least surprise). Users, even admins,
generally don't read kernel code...
I intended to point out that the functionality is there under different
name, but the 'least surprise' and tool compatibility plays for
'zero-superblock' naming.
I'm not sure if this is useful and sensible usecase, clearing superblock
is a one-time action anyway, so it's more for the sake of tool
flexibility.
Clearing superblock is not a light decision and should generally be performed
just before formatting the partition with some other fs or physical volume
for
LVM. IMHO recoverability of cleared superblock is a function hardly anyone
would use.
googled, a few users asking about recovering from md zero-superblock, and
the solution was to recreate the array, md is said to be smart and
recognize traces of previous array and will not destroy it if the
parameters are same. Point for md, btrfs does not do this.
To your implementation: I think adding a function doing the superblock
reset would be enough here. Something like this (in pseudocode):
for (i = 0 ; i BTRFS_SUPER_MIRROR_MAX; i++) {
bytenr = btrfs_sb_offset(i);
break if bytenr device size
memset(superblock buffer, CLEARPATTERN, sizeof(...))
}
write_all_supers(root);
That's exactly what btrfs_prepare_device does. And it's a function run by
btfs
just before btrfs dev add and by mkfs. Duplicating its code would be a bad
idea.
Not 'exactly' IMO:
* calls TRIM/discard on the device
* zeroes first 2 megabytes
* zeroes all reachable superblocks
* zeroes last 2 megabytes
Too many undocumented and unobvious side-efects.
Code duplication can be avoided by factoring the 'zero superblock' into
a function and calling it from btrfs_prepare_device() .
david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
