Re: experimental raid5/6 code in git
Hey Chris, On 02/02/2013 05:02 PM, Chris Mason wrote: Btrfs -- 604MB/s MD-- 162MB/s MD -- 800MB/s very little system time Btrfs -- 3.8GB/s one CPU mostly pegged Btrfs -- 380MB/s seen by fio MD-- 174MB/s seen by fio Creating 12 million files on Btrfs raid5 took 226 seconds, vs 485 seconds on MD. Do I read these numbers incorrectly, or does even this first iteration of btrfs' raid5/6 code run circles around MD? Thanks for all the work! Kaspar -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
On Tue, Feb 12, 2013 at 08:16:49AM -0700, Kaspar Schleiser wrote: Hey Chris, On 02/02/2013 05:02 PM, Chris Mason wrote: Btrfs -- 604MB/s MD-- 162MB/s MD -- 800MB/s very little system time Btrfs -- 3.8GB/s one CPU mostly pegged Btrfs -- 380MB/s seen by fio MD-- 174MB/s seen by fio Creating 12 million files on Btrfs raid5 took 226 seconds, vs 485 seconds on MD. Do I read these numbers incorrectly, or does even this first iteration of btrfs' raid5/6 code run circles around MD? Yes and no. Most of the differences were on flash, and really it just looks like MD needs tuning for IO latency and concurrency. There are some MD patches for this recently to add more threads for parity calculations, and these solve some throughput problems. But one thing that we've proven with btrfs is that helper threads mean more IO latencies. So the MD code probably needs some short cuts to do the parity inline as well. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
On Sun, Feb 10, 2013 at 03:35:05PM -0700, Gordon Manning wrote: Hi, Is the BTRFS raid code susceptible to RAID-5 write holes? �I think with the original plan, the problem was avoided by always giving full stripe writes to the raid layers. �Does the current plan deal with the hole in a different manner? The current code in my git tree does not deal with the raid-5 write hole. That's the part I'm finishing off now. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
Hi, I believe XOR_BLOCKS must be selected, otherwise build fails with: ERROR: xor_blocks [fs/btrfs/btrfs.ko] undefined! diff --git a/fs/btrfs/Kconfig b/fs/btrfs/Kconfig index 4f5dc93..5f583c8 100644 --- a/fs/btrfs/Kconfig +++ b/fs/btrfs/Kconfig @@ -7,6 +7,7 @@ config BTRFS_FS select LZO_COMPRESS select LZO_DECOMPRESS select RAID6_PQ + select XOR_BLOCKS help Btrfs is a new filesystem with extents, writable snapshotting, -- Tomasz .. oo o. oo o. .o .o o. o. oo o. .. Torcz.. .o .o .o .o oo oo .o .. .. oo oo o.o.o. .o .. o. o. o. o. o. o. oo .. .. o. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
On Tue, Feb 05, 2013 at 07:22:36AM -0700, Tomasz Torcz wrote: Hi, I believe XOR_BLOCKS must be selected, otherwise build fails with: ERROR: xor_blocks [fs/btrfs/btrfs.ko] undefined! diff --git a/fs/btrfs/Kconfig b/fs/btrfs/Kconfig index 4f5dc93..5f583c8 100644 --- a/fs/btrfs/Kconfig +++ b/fs/btrfs/Kconfig @@ -7,6 +7,7 @@ config BTRFS_FS select LZO_COMPRESS select LZO_DECOMPRESS select RAID6_PQ + select XOR_BLOCKS help Btrfs is a new filesystem with extents, writable snapshotting, Thanks, I've put this in. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root, char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+ if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+ printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ ret = -EINVAL;
+ goto out;
+ }
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root, char
*device_path)
goto out;
}
+ if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+ root-fs_info-fs_devices-rw_devices = 2) {
+ printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ ret = -EINVAL;
+ goto out;
+ }
+ if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+ root-fs_info-fs_devices-rw_devices = 3) {
+ printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ ret = -EINVAL;
+ goto out;
+ }
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
-hpa
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root, char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+ if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+ printk(KERN_ERR btrfs: unable to go below three devices
+on raid5 or raid6\n);
+ ret = -EINVAL;
+ goto out;
+ }
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root, char
*device_path)
goto out;
}
+ if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+ root-fs_info-fs_devices-rw_devices = 2) {
+ printk(KERN_ERR btrfs: unable to go below two
+devices on raid5\n);
+ ret = -EINVAL;
+ goto out;
+ }
+ if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+ root-fs_info-fs_devices-rw_devices = 3) {
+ printk(KERN_ERR btrfs: unable to go below three
+devices on raid6\n);
+ ret = -EINVAL;
+ goto out;
+ }
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
Also, a 2-member raid5 or 3-member raid6 are a raid1 and can be treated as such.
Chris Mason chris.ma...@fusionio.com wrote:
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
goto out;
}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+root-fs_info-fs_devices-rw_devices = 2) {
+printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ret = -EINVAL;
+goto out;
+}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+root-fs_info-fs_devices-rw_devices = 3) {
+printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
Sent from my mobile phone. Please excuse brevity and lack of formatting.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
I felt like having a small play with this stuff, as I've been wanting
it for so long :)
But apparently I've made some incredibly newb error.
I used the following two lines to check out the code:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git
raid56-experimental
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
raid56-experimental-progs
Then I did not very much to compile both of them (installed lots and
lots of packages that various places told me would be needed so they'd
both compile) finishing up with a sudo make install for both the
kernel and the tools.
Rebooting miracuously it came up with the new kernel and uname -a
assures me that I have a new kernel running:
btrfs@ubuntu:/kernel/raid56-experimental$ uname -a
Linux ubuntu 3.6.0+ #1 SMP Tue Feb 5 12:26:03 EST 2013 x86_64 x86_64
x86_64 GNU/Linux
but 3.6.0 sounds rather low, but it is newer than Ubuntu 12.10's 3.5
so I believe I am running the kernel I just compiled
Where things fail is that I can figure out how to make a raid5 btrfs,
I'm certain I'm using the mkfs.btrfs that I just compiled (by
explicitly calling it in the make folder) but it wont recognise what I
assume the parameter to be:
btrfs@ubuntu:/kernel/raid56-experimental-progs$ ./mkfs.btrfs -m raid5
-d raid5 /dev/sd[bcdef]
Unknown profile raid5
Which flavour of newb am I today?
PS: I use newb in a very friendly way, I feel no shame over that term :)
On Tue, Feb 5, 2013 at 1:26 PM, H. Peter Anvin h...@zytor.com wrote:
Also, a 2-member raid5 or 3-member raid6 are a raid1 and can be treated as
such.
Chris Mason chris.ma...@fusionio.com wrote:
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
goto out;
}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+root-fs_info-fs_devices-rw_devices = 2) {
+printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ret = -EINVAL;
+goto out;
+}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+root-fs_info-fs_devices-rw_devices = 3) {
+printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
Sent from my mobile phone. Please excuse brevity and lack of formatting.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gar...@cerberos.id.au - www.rockpaperdynamite.wordpress.com
Dear God, I would like to file a bug report
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
The last argument should be the directory you want to clone into. Use
'-b branch' to specify the branch you want to clone. I'm pretty sure
you've compiled just the master branch of both linux-btrfs and
btrfs-progs.
On Mon, Feb 4, 2013 at 8:59 PM, Gareth Pye gar...@cerberos.id.au wrote:
I felt like having a small play with this stuff, as I've been wanting
it for so long :)
But apparently I've made some incredibly newb error.
I used the following two lines to check out the code:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git
raid56-experimental
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
raid56-experimental-progs
Then I did not very much to compile both of them (installed lots and
lots of packages that various places told me would be needed so they'd
both compile) finishing up with a sudo make install for both the
kernel and the tools.
Rebooting miracuously it came up with the new kernel and uname -a
assures me that I have a new kernel running:
btrfs@ubuntu:/kernel/raid56-experimental$ uname -a
Linux ubuntu 3.6.0+ #1 SMP Tue Feb 5 12:26:03 EST 2013 x86_64 x86_64
x86_64 GNU/Linux
but 3.6.0 sounds rather low, but it is newer than Ubuntu 12.10's 3.5
so I believe I am running the kernel I just compiled
Where things fail is that I can figure out how to make a raid5 btrfs,
I'm certain I'm using the mkfs.btrfs that I just compiled (by
explicitly calling it in the make folder) but it wont recognise what I
assume the parameter to be:
btrfs@ubuntu:/kernel/raid56-experimental-progs$ ./mkfs.btrfs -m raid5
-d raid5 /dev/sd[bcdef]
Unknown profile raid5
Which flavour of newb am I today?
PS: I use newb in a very friendly way, I feel no shame over that term :)
On Tue, Feb 5, 2013 at 1:26 PM, H. Peter Anvin h...@zytor.com wrote:
Also, a 2-member raid5 or 3-member raid6 are a raid1 and can be treated as
such.
Chris Mason chris.ma...@fusionio.com wrote:
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
goto out;
}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+root-fs_info-fs_devices-rw_devices = 2) {
+printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ret = -EINVAL;
+goto out;
+}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+root-fs_info-fs_devices-rw_devices = 3) {
+printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
Sent from my mobile phone. Please excuse brevity and lack of formatting.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gar...@cerberos.id.au - www.rockpaperdynamite.wordpress.com
Dear God, I would like to file a bug report
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
Thank you, that makes a lot of sense :)
It's been a good day, I've learnt something :)
On Tue, Feb 5, 2013 at 4:29 PM, Chester somethingsome2...@gmail.com wrote:
The last argument should be the directory you want to clone into. Use
'-b branch' to specify the branch you want to clone. I'm pretty sure
you've compiled just the master branch of both linux-btrfs and
btrfs-progs.
On Mon, Feb 4, 2013 at 8:59 PM, Gareth Pye gar...@cerberos.id.au wrote:
I felt like having a small play with this stuff, as I've been wanting
it for so long :)
But apparently I've made some incredibly newb error.
I used the following two lines to check out the code:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git
raid56-experimental
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
raid56-experimental-progs
Then I did not very much to compile both of them (installed lots and
lots of packages that various places told me would be needed so they'd
both compile) finishing up with a sudo make install for both the
kernel and the tools.
Rebooting miracuously it came up with the new kernel and uname -a
assures me that I have a new kernel running:
btrfs@ubuntu:/kernel/raid56-experimental$ uname -a
Linux ubuntu 3.6.0+ #1 SMP Tue Feb 5 12:26:03 EST 2013 x86_64 x86_64
x86_64 GNU/Linux
but 3.6.0 sounds rather low, but it is newer than Ubuntu 12.10's 3.5
so I believe I am running the kernel I just compiled
Where things fail is that I can figure out how to make a raid5 btrfs,
I'm certain I'm using the mkfs.btrfs that I just compiled (by
explicitly calling it in the make folder) but it wont recognise what I
assume the parameter to be:
btrfs@ubuntu:/kernel/raid56-experimental-progs$ ./mkfs.btrfs -m raid5
-d raid5 /dev/sd[bcdef]
Unknown profile raid5
Which flavour of newb am I today?
PS: I use newb in a very friendly way, I feel no shame over that term :)
On Tue, Feb 5, 2013 at 1:26 PM, H. Peter Anvin h...@zytor.com wrote:
Also, a 2-member raid5 or 3-member raid6 are a raid1 and can be treated as
such.
Chris Mason chris.ma...@fusionio.com wrote:
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
goto out;
}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+root-fs_info-fs_devices-rw_devices = 2) {
+printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ret = -EINVAL;
+goto out;
+}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+root-fs_info-fs_devices-rw_devices = 3) {
+printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
Sent from my mobile phone. Please excuse brevity and lack of formatting.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gar...@cerberos.id.au - www.rockpaperdynamite.wordpress.com
Dear God, I would like to file a bug report
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gar...@cerberos.id.au - www.rockpaperdynamite.wordpress.com
Dear God, I would like to file a bug report
On Tue, Feb 5, 2013 at 4:29 PM, Chester somethingsome2...@gmail.com wrote:
The last argument should be the directory you want to clone into. Use
'-b branch' to specify the branch you want to clone. I'm pretty sure
you've compiled just the master branch of both linux-btrfs and
btrfs-progs.
On Mon, Feb 4, 2013 at 8:59 PM, Gareth Pye gar...@cerberos.id.au wrote:
I felt like having a small play with this stuff, as I've been wanting
it for so long
Re: experimental raid5/6 code in git
Hi Chris, I've been keen for raid5/6 in btrfs since I heard of it. I cannot give you any feedback, but I'd like to take the opportunity to thank you -and all contributors (thinking of David for the raid) for your work. Regards, Hendrik -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
