Re: [gentoo-user] Duplicate identical Hard Disk
On Thu, Apr 1, 2010 at 10:09 PM, Mark Knecht markkne...@gmail.com wrote: With only 2 disks I personally think you're on the right path. With 3 disks I'm personally planning on RAID1 using 3 copies. ... My comment about RAID was that I am learning the hard (alas expensive) way that not all disks can actually do RAID, at least not Linux software RAID, and really be usable. From what I understand of software RAID in linux, it works on block devices, not disks. This means if some endeavoring soul was brave enough to RAID even partitions on a device, it would work as normal. Perhaps you mean that not all properly functioning disks can do RAID? What sort of trouble are you running into? I've successfully deployed both RAID1 and RAID5 on my home media server for quite some time now. While the initial time investment in reading documentation was considerable, since that time I've had no cause for trouble. I keep smartmontools looking at the array member disks and regularly read through monthly smart reports of my drives. Also, if you have three disks, why not go for RAID5? It is much quicker and I believe you'll end up with more space. It is a bit of a pain to get mdadm to convert your RAID1 to a RAID5, but it is doable. DC
Re: [gentoo-user] Duplicate identical Hard Disk
On Fri, Apr 2, 2010 at 5:02 AM, Dan Cowsill danthe...@gmail.com wrote: On Thu, Apr 1, 2010 at 10:09 PM, Mark Knecht markkne...@gmail.com wrote: With only 2 disks I personally think you're on the right path. With 3 disks I'm personally planning on RAID1 using 3 copies. ... My comment about RAID was that I am learning the hard (alas expensive) way that not all disks can actually do RAID, at least not Linux software RAID, and really be usable. From what I understand of software RAID in linux, it works on block devices, not disks. This means if some endeavoring soul was brave enough to RAID even partitions on a device, it would work as normal. Perhaps you mean that not all properly functioning disks can do RAID? What sort of trouble are you running into? I've successfully deployed both RAID1 and RAID5 on my home media server for quite some time now. While the initial time investment in reading documentation was considerable, since that time I've had no cause for trouble. I keep smartmontools looking at the array member disks and regularly read through monthly smart reports of my drives. Also, if you have three disks, why not go for RAID5? It is much quicker and I believe you'll end up with more space. It is a bit of a pain to get mdadm to convert your RAID1 to a RAID5, but it is doable. DC Good questions: 1) Yes, you can RAID partitions of drives. That's what I'm doing. You can look at the Gentoo RAID/LVM Install guide to see an example of using RAID0 and RAID1 on a single drive. http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml 2) I'm certainly not suggesting RAID doesn't work. It's just not working for me, either due to new motherboard hardware or due to the drives themselves. I'm currently betting it's the drives. The background info, without getting too deeply into it, is that if the drive supports SMART and SMART is enabled, then when doing RAID you need guaranteed Time Limited Error Recovery (TLER) to ensure (I think) that SMART works doesn't get in the way of the drive responding in the appropriate amount of time or else the drive will fall out of the RAID array. Turns out the WD (according to different mailing list and forums I've been looking at) has removed TLER on almost all of their Green drive and some/many/most of the Blue and Black series. They are supporting this in the RE drives though of which I've obtained two. They are smaller and more expensive, but built for RAID, so I'm going to try them out next. http://en.wikipedia.org/wiki/Time-Limited_Error_Recovery 3) As I understand the subject you are correct about size and speed, but a 3-disk RAID5 array can stand 1 disk failing whereas a 3-disk RAID1 array can stand 2 disks failing. For this app (MythTV and seldom used backup server) I don't need speed and size isn't a huge issue so I chose 3-disk RAID1. (Note that the HTPC case I'm using supports up to 3 drives only.) Because multiple drives purchased at the same time generally come from the same production lot there's an additional danger that if one drive fails then one more (or all) could fail at the same time so I'm protecting myself against that. Again, this is very specific to my current needs which is really to back up another machine which will be RAID0 as it needs more disk I/O speed to support 12 processor cores. As always, I'm certainly interested in info and ideas on this subject, most especially now when I'm buying and building. Cheers, Mark
Re: [gentoo-user] Duplicate identical Hard Disk
On 04/02/10 07:59, Mark Knecht wrote: Good questions: 1) Yes, you can RAID partitions of drives. That's what I'm doing. You can look at the Gentoo RAID/LVM Install guide to see an example of using RAID0 and RAID1 on a single drive. http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml 2) I'm certainly not suggesting RAID doesn't work. It's just not working for me, either due to new motherboard hardware or due to the drives themselves. I'm currently betting it's the drives. The background info, without getting too deeply into it, is that if the drive supports SMART and SMART is enabled, then when doing RAID you need guaranteed Time Limited Error Recovery (TLER) to ensure (I think) that SMART works doesn't get in the way of the drive responding in the appropriate amount of time or else the drive will fall out of the RAID array. Turns out the WD (according to different mailing list and forums I've been looking at) has removed TLER on almost all of their Green drive and some/many/most of the Blue and Black series. They are supporting this in the RE drives though of which I've obtained two. They are smaller and more expensive, but built for RAID, so I'm going to try them out next. http://en.wikipedia.org/wiki/Time-Limited_Error_Recovery 3) As I understand the subject you are correct about size and speed, but a 3-disk RAID5 array can stand 1 disk failing whereas a 3-disk RAID1 array can stand 2 disks failing. For this app (MythTV and seldom used backup server) I don't need speed and size isn't a huge issue so I chose 3-disk RAID1. (Note that the HTPC case I'm using supports up to 3 drives only.) Because multiple drives purchased at the same time generally come from the same production lot there's an additional danger that if one drive fails then one more (or all) could fail at the same time so I'm protecting myself against that. Again, this is very specific to my current needs which is really to back up another machine which will be RAID0 as it needs more disk I/O speed to support 12 processor cores. As always, I'm certainly interested in info and ideas on this subject, most especially now when I'm buying and building. Cheers, Mark I'm not even sure if any RAID is a solution for me. My situation is a follow, I've configured Gentoo box for a medical clinic and I'll administer it reportedly via ssh. One server running Windows XP in VirtualBox and some other Linux programs. Server is a quad core ADM and has two identical SATA drives about 600GB There is another smaller box (Intel ATOM CPU) running Gentoo, this box runs Asterisk and VirtualBox as well, it is a server backup. If something happens to mains Server, user just presses Scroll Lock twice (KVM), logs in into smaller and runs the main program from there. So I have a backup in place. I just want to utilize the second drive of the main server. I'm mostly concerns about the problems with emerge, not the hard drive failure (I've plenty of backups). So, I think the best option for me is to just mirror the first HD and modify it to use it a sdb. I'm just making steps I need to do: 1.) Boot from external CD dd if=/dev/sda of=/dev/sdb 2.) modify (add to) grub.conf on sda #title boot sda current title=1st HD sda Kernel Current root (hd0,0) kernel /boot/kernel-current root=/dev/sda3 #title boot sdb current title=2nd HD sdb Kernel Current root (hd1,0) kernel /boot/kernel-current root=/dev/sdb3 3.) Modify fstab Walt has mentioned to use rdev but reading man pages it is only i386, and all my boxes running amd64 (x86_64). What else did I miss. -- Joseph
[gentoo-user] Duplicate identical Hard Disk
I have two identical HD in the box and want to duplicate sda to sdb; sdb is not even partitioned. I think I could do: dd if=/dev/sda of=/dev/sdb but I need to boot from CD isn't it? I would like the second disk to function as an identical bootable backup (just in case something will malfunction after emerge :-/) So I would normally boot from sda but have an ability to boot from sdb, just in case) So after duplicating the disk I would need to modify the grub, fstab on sdb hard disk. I'm not even sure if it will do what I need. What are my other options RAID-1? RAID-1 will not protect me if something happens after emerge. -- Joseph
Re: [gentoo-user] Duplicate identical Hard Disk
On Thu, Apr 1, 2010 at 5:48 PM, Joseph syscon...@gmail.com wrote: I have two identical HD in the box and want to duplicate sda to sdb; sdb is not even partitioned. I think I could do: dd if=/dev/sda of=/dev/sdb but I need to boot from CD isn't it? Yes, basically, boot from USB or CD and use ddrescue to clone it, then edit your fstab and I think you should be good. RAID1 would help if a drive physically dies, but if you had any filesystem corruption or anything you'd just have an identically corrupt copy on the second disk.
Re: [gentoo-user] Duplicate identical Hard Disk
On Thu, Apr 1, 2010 at 4:02 PM, Paul Hartman paul.hartman+gen...@gmail.com wrote: On Thu, Apr 1, 2010 at 5:48 PM, Joseph syscon...@gmail.com wrote: I have two identical HD in the box and want to duplicate sda to sdb; sdb is not even partitioned. I think I could do: dd if=/dev/sda of=/dev/sdb but I need to boot from CD isn't it? Yes, basically, boot from USB or CD and use ddrescue to clone it, then edit your fstab and I think you should be good. RAID1 would help if a drive physically dies, but if you had any filesystem corruption or anything you'd just have an identically corrupt copy on the second disk. A big part of my struggles over the last few days has been with mdadm RAID1. I'm learning that we don't want to send someone down that path unless he has the right sort of disks. I'm having to deal with returns and reordering due to this. People should be aware of what is really required to do RAID before they get started so they don't duplicate my trials. I wasn't and I'm paying for it. (Almost literally if I don't get the drives in the mail!) ;-) Cheers, Mark
Re: [gentoo-user] Duplicate identical Hard Disk
On 04/01/10 17:43, Mark Knecht wrote: On Thu, Apr 1, 2010 at 4:02 PM, Paul Hartman paul.hartman+gen...@gmail.com wrote: On Thu, Apr 1, 2010 at 5:48 PM, Joseph syscon...@gmail.com wrote: I have two identical HD in the box and want to duplicate sda to sdb; sdb is not even partitioned. I think I could do: dd if=/dev/sda of=/dev/sdb but I need to boot from CD isn't it? Yes, basically, boot from USB or CD and use ddrescue to clone it, then edit your fstab and I think you should be good. RAID1 would help if a drive physically dies, but if you had any filesystem corruption or anything you'd just have an identically corrupt copy on the second disk. A big part of my struggles over the last few days has been with mdadm RAID1. I'm learning that we don't want to send someone down that path unless he has the right sort of disks. I'm having to deal with returns and reordering due to this. People should be aware of what is really required to do RAID before they get started so they don't duplicate my trials. I wasn't and I'm paying for it. (Almost literally if I don't get the drives in the mail!) ;-) Cheers, Mark So what you are folks saying is to stay away from RAID-1, beside as Paul mention if I get any corruption and/or configuration (due to ebuild) with RAID I'll be screwed anyhow. So my best option is bootable CD and: dd if=/dev/sda of=/dev/sdb But I'm kind of confused as to how to edit the sdb second drive. I know I'll have to edit at lest: grub.conf and fstab But how? 1.) Both disk are bootable, (have a boot sector) if I disconnect first one sda, I think the second one will be recognize automatically as sda isn't it? 2.) If configure second drive after copying as sdb will it still boot if fist disk is disconnected? -- Joseph
Re: [gentoo-user] Duplicate identical Hard Disk
On Thu, Apr 1, 2010 at 6:40 PM, Joseph syscon...@gmail.com wrote: On 04/01/10 17:43, Mark Knecht wrote: On Thu, Apr 1, 2010 at 4:02 PM, Paul Hartman paul.hartman+gen...@gmail.com wrote: On Thu, Apr 1, 2010 at 5:48 PM, Joseph syscon...@gmail.com wrote: I have two identical HD in the box and want to duplicate sda to sdb; sdb is not even partitioned. I think I could do: dd if=/dev/sda of=/dev/sdb but I need to boot from CD isn't it? Yes, basically, boot from USB or CD and use ddrescue to clone it, then edit your fstab and I think you should be good. RAID1 would help if a drive physically dies, but if you had any filesystem corruption or anything you'd just have an identically corrupt copy on the second disk. A big part of my struggles over the last few days has been with mdadm RAID1. I'm learning that we don't want to send someone down that path unless he has the right sort of disks. I'm having to deal with returns and reordering due to this. People should be aware of what is really required to do RAID before they get started so they don't duplicate my trials. I wasn't and I'm paying for it. (Almost literally if I don't get the drives in the mail!) ;-) Cheers, Mark So what you are folks saying is to stay away from RAID-1, beside as Paul mention if I get any corruption and/or configuration (due to ebuild) with RAID I'll be screwed anyhow. So my best option is bootable CD and: dd if=/dev/sda of=/dev/sdb But I'm kind of confused as to how to edit the sdb second drive. I know I'll have to edit at lest: grub.conf and fstab But how? 1.) Both disk are bootable, (have a boot sector) if I disconnect first one sda, I think the second one will be recognize automatically as sda isn't it? 2.) If configure second drive after copying as sdb will it still boot if fist disk is disconnected? -- Joseph With only 2 disks I personally think you're on the right path. With 3 disks I'm personally planning on RAID1 using 3 copies. If you disconnect the first disk (current sda) then most likely the original second disk (old sdb) becomes the new first disk. (sda) In that case no edits are required. However if you make them both bootable then assuming your BIOS supports it you can tell it to boot from the second disk. If you want the second disk booting to use the second disk's copy of Gentoo then you need to edit things to use sdb, not sda. that's probably overly complicated for what you are trying to do. I say make the copy, then disconnect the first drive physically and give it a try. This all presumes that you comtinue down this path with by hand copies. My comment about RAID was that I am learning the hard (alas expensive) way that not all disks can actually do RAID, at least not Linux software RAID, and really be usable. Cheers, Mark
