Re: mdadm and UUIDs for its component drives
On Mon, Jun 27, 2011 at 1:59 PM, Philip Hands p...@hands.com wrote: ok, i bring in phil now, who i was talking to yesterday about this. what he said was (and i may get this wrong: it only went in partly) - something along the lines of remember to build the drives with individual mdadm bitmaps enabled. this will save a great deal of arseing about when re-adding drives which didn't get properly added: only 1/2 a 1Tb drive will need syncing, not an entire drive :) the bitmap system he says has hierarchical granularity apparently. What I said was: internal bitmaps ahh. yes. i missed the word internal but heard the good bits, then looked up the man page and went ohh, ok, that must be it. i get there in the end :) also, he recommended taking at least one of the external drives *out* I think I said: WTF? ha ha :) You buy a machine that had 4 hot swap SATA bays, and you're plugging crappy external USB drives into it instead? Are you mental? (or at least, if I didn't say that out loud, that's what I was thinking ;-) i seem to remember the incredulity which definitely had the words are you mental?? behind it I must say that I'm a little beffuddled about how you managed to make the system sensitive to which device contains which MD component -- I seem to remember you mentioning that you had devices listed in your mdadm.conf -- just get rid of them. well, i may have implied that, on account of not being able to express it - i get it now: the things i thought were devices are actually the UUIDs associated with the RAID array... ARRAY /dev/md/2 metadata=1.2 UUID=65c09661:02fc3a16:402916d3:5d4987f4 name=sheikh:2 ... just like this. No mention of devices, which is a good job because that machine seems to randomise the device mapping on each boot, and is capable of moving them about when running if you pop the drive out of the machine and back in again. yehhs, i noticed that. even the bloody boot drive comes up as /dev/sde occasionally. last reboot i was adding drive 4 to the array, it was named /dev/sda. kinda freaky. ok. so. let's have a go at some updating the docs... DESCRIPTION RAID devices are virtual devices created from two or more real block devices. This allows multiple devices (typically disk drives or parti‐ tions thereof) to be combined into a single device to hold (for exam‐ ple) a single filesystem. Some RAID levels include redundancy and so can survive some degree of device failure. Linux Software RAID devices are implemented through the md (Multiple Devices) device driver. UUIDs are used internally through Linux Software RAID to identify any device that is part of a RAID. In this way, names may change but the innocent are protected. ok, scratch that last sentence :) Linux Software RAID devices are implemented through the md (Multiple Devices) device driver. UUIDs are used internally in Linux Software RAID to identify any device that is part of a RAID, thus ensuring that even if the name changes (such as may happen if devices are removed and placed into another system, or if using removable hot-swappable media) Linux RAID can still correctly identify the component devices. can we start with that - what you think, martin? it's right at the top: it spells things out, and it makes linux RAID look good :) i'll try to find appropriate places to put the same info, but the page is really quite long. perhaps on --add somewhere? l. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/capweedxp2gfowv0xj9x6+0bmfi85w8mndr4czgy9xhv7cp1...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
On Mon, Jun 27, 2011 at 3:19 AM, martin f krafft madd...@debian.org wrote: also sprach Tom H tomh0...@gmail.com [2011.06.27.0851 +0200]: Partitions do not have UUIDs. What you are seeing are the MD UUIDs stored in the superblock of the sda1 device. I called them mdadm UUIDs rather than MD UUIDs but they definitely exist, are different from the MD Array UUID, and, AFAIK, unused by the user tools. I misunderstood you. Partitions do not have UUIDs, but you were talking about individual array constituents — those do have UUIDs that are separate from the array UUID. # mdadm --examine /dev/sda2 | grep UUID Array UUID : bfb705a9:69bfc685:92b80aa8:ff445936 Device UUID : ed7cb6d2:32f8dda4:bdd22f74:c4ef720b Exactly (my fault for misusing partition). -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=gvoed6i39afljpri3jyemmu-...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
On Mon, Jun 27, 2011 at 8:59 AM, Philip Hands p...@hands.com wrote: I must say that I'm a little beffuddled about how you managed to make the system sensitive to which device contains which MD component -- I seem to remember you mentioning that you had devices listed in your mdadm.conf -- just get rid of them. I think that you may have resolved the OP's problem. I couldn't understand why the newly-plugged-in USB disk might be considered a member of the array. Having, for example, devices=/dev/sda1,/dev/sdb1 on the ARRAY line would probably do it! Now the question is whether this is the OP's setup. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=zdgaqnhpgkhleuuc0qwba_zr...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
On Mon, Jun 27, 2011 at 12:52 AM, martin f krafft madd...@debian.org wrote: also sprach Tom H tomh0...@gmail.com [2011.06.26.2328 +0200]: mdadm --examine /dev/sda1 returns mdadm UUIDs of the array and the partition. (I've never seen the mdadm UUID of a partition be used for anything. Can an array be assembled by referring to an mdadm UUID of a partition to add a partition? Would it make any sense?!) Partitions do not have UUIDs. What you are seeing are the MD UUIDs stored in the superblock of the sda1 device. I called them mdadm UUIDs rather than MD UUIDs but they definitely exist, are different from the MD Array UUID, and, AFAIK, unused by the user tools. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTim1b7Lr2=onrtnj9npcgb3orve...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
On Mon, Jun 27, 2011 at 1:51 AM, Andrew McGlashan andrew.mcglas...@affinityvision.com.au wrote: Tom H wrote: You have / set up as a RAID 1 array md0 with sda1 and sdb1 as its components. No / would be on an internal drive, right now that is not the concern as it has nothing to do with the external drive array(s) in question for this issue. Thanks for explaining... Forget about /. Your external array is mounted on /path/to/array and its members are sdXA and sdYA. When you plug in another USB drive, it becomes sdXA or sdYA and mdadm tries to assemble it into the array even though it doesn't have any mdadm metadata whatsoever. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTinObgAe22s=1g8lc0vvthoawjq...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
also sprach Tom H tomh0...@gmail.com [2011.06.27.0851 +0200]: Partitions do not have UUIDs. What you are seeing are the MD UUIDs stored in the superblock of the sda1 device. I called them mdadm UUIDs rather than MD UUIDs but they definitely exist, are different from the MD Array UUID, and, AFAIK, unused by the user tools. I misunderstood you. Partitions do not have UUIDs, but you were talking about individual array constituents — those do have UUIDs that are separate from the array UUID. # mdadm --examine /dev/sda2 | grep UUID Array UUID : bfb705a9:69bfc685:92b80aa8:ff445936 Device UUID : ed7cb6d2:32f8dda4:bdd22f74:c4ef720b -- .''`. martin f. krafft madduck@d.o Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduckhttp://vcs-pkg.org `- Debian - when you have better things to do than fixing systems alas, i am dying beyond my means. -- oscar wilde digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
Re: mdadm and UUIDs for its component drives
On Sun, 26 Jun 2011 14:42:03 +0100, Luke Kenneth Casson Leighton
luke.leigh...@gmail.com wrote:
On Sun, Jun 26, 2011 at 2:25 PM, Andrew McGlashan
andrew.mcglas...@affinityvision.com.au wrote:
I hear what you are saying, but I had a related problem which was similar.
well... it's funny, because this is exactly what i need.
Anyway the long and short of it is, I can use mdadm without regard to
what devices are found, such as /dev/sda /dev/sdb /dev/sdc and the like as I
rely purely on the UUID functionality, which as you know, mdadm handles
perfectly well. ;-)
:)
well. that was nice. the scenario you describe is precisely what i
sort-of had planned, but didn't have the expertise to do so was going
to recommend just two drives and then rsync to the other two.
_however_, given that you've solved exactly what is needed / best /
recommended for when you have 4 external drives (which i do) that's
bloody fantastic :)
ok, i bring in phil now, who i was talking to yesterday about this.
what he said was (and i may get this wrong: it only went in partly) -
something along the lines of remember to build the drives with
individual mdadm bitmaps enabled. this will save a great deal of
arseing about when re-adding drives which didn't get properly added:
only 1/2 a 1Tb drive will need syncing, not an entire drive :) the
bitmap system he says has hierarchical granularity apparently.
What I said was: internal bitmaps
madam(8):
-b, --bitmap=
Specify a file to store a write-intent bitmap in. The
file should not exist unless --force is also given. The
same file should be provided when assembling the array.
If the word internal is given, then the bitmap is stored
with the metadata on the array, and so is replicated on
all devices. If the word none is given with --grow mode,
then any bitmap that is present is removed.
To help catch typing errors, the filename must contain at
least one slash ('/') if it is a real file (not 'internal'
or 'none').
Note: external bitmaps are only known to work on ext2 and
ext3. Storing bitmap files on other filesystems may
result in serious problems.
and I probably also gave my half-arsed understanding of what that means.
Feel free to consult actual documentation for a proper understanding of
reality.
also, he recommended taking at least one of the external drives *out*
I think I said: WTF? You buy a machine that had 4 hot swap SATA bays,
and you're plugging crappy external USB drives into it instead? Are you
mental? (or at least, if I didn't say that out loud, that's what I was
thinking ;-)
I must say that I'm a little beffuddled about how you managed to make
the system sensitive to which device contains which MD component -- I
seem to remember you mentioning that you had devices listed in your
mdadm.conf -- just get rid of them.
The mdadm.conf on one of my servers looks like this:
=-=-=-=-
DEVICE partitions
CREATE owner=root group=disk mode=0660 auto=yes
HOMEHOST system
MAILADDR root
ARRAY /dev/md/2 metadata=1.2 UUID=65c09661:02fc3a16:402916d3:5d4987f4
name=sheikh:2
ARRAY /dev/md/3 metadata=1.2 UUID=e82f516b:64bf463c:adf65c9c:fd728d05
name=sheikh:3
ARRAY /dev/md/4 metadata=1.2 UUID=56adc7ca:c7097e9b:00ac12c0:d1d278f2
name=sheikh:4
ARRAY /dev/md/5 metadata=1.2 UUID=6c5362e4:74b56fad:8c74a317:e4dce6d0
name=sheikh:5
ARRAY /dev/md/6 metadata=1.2 UUID=99ed31bd:cc608687:76f7b5a3:7bca24bc
name=sheikh:6
ARRAY /dev/md/7 metadata=1.2 UUID=87cdaf12:94c2a356:4ba1d3bd:c80ac3b3
name=sheikh:7
ARRAY /dev/md/8 metadata=1.2 UUID=08e708b8:0989607b:d99709d2:8b5e4d58
name=sheikh:8
ARRAY /dev/md/11 metadata=1.2 UUID=210e1b53:3937b017:c947361e:2d2884b1
name=sheikh:11
=-=-=-=-
No mention of devices, which is a good job because that machine seems
to randomise the device mapping on each boot, and is capable of moving
them about when running if you pop the drive out of the machine and back
in again.
As also mentioned somewhere in the docs, the output of the command:
mdadm --examine --scan
can be used to populate the relevant bits of mdadm.conf
Cheers, Phil.
--
|)| Philip Hands [+44 (0)20 8530 9560]http://www.hands.com/
|-| HANDS.COM Ltd.http://www.uk.debian.org/
|(| 10 Onslow Gardens, South Woodford, London E18 1NE ENGLAND
pgpufrVxAMzIe.pgp
Description: PGP signature
mdadm and UUIDs for its component drives
at martin's request i'm forwarding this to debian-user so that it can be found for archival purposes and general discussion. this is the context: a follow-up question will be sent, without all the crap below. l. [original] allo martin, haven't spoken to you for a while. got an interesting feature request / bug in mdadm to report. here's a bit of background, a lovely ubuntu user trying to tell the world he's got it right, when it's all quite likely to be spectacularly... inept and ineffective :) http://ubuntuforums.org/archive/index.php/t-582775.html i have set up a system which has WD external USB drives: they're RAID-1 mirrored. the idea is to keep the temperature of the machine down, by having the data drives external: that way, the main fan doesn't fire up and it's all nice and quiet. the issue is this: if put in, say, a USB memory stick in first, and it pops up as /dev/sdc, and _then_ put in the two USB external drives, what happens to mdadm? it sees /dev/sdc as being, instead of one of the RAID drives, as a USB memory stick! from the above thread (and i can confirm it - i've tried): mdadm --manage /dev/md1 --add /dev/disk/by-id/usb-WD_Ext_HDD_1021_574341565933303838393734-0:0 mdadm i can confirm goes and hunts down the symlinks and adds /dev/sdd! i don't _want_ it to add /dev/sdd, i want it to add /dev/disk/by-id/usb-WD_blah_blah :) question is: how? or, does it not matter: does mdadm use UUIDs internally? tia, [martin's reply] On Sat, Jun 25, 2011 at 8:26 PM, martin f krafft madd...@debian.org wrote: also sprach Luke Kenneth Casson Leighton luke.leigh...@gmail.com [2011.06.25.1938 +0200]: mdadm i can confirm goes and hunts down the symlinks and adds /dev/sdd! i don't _want_ it to add /dev/sdd, i want it to add /dev/disk/by-id/usb-WD_blah_blah :) question is: how? or, does it not matter: does mdadm use UUIDs internally? Yes, it uses UUIDs. The problem you describe should not happen. Please direct your questions to debian-user@lists.debian.org so that others can profit from this discussion. -- .''`. martin f. krafft madduck@d.o Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduck http://vcs-pkg.org `- Debian - when you have better things to do than fixing systems without a god, life is only a matter of opinion. -- douglas adams -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=r-cA1TSwkGk=qq1t0anxmzdq...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
moorning martin: thanks for responding. apologies for not thinking to ask on debian-user earlier, and apologies for the long-winded style: just got ddragged out of bed to go chase a lamb out of the garden that was eating our flowers and vegetables. if i wasn't stumbling about half-asleep or concerned for our future food supply i'd find a lamb head-butting fence posts incredibly funny. On Sat, Jun 25, 2011 at 8:26 PM, martin f krafft madd...@debian.org wrote: also sprach Luke Kenneth Casson Leighton luke.leigh...@gmail.com [2011.06.25.1938 +0200]: mdadm i can confirm goes and hunts down the symlinks and adds /dev/sdd! i don't _want_ it to add /dev/sdd, i want it to add /dev/disk/by-id/usb-WD_blah_blah :) question is: how? or, does it not matter: does mdadm use UUIDs internally? Yes, it uses UUIDs. The problem you describe should not happen. ok, so the question therefore morphs into a long-winded self-answering one: what is it about mdadm that causes people not to be aware that UUIDs are used internally, such that they invest quite a bit of time to e.g. modify their udev rules in /etc/ (rather than add alternatives to /usr/local) and thus make their lives more awkward for future upgrades, and search for solutions such as endeavouring to use --add /dev/disk/by-id/XXX? the answer is that mdadm tracks down the hardlink and displays, as best i can tell, only that, with no immediately obvious options to get it to display the disk UUIDs. sooo here's some further questions: * is there an option to mdadm to make it display UUIDs instead of or as well as the disk name? * if not, would adding one be a good idea? * also, how about making mention of how mdadm works, in the man page somewhere reaaasonably prominently? the basic gist is that mdadm is a fantastic tool, does a far better job than people believe or understand it to be doing, protects them from themselves and any lack of knowledge of its inner workings, but that means that unfortunately it's under-promoted and in danger of being ignored (or worse, NIH-rewritten!) i would hate to see a better tool being written which has, at the very top of its home page, and in all freshmeat and sourceforget prominent short descriptions, yeah! we're l33t! our software RAID tool uses UUIDs, which makes it better than mdadm. we r0ck! :) l. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTikbVC9ZsXT4SF-Lb3=gk2nqlmx...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
Hi Luke,
Luke Kenneth Casson Leighton wrote:
the answer is that mdadm tracks down the hardlink and displays, as
best i can tell, only that, with no immediately obvious options to get
it to display the disk UUIDs.
I hear what you are saying, but I had a related problem which was similar.
When starting up a machine with two external 2TB drives which had been
set up as a mirror, it would sometimes only find one drive and then it
would happily mount the RAID1 array in a degraded state. Then, when the
other drive was added in, it had to do a rebuild of the array. It's not
much good having to rebuild the array after each boot when the mirror
should be perfectly fine.
So I solved it by adding the following to my /etc/rc.local
nohup /usr/local/bin/u1-mirror-drive.sh 21 /dev/null
Note that external mirror drive, which mounts at /u1, has this
/etc/fstab entry:
/dev/mapper/vg--external--1-vg--external--1--u1 /u1 ext4
noauto,rw 0 0
I've masked the UUID below, but I don't see how it could cause any
trouble if I did not do that.
The RAID1 is identified ONLY with UUID in /etc/mdadm/mdadm.conf
# grep ARRAY /etc/mdadm/mdadm.conf
ARRAY /dev/md0 UUID=06fd3d46----
Here's my script that handles getting everything working after a boot:
# cat /usr/local/bin/u1-mirror-drive.sh
#!/bin/bash
RAID_DRIVE_ID=06fd3d46----
RAID_DRIVES=2
TIME_LIMIT=60
echo RAID Drive ID: $RAID_DRIVE_ID
echo Number of Devices required: $RAID_DRIVES
echo Time Limit: $TIME_LIMIT
function error_drive_missing ()
{
echo
echo -en \aMissing drive(s) ... cannot assemble /dev/md0\n\n
/sbin/blkid|/bin/grep $RAID_DRIVE_ID
exit
}
(
echo ==
echo -en Waiting for $RAID_DRIVES drives to be visible for
\linux_raid_member(s)\ with blkid of: \$RAID_DRIVE_ID\ ... \n\t
CNT=1
echo -en 00
while [ $(/sbin/blkid|/bin/grep $RAID_DRIVE_ID|/usr/bin/wc -l) -lt
$RAID_DRIVES ]
do
if [ $CNT -lt 10 ]; then echo -en \b$CNT; else echo -en
\b\b$CNT;fi
if [ $CNT -eq $TIME_LIMIT ]; then echo -en \b\b$TIME_LIMIT
seconds \n;error_drive_missing;fi
CNT=$(($CNT + 1))
/bin/sleep 1
done
echo -en \n\nAll required drives found in $CNT seconds\n
echo ==
echo -en \n\n
cmds='/sbin/mdadm --assemble /dev/md0~
/sbin/vgscan~
/sbin/vgchange -ay vg-external-1~
/bin/mount /u1~
/bin/mount~
/bin/df -Th~
/bin/date~
/sbin/mdadm -D /dev/md0~
/bin/cat /proc/mdstat'
echo ..
IFS='~'
echo $cmds|
while read cmd
do
IFS=$' \t\n'
echo ==
echo ${cmd}
echo --
$cmd
echo -en ==\n\n
done
echo ..
) 21 | /usr/bin/tee /var/log/md0-vg-external-1-u1-wrk.$(date
+%Y%m%d%H%M).out
Right now, I am using 2x 2TB drives as mirrors, I plan to add a 3rd
drive as a 3-way mirror and to let it sync up, then remove the drive for
off-site storage. A 4th drive will come into play as well to rotate
off-site storage. Consequently, I catered for that scenario in the
mount script above -- ie I can easily change the number of RAID
devices to find before continuing if I choose to have more online or not
during boot. The script gives up if it cannot find the required number
of devices within 60 seconds, then I will have to manually intervene.
As you can see from the script, there is some logging taking place so
that I can check things over if necessary.
I may end up using multiple external mirrors at some stage; if I do that
then I'll likely have duplicated scripts for each metadevice and the
scripts will be [slightly] modified as required. I may end up with a
parameter file with a single script, but it's probably not worth the
further effort. Although using command line variables would be an easy
and viable option.
Anyway the long and short of it is, I can use mdadm without regard
to what devices are found, such as /dev/sda /dev/sdb /dev/sdc and the
like as I rely purely on the UUID functionality, which as you know,
mdadm handles perfectly well. ;-)
--
Kind Regards
AndrewM
Andrew McGlashan
Broadband Solutions now including VoIP
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4e07335a.4010...@affinityvision.com.au
Re: mdadm and UUIDs for its component drives
On Sun, Jun 26, 2011 at 2:25 PM, Andrew McGlashan andrew.mcglas...@affinityvision.com.au wrote: I hear what you are saying, but I had a related problem which was similar. well... it's funny, because this is exactly what i need. Anyway the long and short of it is, I can use mdadm without regard to what devices are found, such as /dev/sda /dev/sdb /dev/sdc and the like as I rely purely on the UUID functionality, which as you know, mdadm handles perfectly well. ;-) :) well. that was nice. the scenario you describe is precisely what i sort-of had planned, but didn't have the expertise to do so was going to recommend just two drives and then rsync to the other two. _however_, given that you've solved exactly what is needed / best / recommended for when you have 4 external drives (which i do) that's bloody fantastic :) ok, i bring in phil now, who i was talking to yesterday about this. what he said was (and i may get this wrong: it only went in partly) - something along the lines of remember to build the drives with individual mdadm bitmaps enabled. this will save a great deal of arseing about when re-adding drives which didn't get properly added: only 1/2 a 1Tb drive will need syncing, not an entire drive :) the bitmap system he says has hierarchical granularity apparently. also, he recommended taking at least one of the external drives *out* of its external box and making it *internal*. the reason for that is that a) if the drives ever get powered down accidentally (e.g. by cleaning ladies) you're fd b) if you move a drive or two internally, it's possible to prioritise those drives as reading ones, and to make the external ones write priority. so, the data gets read from the internal one, and changes get propagated to all drives. ... thoughts? l. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=9WA9ayQ8sioeBsaocqJBigN+=7...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
also sprach Luke Kenneth Casson Leighton luke.leigh...@gmail.com [2011.06.26.1241 +0200]: * is there an option to mdadm to make it display UUIDs instead of or as well as the disk name? mdadm -Es * also, how about making mention of how mdadm works, in the man page somewhere reaaasonably prominently? Search manpage for partitions. Please suggest patches if you find the information insufficient. -- .''`. martin f. krafft madduck@d.o Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduckhttp://vcs-pkg.org `- Debian - when you have better things to do than fixing systems this product is under strict quality contril with perfect packing and quality when leving the factory.please keep away from damp.high temperature or sun expose.If found any detectives when purchasing. please return the productby airmail to our administration section and inform the time, place.and store of this purchase for our improvement.We shall give you a satisfactory reply.Thanks for your patronage and welcome your comments. -- http://www.engrish.com digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
Re: mdadm and UUIDs for its component drives
On Sun, Jun 26, 2011 at 2:25 PM, Andrew McGlashan andrew.mcglas...@affinityvision.com.au wrote: Anyway the long and short of it is, I can use mdadm without regard to what devices are found, such as /dev/sda /dev/sdb /dev/sdc and the like as I rely purely on the UUID functionality, which as you know, mdadm handles perfectly well. ;-) :) ... you see, this is the bit that has me concerned. /dev/mdN can be referred to by its unique UUID, but that's *not* what i'm referring to. and, from what you're saying, you appear to be implying that yes, the external drives can pop up as /dev/sda through /dev/sdc and be confused - and thus it is pure luck (or actually design) that the drives *happen* to all be part of the same identical RAID-1 mirroring array. so i realise martin that you've already answered, but it would be really good if you could explicitly confirm: yes, mdadm names its RAID drives by UUID (as can clearly be seen in /dev/mdadm/mdadm.conf) but does it *also* refer to its *COMPONENT* drives (internally, and non-obviously, and undocumentedly) by UUID and then report to the outside world that it's using whatever name (/dev/sdX) which can, under these external-drives scenario, change. l. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=tzm+p-tgq1hjo6khljg_ufm9...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
On Sun, Jun 26, 2011 at 3:11 PM, martin f krafft madd...@debian.org wrote: also sprach Luke Kenneth Casson Leighton luke.leigh...@gmail.com [2011.06.26.1241 +0200]: * is there an option to mdadm to make it display UUIDs instead of or as well as the disk name? mdadm -Es oo! yaay! there is, however, no mention of the fact that these options display UUIDs, and, confusingly, -s is listed as only working with the -R option... oh wait, that's for Incremental Assembly mode (eek!) ok, so -s (or --scan), scan /proc/mdstat, and -E for show components. * also, how about making mention of how mdadm works, in the man page somewhere reaaasonably prominently? Search manpage for partitions. that's odd. i read around each part (man mdadm^M /partitions^M), paragraph back and forwards: no mention of the UUIDs of drive components of an array was clearly evident. Please suggest patches if you find the information insufficient. ok. feeling slightly overwhelmed by the task, my lack of knowledge on the detailed workings of mdadm somewhat getting in the way, but i'll do my best. l. -- .''`. martin f. krafft madduck@d.o Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduck http://vcs-pkg.org `- Debian - when you have better things to do than fixing systems this product is under strict quality contril with perfect packing and quality when leving the factory.please keep away from damp.high temperature or sun expose.If found any detectives when purchasing. please return the productby airmail to our administration section and inform the time, place.and store of this purchase for our improvement.We shall give you a satisfactory reply.Thanks for your patronage and welcome your comments. -- http://www.engrish.com -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTinX=y6pxcpb2+csk8un6llv31r...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
Luke Kenneth Casson Leighton wrote: yes, mdadm names its RAID drives by UUID (as can clearly be seen in /dev/mdadm/mdadm.conf) but does it *also* refer to its *COMPONENT* drives (internally, and non-obviously, and undocumentedly) by UUID and then report to the outside world that it's using whatever name (/dev/sdX) which can, under these external-drives scenario, change. l. The other thing is that both drives in the array have the same UUID, so you need to be able to tell them apart some way or another and the /dev/sd* view is just fine. And this works fine too fwiw : mdadm -D /dev/disk/by-id/md-uuid-* So long as mdadm can determine the drives in use, I don't care how it uses them internally. However, if a drive goes bad, then I need to know which one. Let's say that /dev/sda has gone bad of a two drive RAID1 array; I can visually detect the drive by doing the following: dd if=/dev/sda of=/dev/null Go look to see which drive is busy [hopefully it will show with a flashing activity LED] and I can see which one has failed -- if that doesn't work, then I can reverse the test and try all drives that are meant to be okay to eliminate them. -- Kind Regards AndrewM Andrew McGlashan Broadband Solutions now including VoIP -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e074a21.6000...@affinityvision.com.au
Re: mdadm and UUIDs for its component drives
also sprach Luke Kenneth Casson Leighton luke.leigh...@gmail.com [2011.06.26.1634 +0200]: Search manpage for partitions. that's odd. i read around each part (man mdadm^M /partitions^M), paragraph back and forwards: no mention of the UUIDs of drive components of an array was clearly evident. I was not trying to suggest that there was a mention of the UUIDs. mdadm's manpage only mentions /proc/partitions; it scans that file and then looks for UUIDs on each of the listed partitions, building a list indexed by UUID [0]. This is called scanning. And now it has the list of devices (partitions) that constitute individual arrays identified by UUID… 0. not sure this is the actual implementation… Please suggest patches if you find the information insufficient. ok. feeling slightly overwhelmed by the task, my lack of knowledge on the detailed workings of mdadm somewhat getting in the way, but i'll do my best. I'll try my best to provide feedback. Thanks, -- .''`. martin f. krafft madduck@d.o Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduckhttp://vcs-pkg.org `- Debian - when you have better things to do than fixing systems no, 'eureka' is greek for 'this bath is too hot.' -- dr. who digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
Re: mdadm and UUIDs for its component drives
On 06/27/11 at 01:02am, Andrew McGlashan wrote: Luke Kenneth Casson Leighton wrote: yes, mdadm names its RAID drives by UUID (as can clearly be seen in /dev/mdadm/mdadm.conf) but does it *also* refer to its *COMPONENT* drives (internally, and non-obviously, and undocumentedly) by UUID and then report to the outside world that it's using whatever name (/dev/sdX) which can, under these external-drives scenario, change. l. Let's say that /dev/sda has gone bad of a two drive RAID1 array; I can visually detect the drive by doing the following: dd if=/dev/sda of=/dev/null Go look to see which drive is busy [hopefully it will show with a flashing activity LED] and I can see which one has failed -- if that doesn't work, then I can reverse the test and try all drives that are meant to be okay to eliminate them. It seems to me that you'd be well served by simply using the UUID (by-uuid, not by-id) in all things, including mounting and managing. Then you would never need to figure out which disk sda was, you could just figure out which disk the UUID was (and you'd only have to learn it once). -- Liam signature.asc Description: Digital signature
Re: mdadm and UUIDs for its component drives
On Sun, Jun 26, 2011 at 4:26 PM, martin f krafft madd...@debian.org wrote: also sprach Luke Kenneth Casson Leighton luke.leigh...@gmail.com [2011.06.26.1634 +0200]: Search manpage for partitions. that's odd. i read around each part (man mdadm^M /partitions^M), paragraph back and forwards: no mention of the UUIDs of drive components of an array was clearly evident. I was not trying to suggest that there was a mention of the UUIDs. mdadm's manpage only mentions /proc/partitions; it scans that file and then looks for UUIDs on each of the listed partitions, building a list indexed by UUID [0]. ahh, ok. that's very cool. And now it has the list of devices (partitions) that constitute individual arrays identified by UUID… 0. not sure this is the actual implementation… :) Please suggest patches if you find the information insufficient. ok. feeling slightly overwhelmed by the task, my lack of knowledge on the detailed workings of mdadm somewhat getting in the way, but i'll do my best. I'll try my best to provide feedback. thx. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/banlktinnaoudu409w1+evy+adrv7-sz...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
Hi, Luke Kenneth Casson Leighton wrote: well. that was nice. the scenario you describe is precisely what i sort-of had planned, but didn't have the expertise to do so was going to recommend just two drives and then rsync to the other two. _however_, given that you've solved exactly what is needed / best / recommended for when you have 4 external drives (which i do) that's bloody fantastic :) Great, hope it helps! ok, i bring in phil now, who i was talking to yesterday about this. what he said was (and i may get this wrong: it only went in partly) - something along the lines of remember to build the drives with individual mdadm bitmaps enabled. this will save a great deal of arseing about when re-adding drives which didn't get properly added: only 1/2 a 1Tb drive will need syncing, not an entire drive :) the bitmap system he says has hierarchical granularity apparently. I just added a bitmap, more info. here [1] [2] -- worth a read. You can add and remove them, sometimes you must remove them ... for instance, if growing the array. Quote from [2] reference: In some configurations you may not be able to grow the array until you have removed the internal bitmap. You can add the bitmap back again after the array has been grown. It also seems best to use internal for the bitmap from my reading, so this is what I did for that.tip: # mdadm --grow --bitmap=internal /dev/md0 also, he recommended taking at least one of the external drives *out* of its external box and making it *internal*. the reason for that is that a) if the drives ever get powered down accidentally (e.g. by cleaning ladies) you're fd b) if you move a drive or two internally, it's possible to prioritise those drives as reading ones, and to make the external ones write priority. so, the data gets read from the internal one, and changes get propagated to all drives. Yes, that sounds like a good idea(tm) ... worth considering, but right now I prefer all the RAID drive members external on this particular machine. The drives and machine are all on a suitable UPS and there is no cleaning lady to worry about -- my wife won't touch the drives either. Down the track, I'm sure to move to USB 3.0 and maybe even further down the track to external PCI Express ... [3], which looks interesting. And more so in the future, IBM Racetrack memory [4] which looks to replace the drive as we know it today. [1] https://raid.wiki.kernel.org/index.php/Write-intent_bitmap [2] http://en.wikipedia.org/wiki/Mdadm [3] http://www.zdnet.com/blog/computers/look-out-thunderbolt-external-pci-express-spec-being-developed/6220?tag=nl.e539 [4] http://www.youtube.com/watch?v=q5jRHZWQ0sc -- Kind Regards AndrewM -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e0774de.8070...@affinityvision.com.au
Re: mdadm and UUIDs for its component drives
On Sun, Jun 26, 2011 at 6:41 AM, Luke Kenneth Casson Leighton luke.leigh...@gmail.com wrote: is there an option to mdadm to make it display UUIDs instead of or as well as the disk name? mdadm --examine /dev/sdXY gives you the device and the array UUIDs. mdadm --examine --scan gives you the array UUID(s). mdadm --detail /dev/mdZ gives you the array UUID. mdadm --detail --scan gives you the array UUID(s). -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=pt6m41iwnquyvoc5yuv1agmk...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
On Sun, Jun 26, 2011 at 9:55 AM, Luke Kenneth Casson Leighton luke.leigh...@gmail.com wrote: On Sun, Jun 26, 2011 at 2:25 PM, Andrew McGlashan andrew.mcglas...@affinityvision.com.au wrote: Anyway the long and short of it is, I can use mdadm without regard to what devices are found, such as /dev/sda /dev/sdb /dev/sdc and the like as I rely purely on the UUID functionality, which as you know, mdadm handles perfectly well. ;-) :) ... you see, this is the bit that has me concerned. /dev/mdN can be referred to by its unique UUID, but that's *not* what i'm referring to. and, from what you're saying, you appear to be implying that yes, the external drives can pop up as /dev/sda through /dev/sdc and be confused - and thus it is pure luck (or actually design) that the drives *happen* to all be part of the same identical RAID-1 mirroring array. so i realise martin that you've already answered, but it would be really good if you could explicitly confirm: yes, mdadm names its RAID drives by UUID (as can clearly be seen in /dev/mdadm/mdadm.conf) but does it *also* refer to its *COMPONENT* drives (internally, and non-obviously, and undocumentedly) by UUID and then report to the outside world that it's using whatever name (/dev/sdX) which can, under these external-drives scenario, change. I've never plugged a USB drive into a mdadm'd box so I'm trying to get my head around this - and check whether I'm understanding you correctly. You have / set up as a RAID 1 array md0 with sda1 and sdb1 as its components. You plug in a USB drive and its first/only partition become sda1. mdadm tries to assemble the USB drive's sda1 with sdb1 (whether it's the old sda1 or the old sdb1) even though sda1 doesn't have md0's (array) UUID in its metadata, let alone any mdadm metadata! -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTikSkfmyc1vLkD+cd=jjkyvxq8r...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
On Sun, Jun 26, 2011 at 11:29 AM, William Hopkins we.hopk...@gmail.com wrote: It seems to me that you'd be well served by simply using the UUID (by-uuid, not by-id) in all things, including mounting and managing. Then you would never need to figure out which disk sda was, you could just figure out which disk the UUID was (and you'd only have to learn it once). There are UUIDs and UUIDs. For an array md0 with sda1 and sdb1 as its components. blkid /dev/md0 returns the filesystem UUID. mdadm --detail /dev/md0 returns the mdadm UUID of the array. blkid /dev/sda1 returns the mdadm UUID of the array. mdadm --examine /dev/sda1 returns mdadm UUIDs of the array and the partition. (I've never seen the mdadm UUID of a partition be used for anything. Can an array be assembled by referring to an mdadm UUID of a partition to add a partition? Would it make any sense?!) The /dev/disk/by-id/ symlinks are the most stable ones (for a specific disk) should anyone want to use them because they're hardware IDs. I don't have an mdadm'd box at hand to check but I think that md0's entry in this directory includes the mdadm array UUID of md0 because md0 doesn't have a real hardware ID. So, for md0, /dev/disk/by-id and /dev/disk/by-uuid are equivalent. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=_s+tls+x84q-n_4ewn2mwafv...@mail.gmail.com
Re: mdadm and UUIDs for its component drives
also sprach Tom H tomh0...@gmail.com [2011.06.26.2328 +0200]: mdadm --examine /dev/sda1 returns mdadm UUIDs of the array and the partition. (I've never seen the mdadm UUID of a partition be used for anything. Can an array be assembled by referring to an mdadm UUID of a partition to add a partition? Would it make any sense?!) Partitions do not have UUIDs. What you are seeing are the MD UUIDs stored in the superblock of the sda1 device. -- .''`. martin f. krafft madduck@d.o Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduckhttp://vcs-pkg.org `- Debian - when you have better things to do than fixing systems a woman is like your shadow; follow her, she flies; fly from her, she follows. -- sébastien-roch-nicolas chamfort digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
Re: mdadm and UUIDs for its component drives
Tom H wrote: You have / set up as a RAID 1 array md0 with sda1 and sdb1 as its components. No / would be on an internal drive, right now that is not the concern as it has nothing to do with the external drive array(s) in question for this issue. -- Kind Regards AndrewM -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e081a5e.5060...@affinityvision.com.au
