On Wed, Sep 30, 2009 at 09:48:13PM -0400, Andrew Cathrow wrote: > so if we're using blacklist exceptions (aka whitelist) then anytime a new lun > is provisioned we'd have to edit the multipath configuration. Is that > something that we're going to automate in ovirt? > > Aic > >
There are a couple of things that we will be looking at to improve this functionality long term. Functionality in this patch is geared toward being able to install to a multipath device. There is currently no automated or scripted method for configuring multipath if you are not installing to a multipath device. I'll be looking at a couple different ways to do this, but haven't settled on a final design yet. Once I get there, I'll submit a high level design for comments. A couple ideas that I'm playing around with: - Breaking off multipath configuration into a separate o-c-multipath script - Having a separate script that a user can run that will whitelist a newly provisioned lun. Mike > ----- "Joey Boggs" <[email protected]> wrote: > > Michael Burns wrote: > > > Also added new kernel command line arg to setup multipath. > > > New menu takes user through mpath whitelisting and choosing an > > > mpath device for installation. > > > > > > Signed-off-by: Michael Burns <[email protected]> > > > --- > > > scripts/ovirt-config-boot | 2 +- > > > scripts/ovirt-config-storage | 188 > > > +++++++++++++++++++++++++++++++++++++++- > > > scripts/ovirt-config-uninstall | 8 ++ > > > scripts/ovirt-early | 16 +++- > > > 4 files changed, 207 insertions(+), 7 deletions(-) > > > > > > diff --git a/scripts/ovirt-config-boot b/scripts/ovirt-config-boot > > > index 87cf832..ed2bb9f 100755 > > > --- a/scripts/ovirt-config-boot > > > +++ b/scripts/ovirt-config-boot > > > @@ -218,7 +218,7 @@ set -e\ > > > cp /sbin/dmsetup.static sbin/dmsetup > > > cp /sbin/kpartx.static sbin/kpartx > > > for M in /sbin/mpath_prio_*.static ; do > > > - cp ${M} $/${M%%.static}; > > > + cp ${M} .${M%%.static}; > > > done > > > fi > > > > > > diff --git a/scripts/ovirt-config-storage b/scripts/ovirt-config-storage > > > index 91662c5..c448aee 100755 > > > --- a/scripts/ovirt-config-storage > > > +++ b/scripts/ovirt-config-storage > > > @@ -130,6 +130,172 @@ check_partition_sizes() > > > return $rc > > > } > > > > > > +is_multipath_disabled() > > > +{ > > > + grep -q "^ *devnode \"\*\"" /etc/multipath.conf > > > +} > > > + > > > +multipath_warn() > > > +{ > > > + sp=' ' > > > + w='!!WARNING' > > > + wb="$w"'!!' > > > + w8="$w$w$w$w$w$w$w$w" > > > + printf '%s!!\n' \ > > > + "$w8" \ > > > + "$w8" \ > > > + "$wb$sp$w" \ > > > + "$wb$sp$w" \ > > > + "$wb Multipath Configurations should only be made $w" \ > > > + "$wb on true multipath devices. Configuring any $w" \ > > > + "$wb other devices may cause problems with your $w" \ > > > + "$wb system. $w" \ > > > + "$wb$sp$w" \ > > > + "$wb Proceeding from this menu will erase all $w" \ > > > + "$wb previous multipath configurations. $w" \ > > > + "$wb$sp$w" \ > > > + "$wb$sp$w" \ > > > + "$w8" \ > > > + "$w8" > > > + > > > +} > > > + > > > +abort_multipath() > > > +{ > > > + printf "Cleaning up multipath configuration\n">&2 > > > + sed -i 's/^\s*wwid \*$/ devnode "*"/' /etc/multipath.conf > > > + sed -i 's/^\s*wwid \w.*$//g' /etc/multipath.conf > > > + service multipathd reload >&2 > > > +} > > > + > > > +config_multipath_conf() > > > +{ > > > + ! is_multipath_disabled && abort_multipath > > > + if [[ $OVIRT_MPATH = "ALL" ]]; then > > > + sed -i 's/^\s*devnode ".*$/ #devnode "*"/' /etc/multipath.conf > > > + else > > > + sed -i 's/^\s*devnode ".*$/ wwid */' /etc/multipath.conf > > > + test $(grep ^blacklist_exceptions /etc/multipath.conf | wc -l) = 0 \ > > > + && printf "blacklist_exceptions {\n}" >> /etc/multipath.conf > > > + local wwid="" > > > + for d in $whitelisted > > > + do > > > + d=$(basename $d) > > > + local scsi_id=$(scsi_id --whitelisted --device=/dev/$d) > > > + if [[ -z $scsi_id ]]; then > > > + echo "scsi_id failed for device /dev/$d. Not white listing." >&2 > > > + else > > > + wwid="${wwid} $scsi_id" > > > + fi > > > + done > > > + wwid=$(echo $wwid | sed 's/ /\n/g' | sort -u) > > > + for scsi_id in $wwid > > > + do > > > + sed -i "s/^blacklist_exceptions {$/blacklist_exceptions {\n wwid > > > $scsi_id"/ /etc/multipath.conf > > > + done > > > + for d in $OVIRT_MPATH > > > + do > > > + sed -i "s/^blacklist_exceptions {$/blacklist_exceptions {\n wwid $d"/ > > > /etc/multipath.conf > > > + > > > + done > > > + fi > > > + service multipathd reload >&2 > > > + start_log > > > + echo "Generated multipath.conf" > > > + cat /etc/multipath.conf > > > + echo "Output of multiatph -v6 command" > > > + multipath -v6 > > > + stop_log > > > + return 0 > > > +} > > > + > > > + > > > +generate_whitelist() > > > +{ > > > + whitelisted="" > > > + local devs=$devices > > > + local PS3="Please select devices to add or remove from multipath > > > whitelist: " > > > + while true; do > > > + local choices="" > > > + for d in $devs > > > + do > > > + get_drive_size $d >&2 > > > + if [[ $whitelisted =~ $d ]]; then > > > + choices="$choices $d-whitelisted" > > > + else > > > + choices="$choices $d" > > > + fi > > > + done > > > + > > > + choices="$choices Done Abort" > > > + select whitelist in $choices > > > + do > > > + test "$whitelist" = Abort && return 1 > > > + test "$whitelist" = Done && return 0 > > > + #first check if it is already whitelisted > > > + #and remove from whitelist > > > + if [[ $whitelist =~ "whitelisted" ]]; then > > > + local tmp_whitelist="" > > > + whitelist=${whitelist%-whitelisted} > > > + for d in $whitelisted > > > + do > > > + test $d != $whitelist && tmp_whitelist="$tmp_whitelist $d" > > > + done > > > + whitelisted=$tmp_whitelist > > > + #else add it to whitelist > > > + else > > > + whitelisted="$whitelisted $whitelist" > > > + fi > > > + break > > > + done > > > + done > > > +} > > > + > > > +# Choose a multipathed device for installation > > > +get_multipath_device() > > > +{ > > > + multipath_warn >&2 > > > + local DRIVE_VAR=$1 > > > + generate_whitelist > > > + test $? = 1 && return 1 > > > + config_multipath_conf > > > + local count=0 > > > + echo "Detecting Multipath Devices..." >&2 > > > + while [ $count -lt 60 ] > > > + do > > > + test ! `ls /dev/mapper/mpath* 2> /dev/null | wc -l` = 0 && break > > > + let count=$count+1 > > > + sleep 1 > > > + done > > > + if [ `ls /dev/mapper/mpath* 2> /dev/null | wc -l` = 0 ]; then > > > + echo "No Multipath devices found. Aborting...">&2 > > > + abort_multipath > > > + return 1 > > > + fi > > > + #need to provide a menu to choose a multipath device > > > + while true > > > + do > > > + local choices="" > > > + for dev in $(ls /dev/mapper/mpath* | egrep -v "p[0-9]+$") > > > + do > > > + choices="$choices $dev" > > > + get_drive_size $dev >&2 > > > + done > > > + choices="$choices Abort" > > > + local PS3="Select Multipath Device: " > > > + select mpath_device in $choices > > > + do > > > + if [[ $mpath_device = Abort ]]; then > > > + abort_multipath > > > + return 1 > > > + elif [[ -e $mpath_device ]]; then > > > + eval $DRIVE_VAR=$mpath_device > > > + return 0 > > > + fi > > > + done > > > + done > > > +} > > > + > > > # Find a usable/selected storage device. > > > # If there are none, give a diagnostic and return nonzero. > > > # If there is just one, e.g., /dev/sda, treat it as selected (see below). > > > @@ -140,6 +306,7 @@ check_partition_sizes() > > > get_dev_name() > > > { > > > local udi_list=$(hal-find-by-capability --capability storage) > > > + local DRIVE_VAR=$1 > > > if test -z "$udi_list"; then > > > warn "ERROR: no usable storage devices detected" > > > return 1 > > > @@ -152,6 +319,9 @@ get_dev_name() > > > local block_dev=$(hal-get-property --udi "$d" --key block.device) > > > # Must start with a '/'. > > > case $block_dev in > > > + '') > > > + #if block.device not defined, suppress warning > > > + continue;; > > > *' '*) > > > # we use space as separator > > > warn "block device name '$block_dev' contains space; skipping"; > > > @@ -189,12 +359,13 @@ get_dev_name() > > > for d in $devices; do > > > get_drive_size $d >&2 > > > done > > > - local choices="$devices Abort" > > > + local choices="$devices Multipath Abort" > > > select device in $choices > > > do > > > test "$device" = Abort && return 1 > > > + test "$device" = "Multipath" && get_multipath_device device; > > > test -z "$device" && continue > > > - echo "$device" > > > + eval $DRIVE_VAR=$device > > > return 0 > > > done > > > } > > > > > > > > @@ -202,7 +373,9 @@ get_dev_name() > > > do_configure() > > > { > > > local name_and_size > > > - DRIVE=$(get_dev_name) || return 0 > > > + get_dev_name DRIVE > > > + test ! -e $DRIVE && echo "Drive '$DRIVE' is not a valid drive" >&2 \ > > > + && return 0 > > > get_drive_size $DRIVE SPACE > > > > > > > > In this section altering DRIVE= kills me trying to use local disks. > > Took me awhile to track this one down. Here's how to fix it: > > > > > > local num_devices=$(echo "$devices" | wc -w) > > # If there's only one device, use it. > > case $num_devices in > > 0) warn "ERROR: found no usable block device"; return 1;; > > 1) echo "$devices"; return 0;; > > *) ;; # found more than one > > esac > > > > > > set > > 1) echo "$devices"; return 0;; > > to > > 1) eval $DRIVE_VAR=$devices; return0;; > > > > > > > > > > > > > printf "\n\nPlease configure storage partitions.\n\n" > > > @@ -537,6 +710,15 @@ > > > CONFIG_SIZE=${OVIRT_VOL_CONFIG_SIZE:-$default_config_size} > > > LOGGING_SIZE=${OVIRT_VOL_LOGGING_SIZE:-$default_logging_size} > > > DATA_SIZE=${OVIRT_VOL_DATA_SIZE:-$default_data_size} > > > > > > +if [ -n "$OVIRT_MPATH" ]; then > > > + #if present then setup multipath.conf with value > > > + config_multipath_conf > > > + while true > > > + do > > > + test ! `ls /dev/mapper/mpath* 2> /dev/null | wc -l` = 0 && break > > > + done > > > +fi > > > + > > > if [ -n "$OVIRT_INIT" ]; then > > > # if present, use the drive selected with 'ovirt_init' boot parameter > > > DRIVE=$OVIRT_INIT > > > diff --git a/scripts/ovirt-config-uninstall > > > b/scripts/ovirt-config-uninstall > > > index fd65cbb..20f8c2c 100755 > > > --- a/scripts/ovirt-config-uninstall > > > +++ b/scripts/ovirt-config-uninstall > > > @@ -47,6 +47,14 @@ if ask_yes_or_no "Do you wish to continue and > > > uninstall this node ([Y]es/[N]o)?" > > > log "Unmounting boot partition" > > > umount $partition > > > log "Removing partitions" > > > + eval $(echo $partition | awk ' { > > > + print "drive=" substr($0,1,length($1)-1); > > > + print "drive2=" substr($0,1,length($1)-2); > > > + }') > > > + if [ ! -e "$drive" ]; then > > > + # e.g. c0d0p1 > > > + drive="$drive2" > > > + fi > > > drive=$(echo $partition | awk '{ print substr($0, 1, length($0) - 1) }') > > > parted -s $drive "rm 1" > > > parted -s $drive "rm 2" > > > diff --git a/scripts/ovirt-early b/scripts/ovirt-early > > > index 8990727..5632fb0 100755 > > > --- a/scripts/ovirt-early > > > +++ b/scripts/ovirt-early > > > @@ -148,6 +148,7 @@ start() { > > > # collectd=server[:port] > > > # hostname=fqdn > > > # TBD logrotate maxsize > > > + # mpath format: mpath=wwid:[wwid] > > > > > > # BOOTIF=link|eth*|<MAC> (appended by pxelinux) > > > # network boot interface is assumed to be on management network where > > > @@ -236,6 +237,12 @@ start() { > > > collectd_server= > > > collectd_port= > > > > > > + # mpath=wwid:[wwid...] > > > + # set a list of comma or colon separated wwids to allow > > > + # for multipath install > > > + # Specify ALL to use all devices > > > + mpath= > > > + > > > # save boot parameters like console= for local disk boot menu > > > bootparams= > > > cat /etc/system-release >> $OVIRT_LOGFILE > > > @@ -286,8 +293,7 @@ start() { > > > =/dev/*) > > > bus= > > > serial= > > > - i=${i#=} > > > - init=$(ls -1 "$i" 2>/dev/null | head -n1) > > > + init=${i#=} > > > ;; > > > *) > > > bus= > > > @@ -374,6 +380,10 @@ start() { > > > console=*) > > > bootparams="$bootparams $i" > > > ;; > > > + mpath=*) > > > + i=${i#mpath=} > > > + mpath=$(echo $i|tr ",:;" " ") > > > + ;; > > > esac > > > done > > > > > > @@ -384,7 +394,7 @@ start() { > > > ip_gateway=$gateway > > > fi > > > # save boot parameters as defaults for ovirt-config-* > > > - params="bootif init vol_boot_size vol_swap_size vol_root_size > > > vol_config_size vol_logging_size vol_data_size local_boot standalone > > > overcommit ip_address ip_netmask ip_gateway ipv6 dns ntp vlan ssh_pwauth > > > syslog_server syslog_port collectd_server collectd_port bootparams > > > hostname firstboot" > > > + params="bootif init vol_boot_size vol_swap_size vol_root_size > > > vol_config_size vol_logging_size vol_data_size local_boot standalone > > > overcommit ip_address ip_netmask ip_gateway ipv6 dns ntp vlan ssh_pwauth > > > syslog_server syslog_port collectd_server collectd_port bootparams > > > hostname firstboot mpath" > > > # mount /config unless firstboot is forced > > > if [ "$firstboot" != "1" ]; then > > > mount_config > > > > > > > _______________________________________________ > > Ovirt-devel mailing list > > [email protected] > > https://www.redhat.com/mailman/listinfo/ovirt-devel > > _______________________________________________ Ovirt-devel mailing list [email protected] https://www.redhat.com/mailman/listinfo/ovirt-devel
