Re: multipath iSCSI installs
Hannes Reinecke [h...@suse.de] wrote: > mala...@us.ibm.com wrote: > > Hi all, > > I also tried installing SLES10 and SLES11. I believe they recommend > > installing on a single path and then converting to multipath. I have > > found that SLES11's initrd image can only find one path even after > > including "multipath" support into the initrd image. It creates a > > dm-multipath device with a single path and later converts to > > dm-multipath device with two paths later when it runs scripts in > > /etc/init.d. SLES10's behavior might be same, but I didn't analyse. Does > > anyone know if SLES11's initrd image can find more than one iSCSI path? > > > Yes, I do :-) > > I have patched the SUSE initrd to activate all network interfaces which > are configured via ifup in the running system. > > You'd need the attached patch to /lib/mkinitrd to achieve this. I am using only one NIC. I have dual ported NetApp LUNs. Does the attached patch help in my case? I don't think so, but I wanted to ask anyway! Thanks, Malahal. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: multipath iSCSI installs
Hannes Reinecke wrote: > mala...@us.ibm.com wrote: >> Hi all, >> >> I am trying to install RHEL5.3 on an iSCSI disk with two paths. >> I booted with "mapth" option but the installer picked up only a single >> path. Is this the expected behavior when I use "iBFT"? >> >> The install went fine on a single path. I was trying to convert the >> single path to multi-path by running "mkinitrd". RHEL was unable to boot >> (panics) with the new "initrd" image. The only difference between the >> old initrd image and the new one is that the old initrd image was using >> iBFT method and the new image was trying to use the values from the >> existing session(s) at "initrd" creation time. For some reason the >> latter method doesn't work. Is this a known bug? >> >> I also tried installing SLES10 and SLES11. I believe they recommend >> installing on a single path and then converting to multipath. I have >> found that SLES11's initrd image can only find one path even after >> including "multipath" support into the initrd image. It creates a >> dm-multipath device with a single path and later converts to >> dm-multipath device with two paths later when it runs scripts in >> /etc/init.d. SLES10's behavior might be same, but I didn't analyse. Does >> anyone know if SLES11's initrd image can find more than one iSCSI path? >> > Yes, I do :-) > > I have patched the SUSE initrd to activate all network interfaces which > are configured via ifup in the running system. > > You'd need the attached patch to /lib/mkinitrd to achieve this. > I am not sure if it will help, but if you wanted to try and just activate the nics that were set in the ibft info then you can run "iscsiadm -m fw" and it will print out all the targets in ibft and the info for the nic that is associated with the target. I think that might be helpful if a user were to swap nics between boots. You could parse the iscsiadm -m fw info and start the nics in there using the info that is spit out. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: multipath iSCSI installs
mala...@us.ibm.com wrote: > Hi all, > > I am trying to install RHEL5.3 on an iSCSI disk with two paths. > I booted with "mapth" option but the installer picked up only a single > path. Is this the expected behavior when I use "iBFT"? > > The install went fine on a single path. I was trying to convert the > single path to multi-path by running "mkinitrd". RHEL was unable to boot > (panics) with the new "initrd" image. The only difference between the > old initrd image and the new one is that the old initrd image was using > iBFT method and the new image was trying to use the values from the > existing session(s) at "initrd" creation time. For some reason the > latter method doesn't work. Is this a known bug? > > I also tried installing SLES10 and SLES11. I believe they recommend > installing on a single path and then converting to multipath. I have > found that SLES11's initrd image can only find one path even after > including "multipath" support into the initrd image. It creates a > dm-multipath device with a single path and later converts to > dm-multipath device with two paths later when it runs scripts in > /etc/init.d. SLES10's behavior might be same, but I didn't analyse. Does > anyone know if SLES11's initrd image can find more than one iSCSI path? > Yes, I do :-) I have patched the SUSE initrd to activate all network interfaces which are configured via ifup in the running system. You'd need the attached patch to /lib/mkinitrd to achieve this. Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage h...@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~--- diff --git a/scripts/boot-network.sh b/scripts/boot-network.sh index ac771e0..5c1bfad 100644 --- a/scripts/boot-network.sh +++ b/scripts/boot-network.sh @@ -4,7 +4,7 @@ # dhcpcd reqires the af_packet module #%modules: af_packet $bonding_module #%udevmodules: $drvlink -#%if: "$interface" -o "$dhcp" -o "$ip" -o "$nfsaddrs" +#%if: "$interface" -o "$dhcp" -o "$ip" -o "$nfsaddrs" -o "$drvlink" # # network initialization ## @@ -148,5 +148,12 @@ elif [ "$nettype" = "static" ]; then INTERFACE="${ip%%:*}" fi echo 'hosts: files dns' > /etc/nsswitch.conf +elif [ "$nettype" = "ifup" ] ; then +for i in /etc/sysconfig/network/ifcfg-* ; do + interface=${i##*/ifcfg-} + [ -d /sys/class/net/$interface/device ] || continue + [ "$interface" = "lo" ] && continue + ifup $interface +done fi diff --git a/scripts/setup-network.sh b/scripts/setup-network.sh index 031774c..b212f2b 100644 --- a/scripts/setup-network.sh +++ b/scripts/setup-network.sh @@ -159,13 +159,17 @@ get_network_module() echo "$drvlink" } -if [ -z "$interface" ] ; then -for addfeature in $ADDITIONAL_FEATURES; do -if [ "$addfeature" = "network" ]; then +for addfeature in $ADDITIONAL_FEATURES; do +if [ "$addfeature" = "network" ]; then + if [ -z "$interface" ] ; then interface=default fi -done -fi +fi +if [ "$addfeature" = "ifup" ] ; then + nettype=ifup + interface= +fi +done ip= interface=${interface#/dev/} @@ -215,6 +219,21 @@ if [ -n "$interface" ] ; then fi fi +# Copy ifcfg settings +if [ "$nettype" = "ifup" ] ; then +mkdir -p $tmp_mnt/etc/sysconfig +cp -rp /etc/sysconfig/network $tmp_mnt/etc/sysconfig +for i in /etc/sysconfig/network/ifcfg-*; do + interface=${i##*/ifcfg-} + if [ -d /sys/class/net/$interface/device ] ; then + mod=$(get_network_module $interface) + drvlink="$drvlink $mod" + verbose "[NETWORK]\t$interface ($nettype)" + fi +done +interface= +fi + # Copy the /etc/resolv.conf when the IP is static if [ "$interface" -a "$nettype" = "static" -a -f /etc/resolv.conf ] ; then verbose "[NETWORK]\tUsing /etc/resolv.conf from the system in the initrd" @@ -236,6 +255,14 @@ mkdir -p $tmp_mnt/var/run cp_bin /lib/mkinitrd/bin/ipconfig.sh $tmp_mnt/bin/ipconfig if [ -f /etc/udev/rules.d/70-persistent-net.rules ] ; then cp /etc/udev/rules.d/70-persistent-net.rules $tmp_mnt/etc/udev/rules.d +if [ "$nettype" = "ifup" ] ; then + cp /etc/udev/rules.d/77-network.rules $tmp_mnt/etc/udev/rules.d + cp_bin /sbin/ifup $tmp_mnt/sbin/ifup + cp_bin /bin/awk $tmp_mnt/bin/awk + cp_bin /bin/grep $tmp_mnt/bin/grep + cp_bin /bin/logger $tmp_mnt/bin/logger + cp_bin /bin/touch $tmp_mnt/bin/touch +fi fi [ "$interface" ] && verbose "[NETWORK]\t$interface ($nettype)" diff --git a/scripts/setup-prepare.sh b/scripts
Re: multipath iSCSI installs
On Thu, Apr 02, 2009 at 11:38:29PM -0700, mala...@us.ibm.com wrote: > > Mike Christie [micha...@cs.wisc.edu] wrote: > > If the ibft implementation uses one session, but exports all the targets > > in the ibft info, then in RHEL 5.3 the installer only picks up the > > session used for the ibft boot up, but the initrd root-boot code used > > after the install should log into all the targets in ibft whether they > > were used for the ibft boot or not. There different behavior is a result > > of the installer goofing up where is used the wrong api. > > It is quite likely that my iBFT implementation uses&exports a single > session. > Btw is this on HS21 blade? Or something else..? I could test also, if it's an IBM blade.. -- Pasi --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
RE: multipath iSCSI installs
Mike Christie wrote: >That is what I said in the mail you replied to more or less. For boot it is fixed. iscsiadm/iscsistart logs into all the targets found in ibft whehter >they were used for the ibft boot or not. For install there is a bug still, because the isntaller code used the wrong API which only logs into the session >used for boot. >When I did the fix for the iscsi tools, I think you even replied on the thread :) I need coffee (:. Sorry about that. So, for the installers will that be fixed with the new library approach that we are implementing? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: multipath iSCSI installs
shyam_i...@dell.com wrote: > > Mike Christie wrote: >> mala...@us.ibm.com wrote: >>> Hi all, >>> >>> I am trying to install RHEL5.3 on an iSCSI disk with two paths. >>> I booted with "mapth" option but the installer picked up only a > single >>> path. Is this the expected behavior when I use "iBFT"? >>> >> For this mail ibft boot means the boot process where the ibft > implementation hooks into the box and logs into the target and brings > over the kernel and >> initrd. >> >> >> It could be. If the ibft implementation uses only one session during > the ibft boot and then only exports that one session then yeah, it is > expected >> because the iscsi tools only know what ibft tells us. > > Mike - Here is an old thread that we had on the limitation of the > fwparam tool's limitation with multiple sessions at boot time. > http://www.mail-archive.com/open-iscsi@googlegroups.com/msg01659.html > > You had suggested a few iface bindings that could be done to have > multiple sessions with an extra flag that uses the hint from the iBFT to > have multiple sessions. > > I guess this might be the problem that malahal might be having. The iBFT There are multiple issues remember? 1. Do we want to bind sessions to specific nics. - You need the iface bindings like you mentioned above. 2. If the ibft exports all the targets setup even if they were not used for ibft boot do we want to log into them? - We used to only log into the one used for boot. Now for boot we log into all of them using the default behavior where we let the network layer route. For install in RHEL, there was a goof where it always only logs into the one used for boot. > might be configured to have multiple sessions and thus have two paths > but the iscsiadm shipped with the installer will not take the hint from > iBFT to connect to two paths. That is what I said in the mail you replied to more or less. For boot it is fixed. iscsiadm/iscsistart logs into all the targets found in ibft whehter they were used for the ibft boot or not. For install there is a bug still, because the isntaller code used the wrong API which only logs into the session used for boot. When I did the fix for the iscsi tools, I think you even replied on the thread :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
RE: multipath iSCSI installs
mala...@us.ibm.com wrote: >Mike Christie [micha...@cs.wisc.edu] wrote: >> If the ibft implementation uses one session, but exports all the >> targets in the ibft info, then in RHEL 5.3 the installer only picks up >> the session used for the ibft boot up, but the initrd root-boot code >> used after the install should log into all the targets in ibft whether >> they were used for the ibft boot or not. There different behavior is a >> result of the installer goofing up where is used the wrong api. > >It is quite likely that my iBFT implementation uses&exports a single session. My bad! Malahal has fixed the issue I guess. But it does seem like we will hit an issue if the iBFT does export multiple sessions and iscsiadm doesn't use them in the installer to find multiple paths. -Shyam --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
RE: multipath iSCSI installs
Mike Christie wrote: >mala...@us.ibm.com wrote: >> Hi all, >> >> I am trying to install RHEL5.3 on an iSCSI disk with two paths. >> I booted with "mapth" option but the installer picked up only a single >> path. Is this the expected behavior when I use "iBFT"? >> > >For this mail ibft boot means the boot process where the ibft implementation hooks into the box and logs into the target and brings over the kernel and >initrd. > > >It could be. If the ibft implementation uses only one session during the ibft boot and then only exports that one session then yeah, it is expected >because the iscsi tools only know what ibft tells us. Mike - Here is an old thread that we had on the limitation of the fwparam tool's limitation with multiple sessions at boot time. http://www.mail-archive.com/open-iscsi@googlegroups.com/msg01659.html You had suggested a few iface bindings that could be done to have multiple sessions with an extra flag that uses the hint from the iBFT to have multiple sessions. I guess this might be the problem that malahal might be having. The iBFT might be configured to have multiple sessions and thus have two paths but the iscsiadm shipped with the installer will not take the hint from iBFT to connect to two paths. -Shyam Iyer --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: multipath iSCSI installs
Mike Christie [micha...@cs.wisc.edu] wrote: > If the ibft implementation uses one session, but exports all the targets > in the ibft info, then in RHEL 5.3 the installer only picks up the > session used for the ibft boot up, but the initrd root-boot code used > after the install should log into all the targets in ibft whether they > were used for the ibft boot or not. There different behavior is a result > of the installer goofing up where is used the wrong api. It is quite likely that my iBFT implementation uses&exports a single session. > This should work, but there are some other issues. For the boot that > panics what was the problem? Did the session get logged in or was it a > panic in the iscsi code? The panic is due to not finding any bootable device. Actually the newly created initrd didn't have any iSCSI code at all! One time, there was some iSCSI code in the "initrd" image , but the session login failed. Maybe real network failure that one time as I can't reproduce the latter problem now. > There was a bug where mkinitrd would use the current sessoins values > then stick them in the initrd and use them. In 5.3, if you were using > ibft then the initrd code should use the ibft values. Check out > /sbin/mkinitrd:emit_iscsi_device. It should call iscsi_is_ibft and > figure out that ibft is used. It is still sort of broken. If you changed > the ibft value then ran mkinitrd we would not figure out ibft is being > used because it checks this by matching the ibft values with the current > sessions. Yes, the mkinitrd code looks at ibft values and the current session(s). If the session matches ibft values, it uses ibft. If not, it uses the session values directly. But the problem in my case is that as soon as I run "iscsiadm -m node -l" to login to the second session, it creates a new path /dev/sdb as expected. The udev, I believe, also creating a "/dev/disk/by-id/..." special file for it. LVM displays that "by-id" path as its PV instead of the regular /dev/sdb2 (or /dev/sda2). mkinitrd passes that "by-id" symlink to findstoragedriver() in handlelvordev findstoragedriver() can't handle symlinks and ends up not putting anything in the "initrd". I hacked "/sbin/mkinitrd" to fix the above stuff as well as added the second path. I also modified it to include multipath related changes. Now I am able to boot RHEL5.3 on a multipath iSCSI device! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: multipath iSCSI installs
mala...@us.ibm.com wrote: > Hi all, > > I am trying to install RHEL5.3 on an iSCSI disk with two paths. > I booted with "mapth" option but the installer picked up only a single > path. Is this the expected behavior when I use "iBFT"? > For this mail ibft boot means the boot process where the ibft implementation hooks into the box and logs into the target and brings over the kernel and initrd. It could be. If the ibft implementation uses only one session during the ibft boot and then only exports that one session then yeah, it is expected because the iscsi tools only know what ibft tells us. If the ibft implementation uses one session, but exports all the targets in the ibft info, then in RHEL 5.3 the installer only picks up the session used for the ibft boot up, but the initrd root-boot code used after the install should log into all the targets in ibft whether they were used for the ibft boot or not. There different behavior is a result of the installer goofing up where is used the wrong api. > The install went fine on a single path. I was trying to convert the > single path to multi-path by running "mkinitrd". RHEL was unable to boot > (panics) with the new "initrd" image. The only difference between the > old initrd image and the new one is that the old initrd image was using > iBFT method and the new image was trying to use the values from the > existing session(s) at "initrd" creation time. For some reason the > latter method doesn't work. Is this a known bug? This should work, but there are some other issues. For the boot that panics what was the problem? Did the session get logged in or was it a panic in the iscsi code? Were you trying to force mkinitrd to use the session values at initrd building time or did you want it to use ibft one at runtime? There was a bug where mkinitrd would use the current sessoins values then stick them in the initrd and use them. In 5.3, if you were using ibft then the initrd code should use the ibft values. Check out /sbin/mkinitrd:emit_iscsi_device. It should call iscsi_is_ibft and figure out that ibft is used. It is still sort of broken. If you changed the ibft value then ran mkinitrd we would not figure out ibft is being used because it checks this by matching the ibft values with the current sessions. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
RE: multipath iSCSI installs
Hi Malahal. There is a BZ for SLES. https://bugzilla.novell.com/show_bug.cgi?id=436463 I, too, have not been able to get RHEL 5.3 to install either. Regards, Wayne. -Original Message- From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On Behalf Of mala...@us.ibm.com Sent: Wednesday, April 01, 2009 8:13 AM To: open-iscsi@googlegroups.com Subject: multipath iSCSI installs Hi all, I am trying to install RHEL5.3 on an iSCSI disk with two paths. I booted with "mapth" option but the installer picked up only a single path. Is this the expected behavior when I use "iBFT"? The install went fine on a single path. I was trying to convert the single path to multi-path by running "mkinitrd". RHEL was unable to boot (panics) with the new "initrd" image. The only difference between the old initrd image and the new one is that the old initrd image was using iBFT method and the new image was trying to use the values from the existing session(s) at "initrd" creation time. For some reason the latter method doesn't work. Is this a known bug? I also tried installing SLES10 and SLES11. I believe they recommend installing on a single path and then converting to multipath. I have found that SLES11's initrd image can only find one path even after including "multipath" support into the initrd image. It creates a dm-multipath device with a single path and later converts to dm-multipath device with two paths later when it runs scripts in /etc/init.d. SLES10's behavior might be same, but I didn't analyse. Does anyone know if SLES11's initrd image can find more than one iSCSI path? Thanks, Malahal. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: multipath iSCSI installs
On Wed, Apr 1, 2009 at 6:42 PM, Pasi Kärkkäinen wrote: > > On Wed, Apr 01, 2009 at 05:13:10AM -0700, mala...@us.ibm.com wrote: > > > > Hi all, > > > > I am trying to install RHEL5.3 on an iSCSI disk with two paths. > > I booted with "mapth" option but the installer picked up only a single > > path. Is this the expected behavior when I use "iBFT"? > > > > I've installed RHEL 5.3 (and CentOS 5.3) to multipath-root using "mpath" > installer option. It worked fine. I didn't use iBFT though.. > > > The install went fine on a single path. I was trying to convert the > > single path to multi-path by running "mkinitrd". RHEL was unable to boot > > (panics) with the new "initrd" image. The only difference between the > > old initrd image and the new one is that the old initrd image was using > > iBFT method and the new image was trying to use the values from the > > existing session(s) at "initrd" creation time. For some reason the > > latter method doesn't work. Is this a known bug? > > > > Yeah conversion from single path to multipath root can be tricky.. I think > it > might require manual customization/editing of initrd image (scripts). > I know it is tricky and I hand edited "mkinitrd" script (not the image itself). But it doesn't matter whether I edit the script or not. I installed RHEL5.3 with iBFT and then ran "mkinitrd /boot/initrd-.img " without changing anything anywhere. I couldn't boot! It has to be a bug. Did anyone run "mkinitrd" successfully after an iBFT iSCSI install? Thanks, Malahal. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: multipath iSCSI installs
On Wed, Apr 01, 2009 at 05:13:10AM -0700, mala...@us.ibm.com wrote: > > Hi all, > > I am trying to install RHEL5.3 on an iSCSI disk with two paths. > I booted with "mapth" option but the installer picked up only a single > path. Is this the expected behavior when I use "iBFT"? > I've installed RHEL 5.3 (and CentOS 5.3) to multipath-root using "mpath" installer option. It worked fine. I didn't use iBFT though.. > The install went fine on a single path. I was trying to convert the > single path to multi-path by running "mkinitrd". RHEL was unable to boot > (panics) with the new "initrd" image. The only difference between the > old initrd image and the new one is that the old initrd image was using > iBFT method and the new image was trying to use the values from the > existing session(s) at "initrd" creation time. For some reason the > latter method doesn't work. Is this a known bug? > Yeah conversion from single path to multipath root can be tricky.. I think it might require manual customization/editing of initrd image (scripts). -- Pasi --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---