Bug#850060: 2.0.874-2~exp1 issues
On 01/05/2017 12:10 PM, Christian Seiler wrote:
> On 01/05/2017 07:10 PM, Andrew Patterson wrote:
>> On 01/04/2017 04:11 PM, Christian Seiler wrote:
>>> OTOH, initramfs should write to /run/initramfs only, so maybe
>>> we should pass -p /run/initramfs/iscsiuio.pid to iscsiuio
>>> instead as well.
>>
>> Yes. I believe that will work. I wonder why this option is not in the
>> man-page.
>
> -ENOTIME on part of the developers perhaps. ;-)
>
>>> The ideal solution would be to mirror the check that is done
>>> for -b in -N. In that case we'd either configure the host
>>> interface (and use software iSCSI), or configure offloading
>>> (and use hardware iSCSI), but never both, and never neither.
>>> So instead of the current patch for #850057 I would suggest
>>> to do that instead. That should then also be upstream-able. I
>>> can prepare a patch for that tomorrow.
Sounds good.
>>
>> That was my thought. However, I don't think you can programaticaly
>> determine whether the card is configured for iscsi offload.
>
> Well, according to the commit message I referenced in my
> previous email [1], we have the following situation:
>
> - cxgb*i: always use offloading (which will apparently work,
>since offloading will reuse the MAC address)
>
> - bnx2*i: look at the MAC address to see if the MAC address
>from iBFT matches the offloading mac: if so, assume
>offloading, if not, assume software
>
> - otherwise: always assume software
>
> This is what the code bracketed by -DOFFLOAD_BOOT_SUPPORTED
> checks for, and we just need to mirror that code (or better:
> extract into an own function and call it from both places)
> for -b.
>
> I'll prepare a patch, you can then test it.
>
> [1]
> https://github.com/open-iscsi/open-iscsi/commit/ee115be828362653478e6fe7cd4c6ee3318223ff
>
>> It looks like iscsuio is complaining about a missing libgcc:
>>
>> writev(2, [{"libgcc_s.so.1 must be installed "..., 59]], 1) = 59
>>
>> which is indeed missing. I think pthreads needs libgcc, but shouldn't
>> copy_exec take care of this?
>
> Nope, since libgcc_s.so is dlopened() by pthread_cancel. So
> the automatic library dependency detection doesn't work
> properly.
>
> We are not the only ones with that problem; luckily someone
> already wrote a patch that does just this for btrfs, we can
> just steal that:
> https://bugs.debian.org/830883
I tested this "fix". iscsiuio/iscsistart -b is working now.
>
> I'll work on an updated package.
>
> Regards,
> Christian
>
--
Andrew Patterson
Hewlett-Packard Enterprise
Bug#850060: 2.0.874-2~exp1 issues
On 01/05/2017 07:10 PM, Andrew Patterson wrote:
> On 01/04/2017 04:11 PM, Christian Seiler wrote:
>> OTOH, initramfs should write to /run/initramfs only, so maybe
>> we should pass -p /run/initramfs/iscsiuio.pid to iscsiuio
>> instead as well.
>
> Yes. I believe that will work. I wonder why this option is not in the
> man-page.
-ENOTIME on part of the developers perhaps. ;-)
>> The ideal solution would be to mirror the check that is done
>> for -b in -N. In that case we'd either configure the host
>> interface (and use software iSCSI), or configure offloading
>> (and use hardware iSCSI), but never both, and never neither.
>> So instead of the current patch for #850057 I would suggest
>> to do that instead. That should then also be upstream-able. I
>> can prepare a patch for that tomorrow.
>
> That was my thought. However, I don't think you can programaticaly
> determine whether the card is configured for iscsi offload.
Well, according to the commit message I referenced in my
previous email [1], we have the following situation:
- cxgb*i: always use offloading (which will apparently work,
since offloading will reuse the MAC address)
- bnx2*i: look at the MAC address to see if the MAC address
from iBFT matches the offloading mac: if so, assume
offloading, if not, assume software
- otherwise: always assume software
This is what the code bracketed by -DOFFLOAD_BOOT_SUPPORTED
checks for, and we just need to mirror that code (or better:
extract into an own function and call it from both places)
for -b.
I'll prepare a patch, you can then test it.
[1]
https://github.com/open-iscsi/open-iscsi/commit/ee115be828362653478e6fe7cd4c6ee3318223ff
> It looks like iscsuio is complaining about a missing libgcc:
>
> writev(2, [{"libgcc_s.so.1 must be installed "..., 59]], 1) = 59
>
> which is indeed missing. I think pthreads needs libgcc, but shouldn't
> copy_exec take care of this?
Nope, since libgcc_s.so is dlopened() by pthread_cancel. So
the automatic library dependency detection doesn't work
properly.
We are not the only ones with that problem; luckily someone
already wrote a patch that does just this for btrfs, we can
just steal that:
https://bugs.debian.org/830883
I'll work on an updated package.
Regards,
Christian
Bug#850060: 2.0.874-2~exp1 issues
On 01/05/2017 11:10 AM, Andrew Patterson wrote: > > On 01/04/2017 04:11 PM, Christian Seiler wrote: >> Control: reopen -1 >> >> Hi, >> (reopening the bug report since iscsiuio doesn't actually work >> according to what you're telling me) >> >> On 01/04/2017 11:30 PM, Andrew Patterson wrote: >>> 1. In debian/extra/initramfs.local-top and >>> debian/extra/initramfs.local-bottom /sbin/iscsuio creates a pid file >>> in /run/iscsiuio.pid. You cannot override it on the command-line. The >>> --pidfile option for start-stop-daemon should point to this location >>> instead of /run/initramfs/iscsiuio.pid. >> >> Gah. I thought I had fixed that before the upload. :-( >> A good thing there's experimental... >> >> OTOH, initramfs should write to /run/initramfs only, so maybe >> we should pass -p /run/initramfs/iscsiuio.pid to iscsiuio >> instead as well. > > Yes. I believe that will work. I wonder why this option is not in the > man-page. > >> >>> 2. The /sbin/iscsiuio deamon is trying to create a logfile in >>> /var/log/iscsiuio.log. The /var/log directory does not exist in the >>> initramfs. >> >> Unfortunately that's hardcoded. OTOH from reading the code >> this shouldn't be an issue. >> >>> 3. The iscsiuio daemon is dieing before or during iscsistart -b. I am >>> investigating why. It works fine when run after boot. >> >> This is weird, on my test VM (no offloading NIC though) it >> does start (and shut down again). >> >> Question: does the network card require firmware for offloading >> to work? If that's the case we might need to add the firmware >> to the initramfs or this to work properly... > > Yes, the card requires firmware. But it has been loaded (according to > kernel output). > >> >>> 4. The commit 02c3051 "Don't ignore offloading NICs in iscsistart" >>> will cause the interface to be configured in both iscsistart -N and in >>> iscsistart -b for cards with iscsi hardware offload. I suspect we >>> instead need a flag in iscsistart to bring up the NIC in iscsistart >>> regardless if the NIC uses an offloadable driver. The local-top script >>> can use this flag if /sbin/iscsiuio is not present, e.g., >>> >>> ISCSISTART_FLAGS="" >>> if [ ! -x /sbin/iscsiuio ] ; then >>>ISCSISTART_FLAGS="-S" >>> fi >> >> Hmm, now I see what you mean. I also dug up this commit which >> gives a little insight. >> >> https://github.com/open-iscsi/open-iscsi/commit/ee115be828362653478e6fe7cd4c6ee3318223ff >> >> However, I think the solution is different from what you suggest: the >> "fix" for #850057 is wrong I believe. >> >> Debian sid package (w/o -DOFFLOAD_BOOT_SUPPORTED): >> >> - cxgb*i: iscsistart -N ignores interface >>iscsistart -b doesn't configure offloading >>=> won't work >> >> - bnx2i (any mode): iscsistart -N ignores interface >> iscsistart -b doesn't configure offloading >>=> won't work >> >> Current upstream situation (w/ -DOFFLOAD_BOOT_SUPPORTED): >> >> - cxgb*i: iscsistart -N ignores interface >>iscsistart -b configures offloading >>=> should work >> (but don't have hardware) >> (also: remind me to check whether we copy the >> module into the initramfs) >> >> - bnx2i non-hba: iscsistart -N ignores it >> iscsistart -b doesn't configure offloading >>=> won't work >> >> - bnx2i hba: iscsistart -N ignores it >> iscsistart -b configures offloading >>=> should work (once iscsiuio is running) >> >> "Fix" for 850057 (in experimental): >> >> - cxgb*i: iscsistart -N configures interface on host >>iscsistart -b configures offloading >> => no idea what happens thereafter >> (since MAC address is the same on these cards: is >> this a problem if they have the same IP?) >> >> - bnx2i non-hba: iscsistart -N configures interface >> iscsistart -b doesn't configure offloading, >> but enables software iSCSI >> => should work >> >> - bnx2i hba: iscsistart -N configures interface >> iscsistart -b configures offloading >> => same IP, different MAC addresses >> => will cause trouble >> >> The ideal solution would be to mirror the check that is done >> for -b in -N. In that case we'd either configure the host >> interface (and use software iSCSI), or configure offloading >> (and use hardware iSCSI), but never both, and never neither. >> So instead of the current patch for #850057 I would suggest >> to do that instead. That should then also be upstream-able. I >> can prepare a patch for that tomorrow. > > That was my thought. However, I don't think you can programaticaly > determine whether the card is configured for iscsi offload. We need to > use some sort of signal from the admin, hence my suggestion to check > for the presence of iscsiuio or to use a flag. > >> >> That still leaves us with the question why iscsiuio doesn't >> appear
Bug#850060: 2.0.874-2~exp1 issues
On 01/04/2017 04:11 PM, Christian Seiler wrote: > Control: reopen -1 > > Hi, > (reopening the bug report since iscsiuio doesn't actually work > according to what you're telling me) > > On 01/04/2017 11:30 PM, Andrew Patterson wrote: >> 1. In debian/extra/initramfs.local-top and >> debian/extra/initramfs.local-bottom /sbin/iscsuio creates a pid file >> in /run/iscsiuio.pid. You cannot override it on the command-line. The >> --pidfile option for start-stop-daemon should point to this location >> instead of /run/initramfs/iscsiuio.pid. > > Gah. I thought I had fixed that before the upload. :-( > A good thing there's experimental... > > OTOH, initramfs should write to /run/initramfs only, so maybe > we should pass -p /run/initramfs/iscsiuio.pid to iscsiuio > instead as well. Yes. I believe that will work. I wonder why this option is not in the man-page. > >> 2. The /sbin/iscsiuio deamon is trying to create a logfile in >> /var/log/iscsiuio.log. The /var/log directory does not exist in the >> initramfs. > > Unfortunately that's hardcoded. OTOH from reading the code > this shouldn't be an issue. > >> 3. The iscsiuio daemon is dieing before or during iscsistart -b. I am >> investigating why. It works fine when run after boot. > > This is weird, on my test VM (no offloading NIC though) it > does start (and shut down again). > > Question: does the network card require firmware for offloading > to work? If that's the case we might need to add the firmware > to the initramfs or this to work properly... Yes, the card requires firmware. But it has been loaded (according to kernel output). > >> 4. The commit 02c3051 "Don't ignore offloading NICs in iscsistart" >> will cause the interface to be configured in both iscsistart -N and in >> iscsistart -b for cards with iscsi hardware offload. I suspect we >> instead need a flag in iscsistart to bring up the NIC in iscsistart >> regardless if the NIC uses an offloadable driver. The local-top script >> can use this flag if /sbin/iscsiuio is not present, e.g., >> >> ISCSISTART_FLAGS="" >> if [ ! -x /sbin/iscsiuio ] ; then >>ISCSISTART_FLAGS="-S" >> fi > > Hmm, now I see what you mean. I also dug up this commit which > gives a little insight. > > https://github.com/open-iscsi/open-iscsi/commit/ee115be828362653478e6fe7cd4c6ee3318223ff > > However, I think the solution is different from what you suggest: the > "fix" for #850057 is wrong I believe. > > Debian sid package (w/o -DOFFLOAD_BOOT_SUPPORTED): > > - cxgb*i: iscsistart -N ignores interface >iscsistart -b doesn't configure offloading >=> won't work > > - bnx2i (any mode): iscsistart -N ignores interface > iscsistart -b doesn't configure offloading >=> won't work > > Current upstream situation (w/ -DOFFLOAD_BOOT_SUPPORTED): > > - cxgb*i: iscsistart -N ignores interface >iscsistart -b configures offloading >=> should work > (but don't have hardware) > (also: remind me to check whether we copy the > module into the initramfs) > > - bnx2i non-hba: iscsistart -N ignores it > iscsistart -b doesn't configure offloading >=> won't work > > - bnx2i hba: iscsistart -N ignores it > iscsistart -b configures offloading >=> should work (once iscsiuio is running) > > "Fix" for 850057 (in experimental): > > - cxgb*i: iscsistart -N configures interface on host >iscsistart -b configures offloading > => no idea what happens thereafter > (since MAC address is the same on these cards: is > this a problem if they have the same IP?) > > - bnx2i non-hba: iscsistart -N configures interface > iscsistart -b doesn't configure offloading, > but enables software iSCSI > => should work > > - bnx2i hba: iscsistart -N configures interface > iscsistart -b configures offloading > => same IP, different MAC addresses > => will cause trouble > > The ideal solution would be to mirror the check that is done > for -b in -N. In that case we'd either configure the host > interface (and use software iSCSI), or configure offloading > (and use hardware iSCSI), but never both, and never neither. > So instead of the current patch for #850057 I would suggest > to do that instead. That should then also be upstream-able. I > can prepare a patch for that tomorrow. That was my thought. However, I don't think you can programaticaly determine whether the card is configured for iscsi offload. We need to use some sort of signal from the admin, hence my suggestion to check for the presence of iscsiuio or to use a flag. > > That still leaves us with the question why iscsiuio doesn't > appear to work in the initramfs on your system. You could > just add strace to your initramfs and call iscsiuio in a > detached strace, a la > > old: > start-stop-daemon ...
Bug#850060: 2.0.874-2~exp1 issues
Control: reopen -1 Hi, (reopening the bug report since iscsiuio doesn't actually work according to what you're telling me) On 01/04/2017 11:30 PM, Andrew Patterson wrote: > 1. In debian/extra/initramfs.local-top and > debian/extra/initramfs.local-bottom /sbin/iscsuio creates a pid file > in /run/iscsiuio.pid. You cannot override it on the command-line. The > --pidfile option for start-stop-daemon should point to this location > instead of /run/initramfs/iscsiuio.pid. Gah. I thought I had fixed that before the upload. :-( A good thing there's experimental... OTOH, initramfs should write to /run/initramfs only, so maybe we should pass -p /run/initramfs/iscsiuio.pid to iscsiuio instead as well. > 2. The /sbin/iscsiuio deamon is trying to create a logfile in > /var/log/iscsiuio.log. The /var/log directory does not exist in the > initramfs. Unfortunately that's hardcoded. OTOH from reading the code this shouldn't be an issue. > 3. The iscsiuio daemon is dieing before or during iscsistart -b. I am > investigating why. It works fine when run after boot. This is weird, on my test VM (no offloading NIC though) it does start (and shut down again). Question: does the network card require firmware for offloading to work? If that's the case we might need to add the firmware to the initramfs or this to work properly... > 4. The commit 02c3051 "Don't ignore offloading NICs in iscsistart" > will cause the interface to be configured in both iscsistart -N and in > iscsistart -b for cards with iscsi hardware offload. I suspect we > instead need a flag in iscsistart to bring up the NIC in iscsistart > regardless if the NIC uses an offloadable driver. The local-top script > can use this flag if /sbin/iscsiuio is not present, e.g., > > ISCSISTART_FLAGS="" > if [ ! -x /sbin/iscsiuio ] ; then >ISCSISTART_FLAGS="-S" > fi Hmm, now I see what you mean. I also dug up this commit which gives a little insight. https://github.com/open-iscsi/open-iscsi/commit/ee115be828362653478e6fe7cd4c6ee3318223ff However, I think the solution is different from what you suggest: the "fix" for #850057 is wrong I believe. Debian sid package (w/o -DOFFLOAD_BOOT_SUPPORTED): - cxgb*i: iscsistart -N ignores interface iscsistart -b doesn't configure offloading => won't work - bnx2i (any mode): iscsistart -N ignores interface iscsistart -b doesn't configure offloading => won't work Current upstream situation (w/ -DOFFLOAD_BOOT_SUPPORTED): - cxgb*i: iscsistart -N ignores interface iscsistart -b configures offloading => should work (but don't have hardware) (also: remind me to check whether we copy the module into the initramfs) - bnx2i non-hba: iscsistart -N ignores it iscsistart -b doesn't configure offloading => won't work - bnx2i hba: iscsistart -N ignores it iscsistart -b configures offloading => should work (once iscsiuio is running) "Fix" for 850057 (in experimental): - cxgb*i: iscsistart -N configures interface on host iscsistart -b configures offloading => no idea what happens thereafter (since MAC address is the same on these cards: is this a problem if they have the same IP?) - bnx2i non-hba: iscsistart -N configures interface iscsistart -b doesn't configure offloading, but enables software iSCSI => should work - bnx2i hba: iscsistart -N configures interface iscsistart -b configures offloading => same IP, different MAC addresses => will cause trouble The ideal solution would be to mirror the check that is done for -b in -N. In that case we'd either configure the host interface (and use software iSCSI), or configure offloading (and use hardware iSCSI), but never both, and never neither. So instead of the current patch for #850057 I would suggest to do that instead. That should then also be upstream-able. I can prepare a patch for that tomorrow. That still leaves us with the question why iscsiuio doesn't appear to work in the initramfs on your system. You could just add strace to your initramfs and call iscsiuio in a detached strace, a la old: start-stop-daemon ... /sbin/iscsiuio new: start-stop-daemon ... /bin/strace -- -D -f \ -o /run/initramfs/iscsiuio.trace -- /sbin/iscsiuio (The first -- is for start-stop-daemon, the second one for strace itself.) Regards, Christian
Bug#850060: 2.0.874-2~exp1 issues
I have been testing 2.0.874-2~exp1 and found the following issues: 1. In debian/extra/initramfs.local-top and debian/extra/initramfs.local-bottom /sbin/iscsuio creates a pid file in /run/iscsiuio.pid. You cannot override it on the command-line. The --pidfile option for start-stop-daemon should point to this location instead of /run/initramfs/iscsiuio.pid. 2. The /sbin/iscsiuio deamon is trying to create a logfile in /var/log/iscsiuio.log. The /var/log directory does not exist in the initramfs. I do not know if this is causing a problem (perhaps related to #1). 3. The iscsiuio daemon is dieing before or during iscsistart -b. I am investigating why. It works fine when run after boot. 4. The commit 02c3051 "Don't ignore offloading NICs in iscsistart" will cause the interface to be configured in both iscsistart -N and in iscsistart -b for cards with iscsi hardware offload. I suspect we instead need a flag in iscsistart to bring up the NIC in iscsistart regardless if the NIC uses an offloadable driver. The local-top script can use this flag if /sbin/iscsiuio is not present, e.g., ISCSISTART_FLAGS="" if [ ! -x /sbin/iscsiuio ] ; then ISCSISTART_FLAGS="-S" fi . . . modprobe iscsi_ibft iscsistart -N $ISCSISTART_FLAGS The iscsistart code would have to be modified to add support for this flag. Alternatively, iscsistart could check for the presence of /sbin/iscsiuio and "do the right thing". -- Andrew Patterson Hewlett-Packard Enterprise
