[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
This bug was fixed in the package rdma-core - 17.1-1ubuntu0.2 --- rdma-core (17.1-1ubuntu0.2) bionic; urgency=medium * d/p/u/lp-1838939-Add-Microsoft-Azure-RDMA-device-to-MLX4-HCA-table.patch: Detect Azure RDMA devices as MLX4 (LP: #1838939) -- Christian Ehrhardt Mon, 05 Aug 2019 08:57:39 +0200 ** Changed in: rdma-core (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
** Tags removed: verification-needed verification-needed-bionic ** Tags added: verification-done verification-done-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
Test engineer reports success and no issues. For more details, please refer support case 00237568. test-vm:~# ibv_devices device node GUID -- mlx4_0 00155d33ff0f -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
Still waiting for verification - I'll ask on side channels as well ... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
@Joshua R. Poulson (jrp) would you mind having this tested on Azure to have it SRU-released? Update the bug here once the fix was confirmed so it can be marked accordingly. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
Hello Christian, or anyone else affected, Accepted rdma-core into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/rdma- core/17.1-1ubuntu0.2 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, details of your testing will help us make a better decision. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! ** Changed in: rdma-core (Ubuntu Bionic) Status: Triaged => Fix Committed ** Tags added: verification-needed verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
FYI: as I was asked what this is waiting on - this was waiting for review and acceptance [1] by the SRU Team for a while now. I have pinged the SRU team vaguards of today to get it processed rather soon. But I beg your pardon in advance as there is a sprint going on and due to that there might still some time to pass before this is done. [1]: https://wiki.ubuntu.com/StableReleaseUpdates#Procedure -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
Thanks Josh for the extended checks, I have tagged it up for Bionic correctly and sponsored it for the review of the SRU [1] Team. Once they accept it into Bionic-proposed they will ask for a verification, it would be great if you again could check then that it works. It could, but doesn't necessarily be the full check, but at least if the device recognition that was planned to be fixed really works with that build (of the same source) still. [1]: https://wiki.ubuntu.com/StableReleaseUpdates ** Also affects: rdma-core (Ubuntu Bionic) Importance: Undecided Status: New ** Changed in: rdma-core (Ubuntu Bionic) Status: New => Triaged ** Changed in: rdma-core (Ubuntu Bionic) Importance: Undecided => Medium ** Changed in: rdma-core (Ubuntu) Status: Triaged => Fix Released ** Description changed: [Impact] - * Azure RDMA devices can be a "Network Direct" version of the MLX -InfiniBand card that is exposed to the guest and they need to be -handled by the MLX4 driver. + * Azure RDMA devices can be a "Network Direct" version of the MLX + InfiniBand card that is exposed to the guest and they need to be + handled by the MLX4 driver. But to be enabled properly the system needs to detect them as such. That was added in rdma-core v20 hence being fix released in >=Disco. * Fix [1] is just a new alias to detect the card as what it is. [Test Case] - * TBD: Try to use rdma on "Microsoft Azure RDMA device". - It would be great if someone could add some real content here. + * The TL;DR is to use rdma on "Microsoft Azure RDMA device". +There is no good way without an Azure instance to test this specific +bug. The check is if the userspace drivers are then correctly loaded. +Lacking such an instance myself Microsoft helped on the PPA and will +help on SRU verification. [Regression Potential] * a) The regression potential should be minimal. Existing systems either already match the HCA table or they don't (neither before nor after the fix. An issue I could think of is if there are devices that announce as vmbus:3daf2e8ca732094bab99bd1f1c86b501 but are NOT in fact this device type - they would then run into issues. * b) Of course the fix will "enable rdma support on some more devices" if someone had those devices attached but didn't use/configure them they might now initialize a bit further. But that isn't an issue and especially in cloud environments where HW config is just a click away no one usually pays for extra devices without using them. * c) I happen to know that for DPDK usage of these devices several fixes in later rdma-core are preferred or even needed. I'm not sure about the usage in this case - but enabling could expose those issues which formerly were hidden behind the "not supported" misdetection of the card. For B) and C) I'd want Microsoft to ack and test this from a PPA and do the verification on this - to be not only ok for this but for for Azure in general. [Other Info] * This falls under the SRU exception of "other safe cases" for "For Long Term Support releases we regularly want to enable new hardware". New modaliases are explicitly listed there as cases for this. - * This is reported to work in 16.04, but not as a classic "this commit -broke it regression". Back then the world of rdma/infiniband just -worked totally different as it was before the big revamp into rdma-core -never the less one could see that (abstract, not caring about details) -as an update-regression when going 16.04 to 18.04. + * This is reported to work in 16.04, but not as a classic "this commit + broke it regression". Back then the world of rdma/infiniband just + worked totally different as it was before the big revamp into rdma-core + never the less one could see that (abstract, not caring about details) + as an update-regression when going 16.04 to 18.04. [1]: https://github.com/linux-rdma/rdma- core/commit/b98397f3068a42e7d8e67d4ef2a90393b4c4f08a -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
Please proceed, no regressions. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
The merge request was reviewed and approved, to be as clear as possible - what we now need to unblock this and continue is: - testing the PPA if it is fulfilling the needs (card detection) and works fine (detected card works properly in your tests) - description of test steps done (so it can be used for the SRU template) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
FYI - this can be pre-tested to work against PPA [1]. I'd be very happy if one could a) do that test if it is sufficient b) dump the steps to test here then so that we can complete the SRU template to get things started [1]: https://launchpad.net/~paelzer/+archive/ubuntu/bug-1838939-rdma- azure-mlx4 ** Description changed: [Impact] - * Azure RDMA devices are driven by MLX4 driver (which I assume those - devices are derived from). + * Azure RDMA devices can be a "Network Direct" version of the MLX +InfiniBand card that is exposed to the guest and they need to be +handled by the MLX4 driver. But to be enabled properly the system needs to detect them as such. That was added in rdma-core v20 hence being fix released in >=Disco. * Fix [1] is just a new alias to detect the card as what it is. [Test Case] * TBD: Try to use rdma on "Microsoft Azure RDMA device". It would be great if someone could add some real content here. [Regression Potential] * a) The regression potential should be minimal. Existing systems either already match the HCA table or they don't (neither before nor after the fix. An issue I could think of is if there are devices that announce as vmbus:3daf2e8ca732094bab99bd1f1c86b501 but are NOT in fact this device type - they would then run into issues. * b) Of course the fix will "enable rdma support on some more devices" if someone had those devices attached but didn't use/configure them they might now initialize a bit further. But that isn't an issue and especially in cloud environments where HW config is just a click away no one usually pays for extra devices without using them. - * c) I happen to know that for DPDK usage of these devices several fixes -in later rdma-core are preferred or even needed. I'm not sure about the -usage in this case - but enabling could expose those issues which -formerly were hidden behind the "not supported" misdetection of the -card. + * c) I happen to know that for DPDK usage of these devices several fixes + in later rdma-core are preferred or even needed. I'm not sure about the + usage in this case - but enabling could expose those issues which + formerly were hidden behind the "not supported" misdetection of the + card. - For B) and C) I'd want Microsoft to ack and test this from a PPA and do + For B) and C) I'd want Microsoft to ack and test this from a PPA and do the verification on this - to be not only ok for this but for for Azure in general. [Other Info] * This falls under the SRU exception of "other safe cases" for "For Long Term Support releases we regularly want to enable new hardware". New modaliases are explicitly listed there as cases for this. + * This is reported to work in 16.04, but not as a classic "this commit +broke it regression". Back then the world of rdma/infiniband just +worked totally different as it was before the big revamp into rdma-core +never the less one could see that (abstract, not caring about details) +as an update-regression when going 16.04 to 18.04. + [1]: https://github.com/linux-rdma/rdma- core/commit/b98397f3068a42e7d8e67d4ef2a90393b4c4f08a -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
Packaging the fix is easy, here [1] is a merge proposal for my fellow packagers to review just to be sure. [1]: https://code.launchpad.net/~paelzer/ubuntu/+source/rdma-core/+git /rdma-core/+merge/371072 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/rdma-core/+git/rdma-core/+merge/371072 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
When kernel device is probed, its vendor ID and device ID are used to ask user-mode library to find the correct device driver in user-mode. The patch will enable the correct user-mode driver is selected. There is no additional configuration or time spent on the device initialization, because at this time the kernel-mode driver is already loaded and running, regardless if user-mode driver is loaded or net. To end user, the cost and billing are not dependent on what driver is in use. If the hardware is attached to VM, the billing is already in progress. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
This change only affects the "Network Direct" version of the MLX InfiniBand card that is exposed to the guest, it is not an SRIOV or DPDK case. Those appear as Mellanox cards as expected. For what it's worth this works fine in 16.04 but is broken in 18.04. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1838939] Re: Add alias to support Microsoft Azure RDMA device via MLX4
I pinged Microsoft to add some details/statements here that only they can ... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1838939 Title: Add alias to support Microsoft Azure RDMA device via MLX4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rdma-core/+bug/1838939/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
