[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-12-13 Thread Mathew Hodson
** No longer affects: linux-azure (Ubuntu) ** No longer affects: linux-azure (Ubuntu Hirsute) ** No longer affects: cloud-init (Ubuntu Hirsute) ** No longer affects: cloud-init (Ubuntu) ** Project changed: cloud-init => ubuntu-translations ** No longer affects: ubuntu-translations -- You

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-07-01 Thread Gauthier Jolly
I tried to reproduce the issue with the latest hirsute image pushed to Azure and it appears that I cannot. While I can still reproduce the issue with 20210511.1, I can't with 20210622.1. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-06-22 Thread Gauthier Jolly
@rbalint @ddstreed any update on this issue? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1919177 Title: Azure: issues with accelerated networking on Hirsute To manage notifications about this

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-06-02 Thread Gauthier Jolly
It's been there for almost 3 months, I think it can wait a few more days. It's affecting Azure's users who use Hirsute instances with accelerated networking enabled, I don't know how many users that is. -- You received this bug notification because you are a member of Ubuntu Bugs, which is

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-06-01 Thread Balint Reczey
@ddstreed Could you please look into this SRU candidate? @gjolly What is the importance/urgency of landing this fix? ** Also affects: cloud-init (Ubuntu Hirsute) Importance: Undecided Status: New ** Also affects: systemd (Ubuntu Hirsute) Importance: Undecided Status: New **

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-05-20 Thread Gauthier Jolly
@rbalint, I just tested Impish and I cannot reproduce the issue. It doesn't prove it's not there but it's a very good sign! I can see that our automated testing failed because of this issue on the 2021-05-10 but not after. If I'm not wrong, systemd 248 was pushed on the 14th, this would make a

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-05-20 Thread Gauthier Jolly
@rbalint I was able to reproduce the issue with an image that was running the following: ii cloud-init20.4.1-79-g71564dce-0ubuntu1 all initialization and customization tool for cloud instances ii linux-image-azure 5.8.0.1017.19+21.04.14 amd64Linux kernel image

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-05-19 Thread Marcelo Cerri
Hi, Gauthier and Gauthier. I'm marking linux-azure as invalid for now. If something changes please let us know. Thank you. ** Changed in: linux-azure (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-05-18 Thread Balint Reczey
@gjolly There were networking-related changes in systemd which could have caused this but I also see that the kernel packages also changed between the Azure images: -ii linux-image-azure 5.8.0.1017.19+21.04.14 ... +ii linux-image-5.8.0-1022-azure 5.8.0-1022.24+21.04.2

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-05-18 Thread Gauthier Jolly
Hi there, Is there any update on this issue? I would like to understand who owns the investigation/debugging process? Please tell us if you need any help from the CPC team. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-29 Thread Brian Murray
** Tags removed: rls-hh-incoming -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1919177 Title: Azure: issues with accelerated networking on Hirsute To manage notifications about this bug go to:

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-29 Thread Matthieu Clemenceau
** Tags added: fr-1324 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1919177 Title: Azure: issues with accelerated networking on Hirsute To manage notifications about this bug go to:

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-27 Thread Brian Murray
** Changed in: cloud-init (Ubuntu) Milestone: ubuntu-21.04 => hirsute-updates ** Changed in: linux-azure (Ubuntu) Milestone: ubuntu-21.04 => hirsute-updates -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-23 Thread Paride Legovini
** Changed in: cloud-init (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1919177 Title: Azure: issues with accelerated networking on Hirsute To manage

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-22 Thread Gauthier Jolly
I attach here the full diff of packages between the two serials. ** Attachment added: "Package diff between 20210219 and 20210220 20.04 Azure images" https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1919177/+attachment/5491010/+files/package-diff.txt -- You received this bug

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-22 Thread Gauthier Jolly
I isolated the issue between those two serial: 20210219 (doesn't have the issue) and 20210220 (reproduces the issue). The main difference I can see between those two serial is systemd version: 20210219 -> 247.1-4ubuntu1 20210220 -> 247.3-1ubuntu2 Cloud-init version is the same

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-22 Thread Gauthier Jolly
** Also affects: systemd (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1919177 Title: Azure: issues with accelerated networking on Hirsute To

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-21 Thread Gauthier Jolly
** Description changed: [General] On Azure, when provisioning a Hirsute VM with Accelerated Networking - enabled, sometimes the SSH key is not setup properly and the user cannot - log into the VM. + enabled, sometimes part of the cloud-init configuration is not applied. + + Especially, in

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-21 Thread Gauthier Jolly
The issue probably* started between these two serials: 20210215 and 20210304. *I say "probably" because while I'm sure I can reproduce the issue with 20210304, the fact I didn't manage to reproduce it with 20210215 (in ~15 tries) doesn't **absolutely prove** that the issue wasn't there. -- You

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-21 Thread Gauthier Jolly
Hi, Ok thanks. I am now trying to find at which time the issue was first introduced. So far I tested an image from 20201222 and confirm that I cannot reproduce the issue with this image. Then by modifying the image (always before first boot) I was able to find that: - upgrading cloud-init from

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-19 Thread Paride Legovini
Hi, I don't think that's a viable option as cloud-init may very well be the component that allows the network interface to be configured with a routable address, so we can't wait such an address to be available before configuring the machine, at least not in general. -- You received this bug

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-19 Thread Gauthier Jolly
I don't have any particular knowledge of how systemd waits for the network to be ready but reading systemd-networkd-wait-online.service(8) and networkctl(1) I wonder if cloud-init shouldn't wait for networkd- wait-online to report that the link is "routable" instead of "degradated" (default

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-19 Thread Gauthier Jolly
Hi there, For whether or not the issue affects Groovy. I've been creating Groovy VMs on a loop to confirm if I could reproduce it there and (after ~50 tries) I still haven't seen it happen. I think we can safely assume that this issue does not affect Groovy. Also I was able to reproduce the

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-16 Thread Paride Legovini
Hi again Guilherme, I don't think there's any useful information hiding in the fact that cloud-init logs some messages as DataSourceAzure.py[DEBUG] and others as azure.py[DEBUG]. Look at: https://github.com/canonical/cloud-

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-16 Thread Guilherme G. Piccoli
Thanks Paride. I still think we should precisely understand that difference in the logs, since in the BAD case we always see the "azure.py" messages, not the other one. This could be related or at least a clue on the root cause. Regarding the kernel side, I've build a 5.11 kernel with debug patch

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-15 Thread Paride Legovini
Hi Guilherme, Answering your question in comment #6: > Why one method executes from DataSourceAzure.py > whereas the other from azure.py? The reason is not that interesting and won't really help here. Some log messages generated in DataSourceAzure.py are logged using a helper function defined

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-14 Thread Gauthier Jolly
Hi, Thanks for looking into that. Indeed, while trying to reproduce the issue this morning, I found it more challenging than I originally thought. I want to add a few points here on how I reproduced the issue: 1. Usually, I do not use the Azure CLI directly. I use a custom CLI of my own that

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-13 Thread Guilherme G. Piccoli
(b) Regarding kernel, I found some oddities too [ output of dmesg | grep -v "audit\|apparmor" ]: GOOD: [ 1627.732924] hv_netvsc 00224840-7fbf-0022-4840-7fbf00224840 eth0: VF slot 1 added [ 1627.733637] hv_pci fb9ea909-d0dd-41b6-a1c2-98b1233e987d: PCI VMBus probing: Using version 0x10002 [

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-13 Thread Guilherme G. Piccoli
I'm also taking a look in this one, I couldn't reproduce it (tried 11 times, with the same az-cli command-line provided by gjolly. Found 2 interesting things after Gauthier provide me access to one of his failing instances: (a) Regarding cloud-init, I see the following in the logs (comparing a

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-13 Thread Tim Gardner
gjolly - can you attach the whole /var/log/syslog soon after boot ? I see the same pattern of link UP/DOWN/UP on a non-accelerated Hirsute instance, but systemd correctly waits for networking to be configured in this case. -- You received this bug notification because you are a member of Ubuntu

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-13 Thread Dimitri John Ledkov
** Also affects: linux-azure (Ubuntu) Importance: Undecided Status: New ** Changed in: linux-azure (Ubuntu) Milestone: None => ubuntu-21.04 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1919177] Re: Azure: issues with accelerated networking on Hirsute

2021-04-13 Thread Robert C Jennings
** Also affects: cloud-init (Ubuntu) Importance: Undecided Status: New ** Changed in: cloud-init (Ubuntu) Milestone: None => ubuntu-21.04 ** Tags added: rls-hh-incoming -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.