[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

*** This bug is a duplicate of bug 1813244 *** https://bugs.launchpad.net/bugs/1813244 ** Tags added: cscc -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: openvswitch: kernel oops

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

*** This bug is a duplicate of bug 1813244 *** https://bugs.launchpad.net/bugs/1813244 ** This bug has been marked a duplicate of bug 1813244 systemd cause kernel trace "BUG: unable to handle kernel paging request at 6db23a14" on Cosmic i386 -- You received this bug notification because

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I've done a test with the fix from bug #1813244 and the problem doesn't seem to happen. Probably a duplicate bug. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: openvswitch: kernel

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

There is an openvswitch related issue, bug 1813244. Perhaps these two are identical? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: openvswitch: kernel oops destroying interfaces on

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

** Description changed: + == SRU Justification == + + Commit 120645513f55 ("openvswitch: Add eventmask support to CT action.") + introduced a regression on i386. Simply running the following commands + in a loop will trigger a crash rather quickly: + + ovs-vsctl add-br test + ovs-vsctl del-br

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

https://lore.kernel.org/lkml/20190305104010.6342e9b9@gollum/ ** Also affects: linux (Ubuntu Disco) Importance: High Status: In Progress ** Also affects: openvswitch (Ubuntu Disco) Importance: Undecided Status: Invalid -- You received this bug notification because you are a

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Hmm, so are we giving up on this? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: openvswitch: kernel oops destroying interfaces on i386 To manage notifications about this bug go to:

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

** Changed in: linux (Ubuntu Bionic) Assignee: Joseph Salisbury (jsalisbury) => (unassigned) ** Changed in: linux (Ubuntu Cosmic) Assignee: Joseph Salisbury (jsalisbury) => (unassigned) ** Changed in: linux (Ubuntu Artful) Assignee: Joseph Salisbury (jsalisbury) => (unassigned)

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Repro crash with the case - still triggering Installed 32bit Test kernel It boots this one: Linux 4.15.0-36-generic #40 SMP Fri Oct 12 00:17:54 UTC 2018 Seems to have no "special" version suffix to identify it other than #40 and build time. But #40 and the build time indicate this is the

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I built a i386 version of the Bionic test kernel with a patch from upstream. The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1736390 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Hi Joseph, I'm back from my PTO, but have to ask you again for an update - as in the past I'll need 32bit kernels for this test -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title:

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Note to myself test instructions carried from my original bug Update Kernel: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1712831/comments/12 and sudo qemu-system-i386 -hda autopkgtest-bionic-i386.img -enable-kvm -nographic -curses -m 4096 Test: sudo autopkgtest --shell-fail

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I built a Bionic test kernel with a patch from upstream. The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1736390 Can you test this kernel and see if it resolves this bug? -- You received this bug notification because you are a member of Ubuntu Bugs, which is

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

That should be good. I just like to have the latest mainline already tested in case upstream asks for it. I'll ping upstream and see what the next steps should be. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Hi Joseph, due to some maas accident I got my test system destroyed by a coworker. I tested v4.19-rc3 as I wrote in comment #51 - do you mind accepting that as a valid "test latest mainline" even thou it was not -rc4 as it would be now? -- You received this bug notification because you are a

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Could you give the latest mainline kernel a test before I ping upstream? It is available from: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.19-rc4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Thanks, installed that in the test env, after a manual reboot I got: $ uname -a Linux autopkgtest 4.15.0-34-generic #38~lp1736390Commit12064551Reverted SMP Thu Sep 13 13:28:33 UTC i686 i686 i686 GNU/Linux The change is persistent into the autopkgtest: autopkgtest [05:11:17]: testbed running

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

There is a 32bit kernel now posted to: http://kernel.ubuntu.com/~jsalisbury/lp1736390 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: openvswitch: kernel oops destroying interfaces on

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

While the tests that run on autopkgtest infra suggest it still is an issue, I first I tried to reproduce as-is to be sure the trigger is good (new release, new kernel, new OVS): $ autopkgtest --apt-upgrade --shell --no-built-binaries openvswitch_2.9.0-0ubuntu1.dsc -- qemu

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I built a Bionic test kernel with a revert of commit 120645513f55. The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1736390 Can you test this kernel and see if it resolves this bug? Note about installing test kernels: • If the test kernel is prior to 4.15(Bionic)

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

** Also affects: linux (Ubuntu Cosmic) Importance: High Assignee: Joseph Salisbury (jsalisbury) Status: In Progress ** Also affects: openvswitch (Ubuntu Cosmic) Importance: Undecided Status: Invalid -- You received this bug notification because you are a member of

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Hi Joseph, neither me nor James have realized that this waited for a retest on our side. The old kernel that you had linked is gone by now (together with Artful I assume) :-/ Would you mind prepping a new test kernel of your choice (This still is an issue in cosmic, so whatever works best for

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

It looks like only commit 120645513f55 would need to be reverted in v4.13.0-38. Can you test the following test kernel: http://kernel.ubuntu.com/~jsalisbury/lp1736390/ Be sure to install both the linux-image and linux-image-extra .deb packages. -- You received this bug notification because

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I'm still working on the bisect. It looks like two commits may have introduced the bug, which is slowing the bisect process. I found that the following commit causes a kernel trace with rtmsg_ifa as the EIP: 120645513f55 ("openvswitch: Add eventmask support to CT action.") Reverting this

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I'm still in the process of bisecting this. Some of the testing that did not exhibit the bug, does sometimes. I need to run through the prior testing I did to confirm whether the kernels were really good or bad. -- You received this bug notification because you are a member of Ubuntu Bugs,

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

The first bisect report the following commit as the first bad commit: 2f34c1231bfc ("Merge tag 'drm-for-v4.12' of git://people.freedesktop.org/~airlied/linux") However, because this commit is a merge, I have to perform another round of bisecting between this commits parents. I'll have another

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Marking OVS tasks as invalid as we think this issue is in the kernel. ** Changed in: openvswitch (Ubuntu Artful) Status: Confirmed => Invalid ** Changed in: openvswitch (Ubuntu Bionic) Status: Confirmed => Invalid -- You received this bug notification because you are a member of

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I'm still working on bisecting the issue down. I should have another update in a day or two. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: openvswitch: kernel oops destroying

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

@Joseph Any update on your bisecting? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: openvswitch: kernel oops destroying interfaces on i386 To manage notifications about this bug go

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

After some more testing, it appears v4.11 final does not have the bug. Version 4.12-rc1 appears to be the first kernel version to hit the bug. I'll run a few more tests to confirm this. Once confirmed, I'll start a bisect between v4.11 and v4.12-rc1. If others want to try these two versions,

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

The 4.10 based kernel does not experience this hang, but I am seeing it with 4.12. Maybe the 4.12 based kernels does in fact have the bug. I'll test further to find out. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I'm swapping the bug back in and working on the bisect again. For some reason, all of the prior kernels that worked, are now freezing on the test while "Stopping Links": *** Configuring hosts h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 *** Starting controller c1 *** Starting 2 switches s1

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Thanks for the feedback, Christian! This gives me enough to be able to try and bisect this issue down now. I'll post an update shortly. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title:

Re: [Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

On Wed, Jan 10, 2018 at 5:45 PM, Joseph Salisbury wrote: > Thanks for the feedback, Christian. I would much rather install them > manually. I am able to do that without a problem. However, I am unable > to access the GRUB menu in the usual way to select a

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Thanks for the feedback, Christian. I would much rather install them manually. I am able to do that without a problem. However, I am unable to access the GRUB menu in the usual way to select a specific kernel. I tried all the usual way, holding shift, modifying /etc/default/grub setting, but

Re: [Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

On Fri, Jan 5, 2018 at 9:27 PM, Joseph Salisbury wrote: > I'm able to install a kernel using kvm: > kvm -m 512 -hda ./autopkgtest-artful-i386.img If that works for you fine. > Is there a way to modify the autopkgtest command line to tell it which > kernel to

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I'm able to install a kernel using kvm: kvm -m 512 -hda ./autopkgtest-artful-i386.img Is there a way to modify the autopkgtest command line to tell it which kernel to boot? When I have multiple test kernels installed, I usually just select them from the GRUB menu. What would be the equivalent

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

After running qemu-nbd: ls /dev/nbd* nbd0 nbd1 nbd10 nbd11 nbd12 nbd13 nbd14 nbd15 nbd2 nbd3 nbd4 nbd5 nbd6 nbd7 nbd8 nbd9 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I'm running into an issue mounting the image to get a kernel onto it. For some reason, I cannot mount /dev/nbd0p1: sudo modprobe nbd sudo qemu-nbd --connect=/dev/nbd0 ./autopkgtest-artful-i386.img sudo mount /dev/nbd0p1 /mnt/test/ mount: /mnt/test: special device /dev/nbd0p1 does not exist.

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

@Christina, Thanks for the info. I am now able to reproduce the bug using your steps. Thanks again for that! I'm going to dig deeper into this and see if I can bisect down to the commit that caused this. I next need to figure out how to swap in different test kernels. I think you explained

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

@Joseph - if you mean 2nd level so: Machine ->KVM -> $ autopkgtest-buildvm-ubuntu-cloud -a i386 -r artful -s 10G That could be an issue - for 2nd level being famous for only working mostly. But why would you do so - since the tests are in VMs they are already more or less host release agnostic.

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I'm attempting to reproduce the bug. However, I get a hang when I run the following: $ autopkgtest-buildvm-ubuntu-cloud -a i386 -r artful -s 10G Attached is a screen shot of the hang. It seems to be trying to boot the image. ** Attachment added: "hang.jpg"

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

Also, I'm trying to run this on a VM instead of bare hardware. Could that be an issue? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: openvswitch: kernel oops destroying interfaces

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

I tried to build a test kernel with the patch set from Cong mentioned in comment #18. However, there was a build failure due to some missing prereq commits. I also had to backport many of the patches, since they were not clean cherry picks. Per comment 16, the bug did not start happening until

[Bug 1736390] Re: openvswitch: kernel oops destroying interfaces on i386

** Summary changed: - openvswitch: kernel opps destroying interfaces on i386 + openvswitch: kernel oops destroying interfaces on i386 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736390 Title: