Re: virtio-net: kernel panic in virtio_net.c
On Sat, 9 Oct 2021 07:19:39 +0200, Greg KH wrote:
> On Sat, Oct 09, 2021 at 12:27:08AM +0800, Xuan Zhuo wrote:
> > On Fri, 8 Oct 2021 10:06:57 +0200, Greg KH
> > wrote:
> > > On Fri, Oct 08, 2021 at 12:17:26AM +0800, Xuan Zhuo wrote:
> > > > On Thu, 7 Oct 2021 17:25:02 +0200, Greg KH
> > > > wrote:
> > > > > On Thu, Oct 07, 2021 at 11:06:12PM +0800, Xuan Zhuo wrote:
> > > > > > On Thu, 07 Oct 2021 14:04:22 +0200, Corentin Noël
> > > > > > wrote:
> > > > > > > I've been experiencing crashes with 5.14-rc1 and above that do not
> > > > > > > occur with 5.13,
> > > > > >
> > > > > > I should have fixed this problem before. I don't know why, I just
> > > > > > looked at the
> > > > > > latest net code, and this commit seems to be lost.
> > > > > >
> > > > > > 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for
> > > > > > skb_over_panic inside big mode
> > > > > >
> > > > > > Can you test this patch again?
> > > > >
> > > > > That commit showed up in 5.13-rc5, so 5.14-rc1 and 5.13 should have
> > > > > had
> > > > > it in it, right?
> > > > >
> > > >
> > > > Yes, it may be lost due to conflicts during a certain merge.
> > >
> > > Really? I tried to apply that again to 5.14 and it did not work. So I
> > > do not understand what to do here, can you try to explain it better?
> >
> > I took a look, and there is actually another missing patch:
> >
> > A. 8fb7da9e990793299c89ed7a4281c235bfdd31f8 virtio_net: get build_skb() buf
> > by data ptr
> > B. 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for
> > skb_over_panic inside big mode
> >
> > A is replaced by another patch:
> >
> > commit c32325b8fdf2f979befb9fd5587918c0d5412db3
> > Author: Jakub Kicinski
> > Date: Mon Aug 2 10:57:29 2021 -0700
> >
> > virtio-net: realign page_to_skb() after merges
> >
> > We ended up merging two versions of the same patch set:
> >
> > commit 8fb7da9e9907 ("virtio_net: get build_skb() buf by data ptr")
> > commit 5c37711d9f27 ("virtio-net: fix for unable to handle page
> > fault for address")
> >
> > into net, and
> >
> > commit 7bf64460e3b2 ("virtio-net: get build_skb() buf by data ptr")
> > commit 6c66c147b9a4 ("virtio-net: fix for unable to handle page
> > fault for address")
> >
> > into net-next. Redo the merge from commit 126285651b7f ("Merge
> > ra.kernel.org:/pub/scm/linux/kernel/git/netdev/net"), so that
> > the most recent code remains.
> >
> > Acked-by: Michael S. Tsirkin
> > Signed-off-by: Jakub Kicinski
> > Acked-by: Jason Wang
> > Signed-off-by: David S. Miller
> >
> > So after this patch, patch B can be applied normally.
> >
> > So on the latest net branch, only lost
> >
> > 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for
> > skb_over_panic inside big mode
>
> Again, I do not know what to do here, can you submit the needed fix to
> the networking developers so this gets fixed?
Michael has already submitted the patch.
https://lore.kernel.org/netdev/20211009091604.84141-1-...@redhat.com/T/#u
Thanks.
>
> thanks,
>
> greg k-h
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Sat, Oct 09, 2021 at 12:27:08AM +0800, Xuan Zhuo wrote:
> On Fri, 8 Oct 2021 10:06:57 +0200, Greg KH wrote:
> > On Fri, Oct 08, 2021 at 12:17:26AM +0800, Xuan Zhuo wrote:
> > > On Thu, 7 Oct 2021 17:25:02 +0200, Greg KH
> > > wrote:
> > > > On Thu, Oct 07, 2021 at 11:06:12PM +0800, Xuan Zhuo wrote:
> > > > > On Thu, 07 Oct 2021 14:04:22 +0200, Corentin Noël
> > > > > wrote:
> > > > > > I've been experiencing crashes with 5.14-rc1 and above that do not
> > > > > > occur with 5.13,
> > > > >
> > > > > I should have fixed this problem before. I don't know why, I just
> > > > > looked at the
> > > > > latest net code, and this commit seems to be lost.
> > > > >
> > > > > 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for
> > > > > skb_over_panic inside big mode
> > > > >
> > > > > Can you test this patch again?
> > > >
> > > > That commit showed up in 5.13-rc5, so 5.14-rc1 and 5.13 should have had
> > > > it in it, right?
> > > >
> > >
> > > Yes, it may be lost due to conflicts during a certain merge.
> >
> > Really? I tried to apply that again to 5.14 and it did not work. So I
> > do not understand what to do here, can you try to explain it better?
>
> I took a look, and there is actually another missing patch:
>
> A. 8fb7da9e990793299c89ed7a4281c235bfdd31f8 virtio_net: get build_skb() buf
> by data ptr
> B. 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for
> skb_over_panic inside big mode
>
> A is replaced by another patch:
>
> commit c32325b8fdf2f979befb9fd5587918c0d5412db3
> Author: Jakub Kicinski
> Date: Mon Aug 2 10:57:29 2021 -0700
>
> virtio-net: realign page_to_skb() after merges
>
> We ended up merging two versions of the same patch set:
>
> commit 8fb7da9e9907 ("virtio_net: get build_skb() buf by data ptr")
> commit 5c37711d9f27 ("virtio-net: fix for unable to handle page
> fault for address")
>
> into net, and
>
> commit 7bf64460e3b2 ("virtio-net: get build_skb() buf by data ptr")
> commit 6c66c147b9a4 ("virtio-net: fix for unable to handle page
> fault for address")
>
> into net-next. Redo the merge from commit 126285651b7f ("Merge
> ra.kernel.org:/pub/scm/linux/kernel/git/netdev/net"), so that
> the most recent code remains.
>
> Acked-by: Michael S. Tsirkin
> Signed-off-by: Jakub Kicinski
> Acked-by: Jason Wang
> Signed-off-by: David S. Miller
>
> So after this patch, patch B can be applied normally.
>
> So on the latest net branch, only lost
>
> 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for
> skb_over_panic inside big mode
Again, I do not know what to do here, can you submit the needed fix to
the networking developers so this gets fixed?
thanks,
greg k-h
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Fri, 8 Oct 2021 10:06:57 +0200, Greg KH wrote:
> On Fri, Oct 08, 2021 at 12:17:26AM +0800, Xuan Zhuo wrote:
> > On Thu, 7 Oct 2021 17:25:02 +0200, Greg KH
> > wrote:
> > > On Thu, Oct 07, 2021 at 11:06:12PM +0800, Xuan Zhuo wrote:
> > > > On Thu, 07 Oct 2021 14:04:22 +0200, Corentin Noël
> > > > wrote:
> > > > > I've been experiencing crashes with 5.14-rc1 and above that do not
> > > > > occur with 5.13,
> > > >
> > > > I should have fixed this problem before. I don't know why, I just
> > > > looked at the
> > > > latest net code, and this commit seems to be lost.
> > > >
> > > > 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for
> > > > skb_over_panic inside big mode
> > > >
> > > > Can you test this patch again?
> > >
> > > That commit showed up in 5.13-rc5, so 5.14-rc1 and 5.13 should have had
> > > it in it, right?
> > >
> >
> > Yes, it may be lost due to conflicts during a certain merge.
>
> Really? I tried to apply that again to 5.14 and it did not work. So I
> do not understand what to do here, can you try to explain it better?
I took a look, and there is actually another missing patch:
A. 8fb7da9e990793299c89ed7a4281c235bfdd31f8 virtio_net: get build_skb() buf by
data ptr
B. 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for skb_over_panic
inside big mode
A is replaced by another patch:
commit c32325b8fdf2f979befb9fd5587918c0d5412db3
Author: Jakub Kicinski
Date: Mon Aug 2 10:57:29 2021 -0700
virtio-net: realign page_to_skb() after merges
We ended up merging two versions of the same patch set:
commit 8fb7da9e9907 ("virtio_net: get build_skb() buf by data ptr")
commit 5c37711d9f27 ("virtio-net: fix for unable to handle page
fault for address")
into net, and
commit 7bf64460e3b2 ("virtio-net: get build_skb() buf by data ptr")
commit 6c66c147b9a4 ("virtio-net: fix for unable to handle page
fault for address")
into net-next. Redo the merge from commit 126285651b7f ("Merge
ra.kernel.org:/pub/scm/linux/kernel/git/netdev/net"), so that
the most recent code remains.
Acked-by: Michael S. Tsirkin
Signed-off-by: Jakub Kicinski
Acked-by: Jason Wang
Signed-off-by: David S. Miller
So after this patch, patch B can be applied normally.
So on the latest net branch, only lost
1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for
skb_over_panic inside big mode
Thanks.
>
> thanks,
>
> greg k-h
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Fri, Oct 08, 2021 at 10:06:57AM +0200, Greg KH wrote: > On Fri, Oct 08, 2021 at 12:17:26AM +0800, Xuan Zhuo wrote: > > On Thu, 7 Oct 2021 17:25:02 +0200, Greg KH > > wrote: > > > On Thu, Oct 07, 2021 at 11:06:12PM +0800, Xuan Zhuo wrote: > > > > On Thu, 07 Oct 2021 14:04:22 +0200, Corentin Noël > > > > wrote: > > > > > I've been experiencing crashes with 5.14-rc1 and above that do not > > > > > occur with 5.13, > > > > > > > > I should have fixed this problem before. I don't know why, I just > > > > looked at the > > > > latest net code, and this commit seems to be lost. > > > > > > > > 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for > > > > skb_over_panic inside big mode > > > > > > > > Can you test this patch again? > > > > > > That commit showed up in 5.13-rc5, so 5.14-rc1 and 5.13 should have had > > > it in it, right? > > > > > > > Yes, it may be lost due to conflicts during a certain merge. > > Really? I tried to apply that again to 5.14 and it did not work. So I > do not understand what to do here, can you try to explain it better? > > thanks, > > greg k-h Hmm, something like the following perhaps then? Corentin would you like to try this? Warning: untested. diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 096c2ac6b7a6..18dd9f6d107d 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -406,12 +406,13 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi, * add_recvbuf_mergeable() + get_mergeable_buf_len() */ truesize = headroom ? PAGE_SIZE : truesize; - tailroom = truesize - len - headroom; + tailroom = truesize - headroom; buf = p - headroom; len -= hdr_len; offset += hdr_padded_len; p += hdr_padded_len; + tailroom -= hdr_padded_len + len; shinfo_size = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Fri, Oct 08, 2021 at 12:17:26AM +0800, Xuan Zhuo wrote: > On Thu, 7 Oct 2021 17:25:02 +0200, Greg KH wrote: > > On Thu, Oct 07, 2021 at 11:06:12PM +0800, Xuan Zhuo wrote: > > > On Thu, 07 Oct 2021 14:04:22 +0200, Corentin Noël > > > wrote: > > > > I've been experiencing crashes with 5.14-rc1 and above that do not > > > > occur with 5.13, > > > > > > I should have fixed this problem before. I don't know why, I just looked > > > at the > > > latest net code, and this commit seems to be lost. > > > > > > 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for > > > skb_over_panic inside big mode > > > > > > Can you test this patch again? > > > > That commit showed up in 5.13-rc5, so 5.14-rc1 and 5.13 should have had > > it in it, right? > > > > Yes, it may be lost due to conflicts during a certain merge. Really? I tried to apply that again to 5.14 and it did not work. So I do not understand what to do here, can you try to explain it better? thanks, greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Thu, 7 Oct 2021 17:25:02 +0200, Greg KH wrote: > On Thu, Oct 07, 2021 at 11:06:12PM +0800, Xuan Zhuo wrote: > > On Thu, 07 Oct 2021 14:04:22 +0200, Corentin Noël > > wrote: > > > I've been experiencing crashes with 5.14-rc1 and above that do not > > > occur with 5.13, > > > > I should have fixed this problem before. I don't know why, I just looked at > > the > > latest net code, and this commit seems to be lost. > > > > 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for > > skb_over_panic inside big mode > > > > Can you test this patch again? > > That commit showed up in 5.13-rc5, so 5.14-rc1 and 5.13 should have had > it in it, right? > Yes, it may be lost due to conflicts during a certain merge. Thanks. > thanks, > > greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Thu, Oct 07, 2021 at 11:06:12PM +0800, Xuan Zhuo wrote: > On Thu, 07 Oct 2021 14:04:22 +0200, Corentin Noël > wrote: > > I've been experiencing crashes with 5.14-rc1 and above that do not > > occur with 5.13, > > I should have fixed this problem before. I don't know why, I just looked at > the > latest net code, and this commit seems to be lost. > > 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for > skb_over_panic inside big mode > > Can you test this patch again? That commit showed up in 5.13-rc5, so 5.14-rc1 and 5.13 should have had it in it, right? thanks, greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Thu, 07 Oct 2021 14:04:22 +0200, Corentin Noël wrote: > I've been experiencing crashes with 5.14-rc1 and above that do not > occur with 5.13, I should have fixed this problem before. I don't know why, I just looked at the latest net code, and this commit seems to be lost. 1a8024239dacf53fcf39c0f07fbf2712af22864f virtio-net: fix for skb_over_panic inside big mode Can you test this patch again? Thanks. > > here is the crash trace: > > [ 61.346677] skbuff: skb_over_panic: text:881ae2c7 len:3762 > put:3762 head:8a5ec8c22000 data:8a5ec8c22010 tail:0xec2 > end:0xec0 dev: > [ 61.369192] kernel BUG at net/core/skbuff.c:111! > [ 61.372840] invalid opcode: [#1] SMP PTI > [ 61.374892] CPU: 5 PID: 0 Comm: swapper/5 Not tainted 5.14.0- > rc1linux-v5.14-rc1-for-mesa-ci.tar.bz2 #1 > [ 61.376450] Hardware name: ChromiumOS crosvm, BIOS 0 > [ 61.377222] RIP: 0010:skb_panic+0x43/0x45 > [ 61.377833] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 50 > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 18 f1 cf 88 e8 6a 43 fb > ff <0f> 0b 48 8b 14 24 48 c7 c1 20 35 b1 88 e8 ab ff ff ff 48 c7 c6 60 > [ 61.380566] RSP: 0018:ae258017cce0 EFLAGS: 00010246 > [ 61.381267] RAX: 008b RBX: 0010 RCX: > dfff > [ 61.382246] RDX: RSI: ffea RDI: > > [ 61.383376] RBP: de6a80230880 R08: 88f45568 R09: > 9ffb > [ 61.384494] R10: e000 R11: 3fff R12: > 8a5ec7461200 > [ 61.385696] R13: 8a5ec8c22000 R14: R15: > 0eb2 > [ 61.386825] FS: () GS:8a5febd4() > knlGS: > [ 61.388055] CS: 0010 DS: ES: CR0: 80050033 > [ 61.389221] CR2: 0148a060 CR3: 00011ae0e005 CR4: > 00370ee0 > [ 61.390871] DR0: DR1: DR2: > > [ 61.392335] DR3: DR6: fffe0ff0 DR7: > 0400 > [ 61.393635] Call Trace: > [ 61.394127] > [ 61.394488] skb_put.cold+0x10/0x10 > [ 61.395095] page_to_skb+0xf7/0x410 > [ 61.395689] receive_buf+0x81/0x1660 > [ 61.396228] ? netif_receive_skb_list_internal+0x1ad/0x2b0 > [ 61.397180] ? napi_gro_flush+0x97/0xe0 > [ 61.397896] ? detach_buf_split+0x67/0x120 > [ 61.398573] virtnet_poll+0x2cf/0x420 > [ 61.399197] __napi_poll+0x25/0x150 > [ 61.399764] net_rx_action+0x22f/0x280 > [ 61.400394] __do_softirq+0xba/0x257 > [ 61.401012] irq_exit_rcu+0x8e/0xb0 > [ 61.401618] common_interrupt+0x7b/0xa0 > [ 61.402270] > [ 61.402620] asm_common_interrupt+0x1e/0x40 > [ 61.403302] RIP: 0010:default_idle+0xb/0x10 > [ 61.404018] Code: 8b 04 25 00 6d 01 00 f0 80 60 02 df c3 0f ae f0 0f > ae 38 0f ae f0 eb b9 0f 1f 80 00 00 00 00 eb 07 0f 00 2d df 3e 44 00 fb > f4 cc cc cc cc 65 8b 15 31 2f a4 77 89 d2 48 8b 05 d0 a1 0c 01 48 > [ 61.407636] RSP: 0018:ae258008fef8 EFLAGS: 0202 > [ 61.408394] RAX: 885ce620 RBX: 0005 RCX: > 8a5febd56f80 > [ 61.409451] RDX: 00c1ec32 RSI: 7ff1b7a1e726 RDI: > 8a5febd5dd00 > [ 61.410530] RBP: 8a5fc01f8000 R08: 00c1ec32 R09: > > [ 61.411715] R10: 0006 R11: 0002 R12: > > [ 61.412984] R13: R14: R15: > > [ 61.414183] ? mwait_idle+0x70/0x70 > [ 61.414805] ? mwait_idle+0x70/0x70 > [ 61.415592] default_idle_call+0x2a/0xa0 > [ 61.416216] do_idle+0x1e8/0x250 > [ 61.416722] cpu_startup_entry+0x14/0x20 > [ 61.417347] secondary_startup_64_no_verify+0xc2/0xcb > [ 61.418144] Modules linked in: > [ 61.418622] ---[ end trace 3741c3e580a52bbd ]--- > [ 61.419399] RIP: 0010:skb_panic+0x43/0x45 > [ 61.420054] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 50 > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 18 f1 cf 88 e8 6a 43 fb > ff <0f> 0b 48 8b 14 24 48 c7 c1 20 35 b1 88 e8 ab ff ff ff 48 c7 c6 60 > [ 61.422606] RSP: 0018:ae258017cce0 EFLAGS: 00010246 > [ 61.423865] RAX: 008b RBX: 0010 RCX: > dfff > [ 61.425031] RDX: RSI: ffea RDI: > > [ 61.426229] RBP: de6a80230880 R08: 88f45568 R09: > 9ffb > [ 61.427439] R10: e000 R11: 3fff R12: > 8a5ec7461200 > [ 61.428615] R13: 8a5ec8c22000 R14: R15: > 0eb2 > [ 61.429799] FS: () GS:8a5febd4() > knlGS: > [ 61.431048] CS: 0010 DS: ES: CR0: 80050033 > [ 61.431997] CR2: 0148a060 CR3: 00011ae0e005 CR4: > 00370ee0 > [ 61.433206] DR0: DR1: DR2: > > [ 61.434502] DR3: DR6: fffe0ff0 DR7: > 0400 > [
Re: virtio-net: kernel panic in virtio_net.c
On Thu, Oct 07, 2021 at 04:02:10PM +0200, Corentin Noël wrote: > Le jeudi 07 octobre 2021 à 06:51 -0700, Eric Dumazet a écrit : > > On Thu, Oct 7, 2021 at 6:11 AM Michael S. Tsirkin > > wrote: > > > On Thu, Oct 07, 2021 at 02:04:22PM +0200, Corentin Noël wrote: > > > > I've been experiencing crashes with 5.14-rc1 and above that do > > > > not > > > > occur with 5.13, > > > > What about 5.14 ? > > > > 5.14-rc1 has many bugs we do not want to spend time rediscovering > > them... > > > > I've tested on 5.14, 5.15-rc4 and 5.15-rc4 with latest netdev and could > reproduce the crash on them all. Great, any chance you can use 'git bisect' to find the offending commit? thanks, greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Thu, Oct 07, 2021 at 02:04:22PM +0200, Corentin Noël wrote: > I've been experiencing crashes with 5.14-rc1 and above that do not > occur with 5.13, > > here is the crash trace: > > [ 61.346677] skbuff: skb_over_panic: text:881ae2c7 len:3762 > put:3762 head:8a5ec8c22000 data:8a5ec8c22010 tail:0xec2 > end:0xec0 dev: > [ 61.369192] kernel BUG at net/core/skbuff.c:111! > [ 61.372840] invalid opcode: [#1] SMP PTI > [ 61.374892] CPU: 5 PID: 0 Comm: swapper/5 Not tainted 5.14.0- > rc1linux-v5.14-rc1-for-mesa-ci.tar.bz2 #1 > [ 61.376450] Hardware name: ChromiumOS crosvm, BIOS 0 > [ 61.377222] RIP: 0010:skb_panic+0x43/0x45 > [ 61.377833] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 50 > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 18 f1 cf 88 e8 6a 43 fb > ff <0f> 0b 48 8b 14 24 48 c7 c1 20 35 b1 88 e8 ab ff ff ff 48 c7 c6 60 > [ 61.380566] RSP: 0018:ae258017cce0 EFLAGS: 00010246 > [ 61.381267] RAX: 008b RBX: 0010 RCX: > dfff > [ 61.382246] RDX: RSI: ffea RDI: > > [ 61.383376] RBP: de6a80230880 R08: 88f45568 R09: > 9ffb > [ 61.384494] R10: e000 R11: 3fff R12: > 8a5ec7461200 > [ 61.385696] R13: 8a5ec8c22000 R14: R15: > 0eb2 > [ 61.386825] FS: () GS:8a5febd4() > knlGS: > [ 61.388055] CS: 0010 DS: ES: CR0: 80050033 > [ 61.389221] CR2: 0148a060 CR3: 00011ae0e005 CR4: > 00370ee0 > [ 61.390871] DR0: DR1: DR2: > > [ 61.392335] DR3: DR6: fffe0ff0 DR7: > 0400 > [ 61.393635] Call Trace: > [ 61.394127] > [ 61.394488] skb_put.cold+0x10/0x10 > [ 61.395095] page_to_skb+0xf7/0x410 > [ 61.395689] receive_buf+0x81/0x1660 > [ 61.396228] ? netif_receive_skb_list_internal+0x1ad/0x2b0 > [ 61.397180] ? napi_gro_flush+0x97/0xe0 > [ 61.397896] ? detach_buf_split+0x67/0x120 > [ 61.398573] virtnet_poll+0x2cf/0x420 > [ 61.399197] __napi_poll+0x25/0x150 > [ 61.399764] net_rx_action+0x22f/0x280 > [ 61.400394] __do_softirq+0xba/0x257 > [ 61.401012] irq_exit_rcu+0x8e/0xb0 > [ 61.401618] common_interrupt+0x7b/0xa0 > [ 61.402270] > [ 61.402620] asm_common_interrupt+0x1e/0x40 > [ 61.403302] RIP: 0010:default_idle+0xb/0x10 > [ 61.404018] Code: 8b 04 25 00 6d 01 00 f0 80 60 02 df c3 0f ae f0 0f > ae 38 0f ae f0 eb b9 0f 1f 80 00 00 00 00 eb 07 0f 00 2d df 3e 44 00 fb > f4 cc cc cc cc 65 8b 15 31 2f a4 77 89 d2 48 8b 05 d0 a1 0c 01 48 > [ 61.407636] RSP: 0018:ae258008fef8 EFLAGS: 0202 > [ 61.408394] RAX: 885ce620 RBX: 0005 RCX: > 8a5febd56f80 > [ 61.409451] RDX: 00c1ec32 RSI: 7ff1b7a1e726 RDI: > 8a5febd5dd00 > [ 61.410530] RBP: 8a5fc01f8000 R08: 00c1ec32 R09: > > [ 61.411715] R10: 0006 R11: 0002 R12: > > [ 61.412984] R13: R14: R15: > > [ 61.414183] ? mwait_idle+0x70/0x70 > [ 61.414805] ? mwait_idle+0x70/0x70 > [ 61.415592] default_idle_call+0x2a/0xa0 > [ 61.416216] do_idle+0x1e8/0x250 > [ 61.416722] cpu_startup_entry+0x14/0x20 > [ 61.417347] secondary_startup_64_no_verify+0xc2/0xcb > [ 61.418144] Modules linked in: > [ 61.418622] ---[ end trace 3741c3e580a52bbd ]--- > [ 61.419399] RIP: 0010:skb_panic+0x43/0x45 > [ 61.420054] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 50 > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 18 f1 cf 88 e8 6a 43 fb > ff <0f> 0b 48 8b 14 24 48 c7 c1 20 35 b1 88 e8 ab ff ff ff 48 c7 c6 60 > [ 61.422606] RSP: 0018:ae258017cce0 EFLAGS: 00010246 > [ 61.423865] RAX: 008b RBX: 0010 RCX: > dfff > [ 61.425031] RDX: RSI: ffea RDI: > > [ 61.426229] RBP: de6a80230880 R08: 88f45568 R09: > 9ffb > [ 61.427439] R10: e000 R11: 3fff R12: > 8a5ec7461200 > [ 61.428615] R13: 8a5ec8c22000 R14: R15: > 0eb2 > [ 61.429799] FS: () GS:8a5febd4() > knlGS: > [ 61.431048] CS: 0010 DS: ES: CR0: 80050033 > [ 61.431997] CR2: 0148a060 CR3: 00011ae0e005 CR4: > 00370ee0 > [ 61.433206] DR0: DR1: DR2: > > [ 61.434502] DR3: DR6: fffe0ff0 DR7: > 0400 > [ 61.435799] Kernel panic - not syncing: Fatal exception in interrupt > [ 61.439250] Kernel Offset: 0x6a0 from 0x8100 > (relocation range: 0x8000-0xbfff) > > Here is my kernel config: >
Re: virtio-net: kernel panic in virtio_net.c
On Wed, 9 Jun 2021 10:03:53 +0200, Greg KH wrote:
> On Wed, Jun 09, 2021 at 03:51:20PM +0800, Xuan Zhuo wrote:
> > On Wed, 9 Jun 2021 08:24:20 +0200, Greg KH
> > wrote:
> > > On Wed, Jun 09, 2021 at 02:08:17PM +0800, Xuan Zhuo wrote:
> > > > On Wed, 9 Jun 2021 06:50:10 +0200, Greg KH
> > > > wrote:
> > > > > On Wed, Jun 09, 2021 at 09:48:33AM +0800, Xuan Zhuo wrote:
> > > > > > > > With this patch and the latest net branch I no longer get
> > > > > > > > crashes.
> > > > > > >
> > > > > > > Did this ever get properly submitted to the networking tree to
> > > > > > > get into
> > > > > > > 5.13-final?
> > > > > >
> > > > > > The patch has been submitted.
> > > > > >
> > > > > > [PATCH net] virtio-net: fix for skb_over_panic inside big mode
> > > > >
> > > > > Submitted where? Do you have a lore.kernel.org link somewhere?
> > > >
> > > >
> > > > https://lore.kernel.org/netdev/20210603170901.66504-1-xuanz...@linux.alibaba.com/
> > >
> > > So this is commit 1a8024239dac ("virtio-net: fix for skb_over_panic
> > > inside big mode") in Linus's tree, right?
> >
> > YES.
> >
> > >
> > > But why is that referencing:
> > > Fixes: fb32856b16ad ("virtio-net: page_to_skb() use build_skb when
> > > there's sufficient tailroom")
> >
> > This problem was indeed introduced in fb32856b16ad.
> >
> > I confirmed that this commit fb32856b16ad was first entered in 5.13-rc1,
> > and the
> > previous 5.12 did not have this commit fb32856b16ad.
> >
> > I'm not sure if it helped you.
>
> Hm, then what resolves the reported problem that people were having with
> the 5.12.y kernel release? Is that a separate issue?
Has anyone reported a problem with 5.12.y? I don’t seem to see it. Corentin
only reported a problem with 5.13? Did I miss something?
I confirm that 5.12.9 has no modification of fb32856b16ad.
Thanks.
>
> thanks,
>
> greg k-h
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Wed, Jun 09, 2021 at 03:51:20PM +0800, Xuan Zhuo wrote:
> On Wed, 9 Jun 2021 08:24:20 +0200, Greg KH wrote:
> > On Wed, Jun 09, 2021 at 02:08:17PM +0800, Xuan Zhuo wrote:
> > > On Wed, 9 Jun 2021 06:50:10 +0200, Greg KH
> > > wrote:
> > > > On Wed, Jun 09, 2021 at 09:48:33AM +0800, Xuan Zhuo wrote:
> > > > > > > With this patch and the latest net branch I no longer get crashes.
> > > > > >
> > > > > > Did this ever get properly submitted to the networking tree to get
> > > > > > into
> > > > > > 5.13-final?
> > > > >
> > > > > The patch has been submitted.
> > > > >
> > > > > [PATCH net] virtio-net: fix for skb_over_panic inside big mode
> > > >
> > > > Submitted where? Do you have a lore.kernel.org link somewhere?
> > >
> > >
> > > https://lore.kernel.org/netdev/20210603170901.66504-1-xuanz...@linux.alibaba.com/
> >
> > So this is commit 1a8024239dac ("virtio-net: fix for skb_over_panic
> > inside big mode") in Linus's tree, right?
>
> YES.
>
> >
> > But why is that referencing:
> > Fixes: fb32856b16ad ("virtio-net: page_to_skb() use build_skb when
> > there's sufficient tailroom")
>
> This problem was indeed introduced in fb32856b16ad.
>
> I confirmed that this commit fb32856b16ad was first entered in 5.13-rc1, and
> the
> previous 5.12 did not have this commit fb32856b16ad.
>
> I'm not sure if it helped you.
Hm, then what resolves the reported problem that people were having with
the 5.12.y kernel release? Is that a separate issue?
thanks,
greg k-h
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Wed, 9 Jun 2021 08:24:20 +0200, Greg KH wrote:
> On Wed, Jun 09, 2021 at 02:08:17PM +0800, Xuan Zhuo wrote:
> > On Wed, 9 Jun 2021 06:50:10 +0200, Greg KH
> > wrote:
> > > On Wed, Jun 09, 2021 at 09:48:33AM +0800, Xuan Zhuo wrote:
> > > > > > With this patch and the latest net branch I no longer get crashes.
> > > > >
> > > > > Did this ever get properly submitted to the networking tree to get
> > > > > into
> > > > > 5.13-final?
> > > >
> > > > The patch has been submitted.
> > > >
> > > > [PATCH net] virtio-net: fix for skb_over_panic inside big mode
> > >
> > > Submitted where? Do you have a lore.kernel.org link somewhere?
> >
> >
> > https://lore.kernel.org/netdev/20210603170901.66504-1-xuanz...@linux.alibaba.com/
>
> So this is commit 1a8024239dac ("virtio-net: fix for skb_over_panic
> inside big mode") in Linus's tree, right?
YES.
>
> But why is that referencing:
> Fixes: fb32856b16ad ("virtio-net: page_to_skb() use build_skb when
> there's sufficient tailroom")
This problem was indeed introduced in fb32856b16ad.
I confirmed that this commit fb32856b16ad was first entered in 5.13-rc1, and the
previous 5.12 did not have this commit fb32856b16ad.
I'm not sure if it helped you.
Thanks.
>
> when this problem was seen in stable kernels that had a different commit
> backported to it?
>
> Is there nothing needed to be done for the stable kernel trees?
>
> confused,
>
> greg k-h
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Wed, Jun 09, 2021 at 02:08:17PM +0800, Xuan Zhuo wrote:
> On Wed, 9 Jun 2021 06:50:10 +0200, Greg KH wrote:
> > On Wed, Jun 09, 2021 at 09:48:33AM +0800, Xuan Zhuo wrote:
> > > > > With this patch and the latest net branch I no longer get crashes.
> > > >
> > > > Did this ever get properly submitted to the networking tree to get into
> > > > 5.13-final?
> > >
> > > The patch has been submitted.
> > >
> > > [PATCH net] virtio-net: fix for skb_over_panic inside big mode
> >
> > Submitted where? Do you have a lore.kernel.org link somewhere?
>
>
> https://lore.kernel.org/netdev/20210603170901.66504-1-xuanz...@linux.alibaba.com/
So this is commit 1a8024239dac ("virtio-net: fix for skb_over_panic
inside big mode") in Linus's tree, right?
But why is that referencing:
Fixes: fb32856b16ad ("virtio-net: page_to_skb() use build_skb when
there's sufficient tailroom")
when this problem was seen in stable kernels that had a different commit
backported to it?
Is there nothing needed to be done for the stable kernel trees?
confused,
greg k-h
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Wed, 9 Jun 2021 06:50:10 +0200, Greg KH wrote: > On Wed, Jun 09, 2021 at 09:48:33AM +0800, Xuan Zhuo wrote: > > > > With this patch and the latest net branch I no longer get crashes. > > > > > > Did this ever get properly submitted to the networking tree to get into > > > 5.13-final? > > > > The patch has been submitted. > > > > [PATCH net] virtio-net: fix for skb_over_panic inside big mode > > Submitted where? Do you have a lore.kernel.org link somewhere? https://lore.kernel.org/netdev/20210603170901.66504-1-xuanz...@linux.alibaba.com/ Thanks. > > thanks, > > greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Wed, Jun 09, 2021 at 09:48:33AM +0800, Xuan Zhuo wrote: > > > With this patch and the latest net branch I no longer get crashes. > > > > Did this ever get properly submitted to the networking tree to get into > > 5.13-final? > > The patch has been submitted. > > [PATCH net] virtio-net: fix for skb_over_panic inside big mode Submitted where? Do you have a lore.kernel.org link somewhere? thanks, greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: virtio-net: kernel panic in virtio_net.c
On Tue, 8 Jun 2021 14:17:58 +0200, Greg KH wrote: > On Thu, Jun 03, 2021 at 10:57:52AM +0200, Corentin Noël wrote: > > Le jeudi 03 juin 2021 à 10:44 +0800, Xuan Zhuo a écrit : > > > On Wed, 02 Jun 2021 19:54:41 +0200, Corentin Noël < > > > corentin.n...@collabora.com> wrote: > > > > Sure, here is the decoded trace: > > > > > > > > [ 44.523231] skbuff: skb_over_panic: text:ad1a8434 > > > > len:3762 > > > > put:3762 head:9799e6b6b000 data:9799e6b6b010 tail:0xec2 > > > > end:0xec0 dev: > > > > [ 44.525254] kernel BUG at net/core/skbuff.c:110! > > > > [ 44.525910] invalid opcode: [#1] SMP PTI > > > > [ 44.526521] CPU: 2 PID: 245 Comm: llvmpipe-0 Not tainted 5.13.0- > > > > rc4linux-v5.13-rc4-for-mesa-ci-184862285c49.tar.bz2 #1 > > > > [ 44.528109] Hardware name: ChromiumOS crosvm, BIOS 0 > > > > [ 44.529243] RIP: 0010:skb_panic (net/core/skbuff.c:110) > > > > [ 44.530284] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 > > > > 50 > > > > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 f0 af cf ad e8 43 > > > > 4c fb > > > > ff <0f> 0b 48 8b 14 24 48 c7 c1 20 23 b1 ad e8 ab ff ff ff 48 c7 c6 > > > > 60 > > > > All code > > > > > > > >0: 4f 70 50rex.WRXB jo 0x53 > > > >3: 8b 87 bc 00 00 00 mov0xbc(%rdi),%eax > > > >9: 50 push %rax > > > >a: 8b 87 b8 00 00 00 mov0xb8(%rdi),%eax > > > > 10: 50 push %rax > > > > 11: ff b7 c8 00 00 00 pushq 0xc8(%rdi) > > > > 17: 4c 8b 8f c0 00 00 00mov0xc0(%rdi),%r9 > > > > 1e: 48 c7 c7 f0 af cf admov$0xadcfaff0, > > > > %rdi > > > > 25: e8 43 4c fb ff callq 0xfffb4c6d > > > > 2a:* 0f 0b ud2 <-- > > > > trapping > > > > instruction > > > > 2c: 48 8b 14 24 mov(%rsp),%rdx > > > > 30: 48 c7 c1 20 23 b1 admov$0xadb12320, > > > > %rcx > > > > 37: e8 ab ff ff ff callq 0xffe7 > > > > 3c: 48 rex.W > > > > 3d: c7 .byte 0xc7 > > > > 3e: c6 (bad) > > > > 3f: 60 (bad) > > > > > > > > Code starting with the faulting instruction > > > > === > > > >0: 0f 0b ud2 > > > >2: 48 8b 14 24 mov(%rsp),%rdx > > > >6: 48 c7 c1 20 23 b1 admov$0xadb12320, > > > > %rcx > > > >d: e8 ab ff ff ff callq 0xffbd > > > > 12: 48 rex.W > > > > 13: c7 .byte 0xc7 > > > > 14: c6 (bad) > > > > 15: 60 (bad) > > > > [ 44.533988] RSP: :a651c134fc20 EFLAGS: 00010246 > > > > [ 44.534723] RAX: 008b RBX: 0010 RCX: > > > > dfff > > > > [ 44.535772] RDX: RSI: ffea RDI: > > > > > > > > [ 44.536693] RBP: d77b009adac0 R08: adf44b08 R09: > > > > 9ffb > > > > [ 44.537569] R10: e000 R11: 3fff R12: > > > > 979ad2aa5600 > > > > [ 44.538449] R13: R14: 9799e6b6b000 R15: > > > > 0eb2 > > > > [ 44.539300] FS: 7fdb9cb11700() > > > > GS:979aebd0() > > > > knlGS: > > > > [ 44.540376] CS: 0010 DS: ES: CR0: 80050033 > > > > [ 44.541103] CR2: 7f99099f4024 CR3: 000129558005 CR4: > > > > 00370ee0 > > > > [ 44.542057] DR0: DR1: DR2: > > > > > > > > [ 44.543063] DR3: DR6: fffe0ff0 DR7: > > > > 0400 > > > > [ 44.544063] Call Trace: > > > > [ 44.544385] skb_put.cold (net/core/skbuff.c:5254 (discriminator > > > > 1) > > > > net/core/skbuff.c:5252 (discriminator 1)) > > > > [ 44.544864] page_to_skb (drivers/net/virtio_net.c:485) > > > > [ 44.545361] receive_buf (drivers/net/virtio_net.c:849 > > > > drivers/net/virtio_net.c:1131) > > > > [ 44.545870] ? netif_receive_skb_list_internal > > > > (net/core/dev.c:5714) > > > > [ 44.546628] ? dev_gro_receive (net/core/dev.c:6103) > > > > [ 44.547135] ? napi_complete_done (./include/linux/list.h:35 > > > > net/core/dev.c:5867 net/core/dev.c:5862 net/core/dev.c:6565) > > > > [ 44.547672] virtnet_poll (drivers/net/virtio_net.c:1427 > > > > drivers/net/virtio_net.c:1525) > > > > [ 44.548251] __napi_poll (net/core/dev.c:6985) > > > > [ 44.548744] net_rx_action (net/core/dev.c:7054 > > > > net/core/dev.c:7139) > > > > [ 44.549264] __do_softirq (./arch/x86/include/asm/jump_label.h:19 > > > > ./include/linux/jump_label.h:200 ./include/trace/events/irq.h:142 > > > > kernel/softirq.c:560) > > > > [ 44.549762] irq_exit_rcu (kernel/softirq.c:433 > > > > kernel/softirq.c:637 > > > > kernel/softirq.c:649) >
Re: virtio-net: kernel panic in virtio_net.c
On Thu, Jun 03, 2021 at 10:57:52AM +0200, Corentin Noël wrote: > Le jeudi 03 juin 2021 à 10:44 +0800, Xuan Zhuo a écrit : > > On Wed, 02 Jun 2021 19:54:41 +0200, Corentin Noël < > > corentin.n...@collabora.com> wrote: > > > Sure, here is the decoded trace: > > > > > > [ 44.523231] skbuff: skb_over_panic: text:ad1a8434 > > > len:3762 > > > put:3762 head:9799e6b6b000 data:9799e6b6b010 tail:0xec2 > > > end:0xec0 dev: > > > [ 44.525254] kernel BUG at net/core/skbuff.c:110! > > > [ 44.525910] invalid opcode: [#1] SMP PTI > > > [ 44.526521] CPU: 2 PID: 245 Comm: llvmpipe-0 Not tainted 5.13.0- > > > rc4linux-v5.13-rc4-for-mesa-ci-184862285c49.tar.bz2 #1 > > > [ 44.528109] Hardware name: ChromiumOS crosvm, BIOS 0 > > > [ 44.529243] RIP: 0010:skb_panic (net/core/skbuff.c:110) > > > [ 44.530284] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 > > > 50 > > > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 f0 af cf ad e8 43 > > > 4c fb > > > ff <0f> 0b 48 8b 14 24 48 c7 c1 20 23 b1 ad e8 ab ff ff ff 48 c7 c6 > > > 60 > > > All code > > > > > >0: 4f 70 50rex.WRXB jo 0x53 > > >3: 8b 87 bc 00 00 00 mov0xbc(%rdi),%eax > > >9: 50 push %rax > > >a: 8b 87 b8 00 00 00 mov0xb8(%rdi),%eax > > > 10: 50 push %rax > > > 11: ff b7 c8 00 00 00 pushq 0xc8(%rdi) > > > 17: 4c 8b 8f c0 00 00 00mov0xc0(%rdi),%r9 > > > 1e: 48 c7 c7 f0 af cf admov$0xadcfaff0, > > > %rdi > > > 25: e8 43 4c fb ff callq 0xfffb4c6d > > > 2a:*0f 0b ud2 <-- > > > trapping > > > instruction > > > 2c: 48 8b 14 24 mov(%rsp),%rdx > > > 30: 48 c7 c1 20 23 b1 admov$0xadb12320, > > > %rcx > > > 37: e8 ab ff ff ff callq 0xffe7 > > > 3c: 48 rex.W > > > 3d: c7 .byte 0xc7 > > > 3e: c6 (bad) > > > 3f: 60 (bad) > > > > > > Code starting with the faulting instruction > > > === > > >0: 0f 0b ud2 > > >2: 48 8b 14 24 mov(%rsp),%rdx > > >6: 48 c7 c1 20 23 b1 admov$0xadb12320, > > > %rcx > > >d: e8 ab ff ff ff callq 0xffbd > > > 12: 48 rex.W > > > 13: c7 .byte 0xc7 > > > 14: c6 (bad) > > > 15: 60 (bad) > > > [ 44.533988] RSP: :a651c134fc20 EFLAGS: 00010246 > > > [ 44.534723] RAX: 008b RBX: 0010 RCX: > > > dfff > > > [ 44.535772] RDX: RSI: ffea RDI: > > > > > > [ 44.536693] RBP: d77b009adac0 R08: adf44b08 R09: > > > 9ffb > > > [ 44.537569] R10: e000 R11: 3fff R12: > > > 979ad2aa5600 > > > [ 44.538449] R13: R14: 9799e6b6b000 R15: > > > 0eb2 > > > [ 44.539300] FS: 7fdb9cb11700() > > > GS:979aebd0() > > > knlGS: > > > [ 44.540376] CS: 0010 DS: ES: CR0: 80050033 > > > [ 44.541103] CR2: 7f99099f4024 CR3: 000129558005 CR4: > > > 00370ee0 > > > [ 44.542057] DR0: DR1: DR2: > > > > > > [ 44.543063] DR3: DR6: fffe0ff0 DR7: > > > 0400 > > > [ 44.544063] Call Trace: > > > [ 44.544385] skb_put.cold (net/core/skbuff.c:5254 (discriminator > > > 1) > > > net/core/skbuff.c:5252 (discriminator 1)) > > > [ 44.544864] page_to_skb (drivers/net/virtio_net.c:485) > > > [ 44.545361] receive_buf (drivers/net/virtio_net.c:849 > > > drivers/net/virtio_net.c:1131) > > > [ 44.545870] ? netif_receive_skb_list_internal > > > (net/core/dev.c:5714) > > > [ 44.546628] ? dev_gro_receive (net/core/dev.c:6103) > > > [ 44.547135] ? napi_complete_done (./include/linux/list.h:35 > > > net/core/dev.c:5867 net/core/dev.c:5862 net/core/dev.c:6565) > > > [ 44.547672] virtnet_poll (drivers/net/virtio_net.c:1427 > > > drivers/net/virtio_net.c:1525) > > > [ 44.548251] __napi_poll (net/core/dev.c:6985) > > > [ 44.548744] net_rx_action (net/core/dev.c:7054 > > > net/core/dev.c:7139) > > > [ 44.549264] __do_softirq (./arch/x86/include/asm/jump_label.h:19 > > > ./include/linux/jump_label.h:200 ./include/trace/events/irq.h:142 > > > kernel/softirq.c:560) > > > [ 44.549762] irq_exit_rcu (kernel/softirq.c:433 > > > kernel/softirq.c:637 > > > kernel/softirq.c:649) > > > [ 44.551384] common_interrupt (arch/x86/kernel/irq.c:240 > > > (discriminator 13)) > > > [ 44.551991] ? asm_common_interrupt > > > (./arch/x86/include/asm/idtentry.h:638) > > > [ 44.552654]
Re: virtio-net: kernel panic in virtio_net.c
On Wed, 02 Jun 2021 19:54:41 +0200, Corentin Noël wrote: > Sure, here is the decoded trace: > > [ 44.523231] skbuff: skb_over_panic: text:ad1a8434 len:3762 > put:3762 head:9799e6b6b000 data:9799e6b6b010 tail:0xec2 > end:0xec0 dev: > [ 44.525254] kernel BUG at net/core/skbuff.c:110! > [ 44.525910] invalid opcode: [#1] SMP PTI > [ 44.526521] CPU: 2 PID: 245 Comm: llvmpipe-0 Not tainted 5.13.0- > rc4linux-v5.13-rc4-for-mesa-ci-184862285c49.tar.bz2 #1 > [ 44.528109] Hardware name: ChromiumOS crosvm, BIOS 0 > [ 44.529243] RIP: 0010:skb_panic (net/core/skbuff.c:110) > [ 44.530284] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 50 > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 f0 af cf ad e8 43 4c fb > ff <0f> 0b 48 8b 14 24 48 c7 c1 20 23 b1 ad e8 ab ff ff ff 48 c7 c6 60 > All code > >0: 4f 70 50rex.WRXB jo 0x53 >3: 8b 87 bc 00 00 00 mov0xbc(%rdi),%eax >9: 50 push %rax >a: 8b 87 b8 00 00 00 mov0xb8(%rdi),%eax > 10: 50 push %rax > 11: ff b7 c8 00 00 00 pushq 0xc8(%rdi) > 17: 4c 8b 8f c0 00 00 00mov0xc0(%rdi),%r9 > 1e: 48 c7 c7 f0 af cf admov$0xadcfaff0,%rdi > 25: e8 43 4c fb ff callq 0xfffb4c6d > 2a:*0f 0b ud2 <-- trapping > instruction > 2c: 48 8b 14 24 mov(%rsp),%rdx > 30: 48 c7 c1 20 23 b1 admov$0xadb12320,%rcx > 37: e8 ab ff ff ff callq 0xffe7 > 3c: 48 rex.W > 3d: c7 .byte 0xc7 > 3e: c6 (bad) > 3f: 60 (bad) > > Code starting with the faulting instruction > === >0: 0f 0b ud2 >2: 48 8b 14 24 mov(%rsp),%rdx >6: 48 c7 c1 20 23 b1 admov$0xadb12320,%rcx >d: e8 ab ff ff ff callq 0xffbd > 12: 48 rex.W > 13: c7 .byte 0xc7 > 14: c6 (bad) > 15: 60 (bad) > [ 44.533988] RSP: :a651c134fc20 EFLAGS: 00010246 > [ 44.534723] RAX: 008b RBX: 0010 RCX: > dfff > [ 44.535772] RDX: RSI: ffea RDI: > > [ 44.536693] RBP: d77b009adac0 R08: adf44b08 R09: > 9ffb > [ 44.537569] R10: e000 R11: 3fff R12: > 979ad2aa5600 > [ 44.538449] R13: R14: 9799e6b6b000 R15: > 0eb2 > [ 44.539300] FS: 7fdb9cb11700() GS:979aebd0() > knlGS: > [ 44.540376] CS: 0010 DS: ES: CR0: 80050033 > [ 44.541103] CR2: 7f99099f4024 CR3: 000129558005 CR4: > 00370ee0 > [ 44.542057] DR0: DR1: DR2: > > [ 44.543063] DR3: DR6: fffe0ff0 DR7: > 0400 > [ 44.544063] Call Trace: > [ 44.544385] skb_put.cold (net/core/skbuff.c:5254 (discriminator 1) > net/core/skbuff.c:5252 (discriminator 1)) > [ 44.544864] page_to_skb (drivers/net/virtio_net.c:485) > [ 44.545361] receive_buf (drivers/net/virtio_net.c:849 > drivers/net/virtio_net.c:1131) > [ 44.545870] ? netif_receive_skb_list_internal (net/core/dev.c:5714) > [ 44.546628] ? dev_gro_receive (net/core/dev.c:6103) > [ 44.547135] ? napi_complete_done (./include/linux/list.h:35 > net/core/dev.c:5867 net/core/dev.c:5862 net/core/dev.c:6565) > [ 44.547672] virtnet_poll (drivers/net/virtio_net.c:1427 > drivers/net/virtio_net.c:1525) > [ 44.548251] __napi_poll (net/core/dev.c:6985) > [ 44.548744] net_rx_action (net/core/dev.c:7054 net/core/dev.c:7139) > [ 44.549264] __do_softirq (./arch/x86/include/asm/jump_label.h:19 > ./include/linux/jump_label.h:200 ./include/trace/events/irq.h:142 > kernel/softirq.c:560) > [ 44.549762] irq_exit_rcu (kernel/softirq.c:433 kernel/softirq.c:637 > kernel/softirq.c:649) > [ 44.551384] common_interrupt (arch/x86/kernel/irq.c:240 > (discriminator 13)) > [ 44.551991] ? asm_common_interrupt > (./arch/x86/include/asm/idtentry.h:638) > [ 44.552654] asm_common_interrupt > (./arch/x86/include/asm/idtentry.h:638) > [ 44.553276] RIP: 0033:0x7fdb981a82e4 > [ 44.553809] Code: d2 48 63 f6 c4 41 7a 6f 0c 01 c4 41 7a 6f 14 09 c4 > 41 7a 6f 24 11 c4 41 7a 6f 2c 31 c4 c1 31 6a c2 c4 c1 19 6a d5 c5 f9 6c > f2 79 6d c2 c5 f9 71 d6 08 c5 f9 db 44 24 20 c5 c1 71 f6 0b c5 f9 > All code > >0: d2 48 63rorb %cl,0x63(%rax) >3: f6 c4 41test $0x41,%ah >6: 7a 6f jp 0x77 >8: 0c 01 or $0x1,%al >a: c4 41 7a 6f 14 09 vmovdqu (%r9,%rcx,1),%xmm10 > 10: c4 41 7a 6f 24 11 vmovdqu (%r9,%rdx,1),%xmm12 > 16: c4 41 7a 6f 2c 31
Re: virtio-net: kernel panic in virtio_net.c
On Tue, 1 Jun 2021 19:47:44 +0200, Eric Dumazet wrote: > On Tue, Jun 1, 2021 at 7:09 PM Corentin Noël > wrote: > > > > Le mardi 01 juin 2021 à 19:07 +0200, Greg KH a écrit : > > > On Tue, Jun 01, 2021 at 06:06:50PM +0200, Corentin Noël wrote: > > > > I've been experiencing crashes with 5.13 that do not occur with > > > > 5.12, > > > > here is the crash trace: > > > > > > > > [ 47.713713] skbuff: skb_over_panic: text:b73a8354 > > > > len:3762 > > > > put:3762 head:9e1e1e48e000 data:9e1e1e48e010 tail:0xec2 > > > > end:0xec0 dev: > > > > [ 47.716267] kernel BUG at net/core/skbuff.c:110! > > > > [ 47.717197] invalid opcode: [#1] SMP PTI > > > > [ 47.718049] CPU: 2 PID: 730 Comm: llvmpipe-0 Not tainted 5.13.0- > > > > rc3linux-v5.13-rc3-for-mesa-ci-87614d7f3282.tar.bz2 #1 > > > > [ 47.719739] Hardware name: ChromiumOS crosvm, BIOS 0 > > > > [ 47.720656] RIP: 0010:skb_panic+0x43/0x45 > > > > [ 47.721426] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 > > > > 00 50 > > > > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 78 ae ef b7 e8 7f > > > > 4c fb > > > > ff <0f> 0b 48 8b 14 24 48 c7 c1 a0 22 d1 b7 e8 ab ff ff ff 48 c7 c6 > > > > e0 > > > > [ 47.725944] RSP: :acec01347c20 EFLAGS: 00010246 > > > > [ 47.726735] RAX: 008b RBX: 0010 RCX: > > > > dfff > > > > [ 47.727820] RDX: RSI: ffea RDI: > > > > > > > > [ 47.729096] RBP: eb2700792380 R08: b8144b08 R09: > > > > 9ffb > > > > [ 47.730260] R10: e000 R11: 3fff R12: > > > > 9e1e1e95b300 > > > > [ 47.731411] R13: R14: 9e1e1e48e000 R15: > > > > 0eb2 > > > > [ 47.732541] FS: 7f3a82b53700() > > > > GS:9e1f2bd0() > > > > knlGS: > > > > [ 47.733858] CS: 0010 DS: ES: CR0: 80050033 > > > > [ 47.734813] CR2: 010d24f8 CR3: 12d6e004 CR4: > > > > 00370ee0 > > > > [ 47.735968] DR0: DR1: DR2: > > > > > > > > [ 47.737091] DR3: DR6: fffe0ff0 DR7: > > > > 0400 > > > > [ 47.738318] Call Trace: > > > > [ 47.738812] skb_put.cold+0x10/0x10 > > > > [ 47.739450] page_to_skb+0xe4/0x400 > > > > [ 47.740072] receive_buf+0x86/0x1660 > > > > [ 47.740693] ? inet_gro_receive+0x54/0x2c0 > > > > [ 47.741279] ? dev_gro_receive+0x194/0x6a0 > > > > [ 47.741846] virtnet_poll+0x2b8/0x3c0 > > > > [ 47.742357] __napi_poll+0x25/0x150 > > > > [ 47.742844] net_rx_action+0x22f/0x280 > > > > [ 47.743388] __do_softirq+0xba/0x264 > > > > [ 47.743947] irq_exit_rcu+0x90/0xb0 > > > > [ 47.744435] common_interrupt+0x40/0xa0 > > > > [ 47.744978] ? asm_common_interrupt+0x8/0x40 > > > > [ 47.745582] asm_common_interrupt+0x1e/0x40 > > > > [ 47.746182] RIP: 0033:0x7f3a7a276ed4 > > > > [ 47.746708] Code: a0 03 00 00 c5 fc 29 84 24 40 0f 00 00 c5 bc > > > > 54 c8 > > > > c5 7c 28 84 24 80 01 00 00 c5 bc 59 e9 c5 fe 5b ed c5 fd 76 c0 c5 > > > > d5 fa > > > > c0 fd db ec c5 fd 7f 84 24 20 0f 00 00 c5 fc 5b ed c4 e2 55 b8 > > > > cb > > > > [ 47.749292] RSP: 002b:7f3a82b4dba0 EFLAGS: 0212 > > > > [ 47.750006] RAX: 7f3a8c210324 RBX: RCX: > > > > > > > > [ 47.750964] RDX: 7f3a8c210348 RSI: 7f3a8c21034c RDI: > > > > 7f3a7c0575a0 > > > > [ 47.752049] RBP: 7f3a82b52ca0 R08: 7f3a8c210350 R09: > > > > 7f3a8c210354 > > > > [ 47.753161] R10: 7f3a8c210358 R11: ffef R12: > > > > 7f3a8c210340 > > > > [ 47.754260] R13: 7f3a8c210344 R14: 7f3a7c057580 R15: > > > > 7f3a8c21033c > > > > [ 47.755354] Modules linked in: > > > > [ 47.755871] ---[ end trace a8b692ea99c9cd9e ]--- > > > > [ 47.756606] RIP: 0010:skb_panic+0x43/0x45 > > > > [ 47.757297] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 > > > > 00 50 > > > > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 78 ae ef b7 e8 7f > > > > 4c fb > > > > ff <0f> 0b 48 8b 14 24 48 c7 c1 a0 22 d1 b7 e8 ab ff ff ff 48 c7 c6 > > > > e0 > > > > [ 47.760168] RSP: :acec01347c20 EFLAGS: 00010246 > > > > [ 47.760896] RAX: 008b RBX: 0010 RCX: > > > > dfff > > > > [ 47.761903] RDX: RSI: ffea RDI: > > > > > > > > [ 47.762945] RBP: eb2700792380 R08: b8144b08 R09: > > > > 9ffb > > > > [ 47.764059] R10: e000 R11: 3fff R12: > > > > 9e1e1e95b300 > > > > [ 47.765169] R13: R14: 9e1e1e48e000 R15: > > > > 0eb2 > > > > [ 47.766261] FS: 7f3a82b53700() > > > > GS:9e1f2bd0() > > > > knlGS: > > > > [ 47.767512] CS: 0010 DS: ES: CR0: 80050033 > > > > [ 47.768389] CR2: 010d24f8 CR3: 12d6e004 CR4:
Re: virtio-net: kernel panic in virtio_net.c
On Tue, Jun 01, 2021 at 06:06:50PM +0200, Corentin Noël wrote: > I've been experiencing crashes with 5.13 that do not occur with 5.12, > here is the crash trace: > > [ 47.713713] skbuff: skb_over_panic: text:b73a8354 len:3762 > put:3762 head:9e1e1e48e000 data:9e1e1e48e010 tail:0xec2 > end:0xec0 dev: > [ 47.716267] kernel BUG at net/core/skbuff.c:110! > [ 47.717197] invalid opcode: [#1] SMP PTI > [ 47.718049] CPU: 2 PID: 730 Comm: llvmpipe-0 Not tainted 5.13.0- > rc3linux-v5.13-rc3-for-mesa-ci-87614d7f3282.tar.bz2 #1 > [ 47.719739] Hardware name: ChromiumOS crosvm, BIOS 0 > [ 47.720656] RIP: 0010:skb_panic+0x43/0x45 > [ 47.721426] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 50 > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 78 ae ef b7 e8 7f 4c fb > ff <0f> 0b 48 8b 14 24 48 c7 c1 a0 22 d1 b7 e8 ab ff ff ff 48 c7 c6 e0 > [ 47.725944] RSP: :acec01347c20 EFLAGS: 00010246 > [ 47.726735] RAX: 008b RBX: 0010 RCX: > dfff > [ 47.727820] RDX: RSI: ffea RDI: > > [ 47.729096] RBP: eb2700792380 R08: b8144b08 R09: > 9ffb > [ 47.730260] R10: e000 R11: 3fff R12: > 9e1e1e95b300 > [ 47.731411] R13: R14: 9e1e1e48e000 R15: > 0eb2 > [ 47.732541] FS: 7f3a82b53700() GS:9e1f2bd0() > knlGS: > [ 47.733858] CS: 0010 DS: ES: CR0: 80050033 > [ 47.734813] CR2: 010d24f8 CR3: 12d6e004 CR4: > 00370ee0 > [ 47.735968] DR0: DR1: DR2: > > [ 47.737091] DR3: DR6: fffe0ff0 DR7: > 0400 > [ 47.738318] Call Trace: > [ 47.738812] skb_put.cold+0x10/0x10 > [ 47.739450] page_to_skb+0xe4/0x400 > [ 47.740072] receive_buf+0x86/0x1660 > [ 47.740693] ? inet_gro_receive+0x54/0x2c0 > [ 47.741279] ? dev_gro_receive+0x194/0x6a0 > [ 47.741846] virtnet_poll+0x2b8/0x3c0 > [ 47.742357] __napi_poll+0x25/0x150 > [ 47.742844] net_rx_action+0x22f/0x280 > [ 47.743388] __do_softirq+0xba/0x264 > [ 47.743947] irq_exit_rcu+0x90/0xb0 > [ 47.744435] common_interrupt+0x40/0xa0 > [ 47.744978] ? asm_common_interrupt+0x8/0x40 > [ 47.745582] asm_common_interrupt+0x1e/0x40 > [ 47.746182] RIP: 0033:0x7f3a7a276ed4 > [ 47.746708] Code: a0 03 00 00 c5 fc 29 84 24 40 0f 00 00 c5 bc 54 c8 > c5 7c 28 84 24 80 01 00 00 c5 bc 59 e9 c5 fe 5b ed c5 fd 76 c0 c5 d5 fa > c0 fd db ec c5 fd 7f 84 24 20 0f 00 00 c5 fc 5b ed c4 e2 55 b8 cb > [ 47.749292] RSP: 002b:7f3a82b4dba0 EFLAGS: 0212 > [ 47.750006] RAX: 7f3a8c210324 RBX: RCX: > > [ 47.750964] RDX: 7f3a8c210348 RSI: 7f3a8c21034c RDI: > 7f3a7c0575a0 > [ 47.752049] RBP: 7f3a82b52ca0 R08: 7f3a8c210350 R09: > 7f3a8c210354 > [ 47.753161] R10: 7f3a8c210358 R11: ffef R12: > 7f3a8c210340 > [ 47.754260] R13: 7f3a8c210344 R14: 7f3a7c057580 R15: > 7f3a8c21033c > [ 47.755354] Modules linked in: > [ 47.755871] ---[ end trace a8b692ea99c9cd9e ]--- > [ 47.756606] RIP: 0010:skb_panic+0x43/0x45 > [ 47.757297] Code: 4f 70 50 8b 87 bc 00 00 00 50 8b 87 b8 00 00 00 50 > ff b7 c8 00 00 00 4c 8b 8f c0 00 00 00 48 c7 c7 78 ae ef b7 e8 7f 4c fb > ff <0f> 0b 48 8b 14 24 48 c7 c1 a0 22 d1 b7 e8 ab ff ff ff 48 c7 c6 e0 > [ 47.760168] RSP: :acec01347c20 EFLAGS: 00010246 > [ 47.760896] RAX: 008b RBX: 0010 RCX: > dfff > [ 47.761903] RDX: RSI: ffea RDI: > > [ 47.762945] RBP: eb2700792380 R08: b8144b08 R09: > 9ffb > [ 47.764059] R10: e000 R11: 3fff R12: > 9e1e1e95b300 > [ 47.765169] R13: R14: 9e1e1e48e000 R15: > 0eb2 > [ 47.766261] FS: 7f3a82b53700() GS:9e1f2bd0() > knlGS: > [ 47.767512] CS: 0010 DS: ES: CR0: 80050033 > [ 47.768389] CR2: 010d24f8 CR3: 12d6e004 CR4: > 00370ee0 > [ 47.769381] DR0: DR1: DR2: > > [ 47.770362] DR3: DR6: fffe0ff0 DR7: > 0400 > [ 47.771339] Kernel panic - not syncing: Fatal exception in interrupt > [ 47.772814] Kernel Offset: 0x35c0 from 0x8100 > (relocation range: 0x8000-0xbfff) > > I've been able to bisect the issue a little bit and the issue > disappeared after reverting the 4 following commits: > * fb32856b16ad9d5bcd75b76a274e2c515ac7b9d7 > * af39c8f72301b268ad8b04bae646b6025918b82b > * f5d7872a8b8a3176e65dc6f7f0705ce7e9a699e6 > * f80bd740cb7c954791279590b2e810ba6c214e52 > > Here is my kernel config: >
