[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
[Expired for linux-lts-saucy (Ubuntu) because there has been no activity for 60 days.] ** Changed in: linux-lts-saucy (Ubuntu) Status: Incomplete = Expired -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
I have the same problem on several different machines, both on 3.11.0-20 and 3.11.0-24. For me it only happens on boot, and not entirely consistantly - Sometimes it works, sometimes it crashes. It does crash pretty often though, and I'm trying to find a way to reproduce consistently. [2.962760] Installing knfsd (copyright (C) 1996 o...@monad.swb.de). [2.973227] BUG: unable to handle kernel NULL pointer dereference at 0010 [2.973436] IP: [a00a9618] __rpc_clnt_handle_event+0x48/0xd0 [sunrpc] [2.973607] PGD 136103067 PUD 136102067 PMD 0 [2.973777] Oops: [#1] SMP [2.973907] Modules linked in: nfsd nfsv4 nfs_acl mac_hid auth_rpcgss shpchp vmw_vmci nfs fscache i2c_piix4 lockd lp sunrpc parport floppy mptspi mptscsih e1000 mptbase vmw_pvscsi vmxnet3 [2.974766] CPU: 1 PID: 511 Comm: mount Not tainted 3.11.0-24-generic #41~precise1-Ubuntu [2.974911] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013 [2.975092] task: 880135df8000 ti: 880136ffc000 task.ti: 880136ffc000 [2.975231] RIP: 0010:[a00a9618] [a00a9618] __rpc_clnt_handle_event+0x48/0xd0 [sunrpc] [2.975441] RSP: 0018:880136ffdc18 EFLAGS: 00010283 [2.975554] RAX: 88013908ea80 RBX: 8801383dbe00 RCX: 0002 [2.975679] RDX: RSI: 0100 RDI: [2.975797] RBP: 880136ffdc38 R08: 88013908e7b8 R09: [2.975916] R10: 0001 R11: 00017960 R12: [2.976032] R13: 880135c86400 R14: R15: [2.976152] FS: 7f82b8da7800() GS:88013fd0() knlGS: [2.976299] CS: 0010 DS: ES: CR0: 80050033 [2.976403] CR2: 0010 CR3: 000136106000 CR4: 000407e0 [2.976621] Stack: [2.976688] 880136ffdc28 8801383dbe00 [2.976933] 880136ffdc68 a00aa14a 880135c86400 [2.977182] a01c3de0 880136ffdca8 8174eeed [2.977427] Call Trace: [2.977505] [a00aa14a] rpc_pipefs_event+0x6a/0xb0 [sunrpc] [2.977617] [8174eeed] notifier_call_chain+0x4d/0x70 [2.977724] [8108ed28] __blocking_notifier_call_chain+0x58/0x80 [2.977837] [8108ed66] blocking_notifier_call_chain+0x16/0x20 [2.977964] [a00c4fdc] rpc_fill_super+0xfc/0x1b0 [sunrpc] [2.978081] [811b75a0] ? set_anon_super+0x30/0x30 [2.978190] [a00c4ee0] ? rpc_kill_sb+0x110/0x110 [sunrpc] [2.978298] [811b8565] mount_ns+0xa5/0xe0 [2.978400] [a00c3599] rpc_mount+0x29/0x30 [sunrpc] [2.978503] [811b91c3] mount_fs+0x43/0x1b0 [2.978603] [811d2e33] ? find_filesystem+0x63/0x80 [2.978706] [811d4116] vfs_kern_mount+0x76/0x120 [2.978807] [811d57a4] do_new_mount+0xa4/0x1f0 [2.978907] [811d6e06] do_mount+0x1e6/0x230 [2.979004] [811d6ee0] SyS_mount+0x90/0xe0 [2.979101] [81753a1d] system_call_fastpath+0x1a/0x1f [2.979204] Code: 01 00 00 48 89 de 48 8b 50 20 e8 54 f5 ff ff 48 85 c0 0f 84 8c 00 00 00 48 3d 00 f0 ff ff 77 5e 48 8b 7b 58 48 89 83 b0 00 00 00 48 8b 47 10 48 8b 50 38 31 c0 48 85 d2 74 44 ff d2 85 c0 74 3e [2.981267] RIP [a00a9618] __rpc_clnt_handle_event+0x48/0xd0 [sunrpc] [2.981436] RSP 880136ffdc18 [2.981516] CR2: 0010 [2.981626] ---[ end trace 3663d7a4a73c156e ]--- Please let me know if you guys need more info to resolve this. I'm afraid it makes NFS pretty useless here, right now. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
Without too many details, the problem seems to be triggered by some sort of race condition between the loading of the sunrpc module and using it (mounting the rpc_pipefs filesystem). This can happen in two places, as far as I can tell... in /etc/init/gssd.conf and /etc/init/idmapd.conf. On my system, I seem to have worked around the problem by: 1) Disabling GSSD in /etc/default/nfs-common (not used on my system anyways) 2) Adding a delay (sleep 1) just after the do_modprobe sunrpc on line 27 in /etc/init/idmapd.conf Before the workaround I was able to reproduce the problem in a few minutes by continously rebooting the system (basically adding a reboot command to /etc/rc.local). After the workaround, my test system rebooted continously for 90 minutes with no issues. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
Chad Wagner, thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command, as it will automatically gather debugging information, in a terminal: apport-collect 1310350 When reporting bugs in the future please use apport by using 'ubuntu-bug' and the name of the package affected. You can learn more about this functionality at https://wiki.ubuntu.com/ReportingBugs. ** Changed in: linux-lts-saucy (Ubuntu) Importance: Undecided = Low ** Changed in: linux-lts-saucy (Ubuntu) Status: Confirmed = Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
linux-image-3.11.0-20-generic seems to have resolved the issue, at least the first time reboot went through fine. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
Chad, just want to know was your problem of 19-generic consistently occurring, what I met is not however. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
It was a problem with 2-3 consecutive reboots with 19. I believe it has to do with mounting rpc_pipefs, and the stack trace also seems to align with this, it looks like some piece of the memory structure is uninitialized. I personally didn't notice anything between 18 and 19 that would have caused this, but I am far from a kernel expert. It may still be present in -20, but generally I do not reboot that frequently. I often tend to skip over several updates at a time before rebooting. (gdb) list *(__rpc_clnt_handle_event+0x48) 0x2618 is in __rpc_clnt_handle_event (/home/buildd/debian/precise/linux-lts-saucy/linux-lts-saucy-3.11.0-19/net/sunrpc/clnt.c:196). 191 if (!dentry) 192 return -ENOENT; 193 if (IS_ERR(dentry)) 194 return PTR_ERR(dentry); 195 clnt-cl_dentry = dentry; 196 if (clnt-cl_auth-au_ops-pipes_create) { 197 err = clnt-cl_auth-au_ops-pipes_create(clnt-cl_auth); 198 if (err) 199 __rpc_clnt_remove_pipedir(clnt); 200 } -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
Also, it does not happen consistently. Seems to be related to network speed as I am booting my VM from qcow2 mounted from NFS. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
Hi, I am seeing the exact same thing when my ubuntu VM is trying to mount the root filesystem. Everything was OK with the 3.11.0-18 kernel : [ 97.595333] BUG: unable to handle kernel NULL pointer dereference at 0010 [ 97.596342] IP: [a00e8618] __rpc_clnt_handle_event+0x48/0xd0 [sunrpc] [ 97.596342] PGD 1d5c87067 PUD 1d416f067 PMD 0 [ 97.596342] Oops: [#1] SMP [ 97.596342] Modules linked in: nfsv4 nfsd nfs_acl auth_rpcgss nfs fscache lockd sunrpc ext2 kvm_intel kvm mac_hid psmouse microcode serio_raw virtio_balloon lp parport [ 97.596342] CPU: 0 PID: 781 Comm: mount Not tainted 3.11.0-19-generic #33~precise1-Ubuntu [ 97.596342] Hardware name: Bochs OVS-ubuntu1204, BIOS Bochs 01/01/2011 [ 97.596342] task: 8801d5e95dc0 ti: 8801d52f8000 task.ti: 8801d52f8000 [ 97.596342] RIP: 0010:[a00e8618] [a00e8618] __rpc_clnt_handle_event+0x48/0xd0 [sunrpc] [ 97.596342] RSP: 0018:8801d52f9c18 EFLAGS: 00010283 [ 97.596342] RAX: 8801d6768d80 RBX: 8801d5e41800 RCX: 0002 [ 97.596342] RDX: RSI: 0100 RDI: [ 97.596342] RBP: 8801d52f9c38 R08: 8801d6768e78 R09: [ 97.596342] R10: 0001 R11: 00017960 R12: [ 97.596342] R13: 8801d3267000 R14: R15: [ 97.596342] FS: 7f4c37235800() GS:8801e0e0() knlGS: [ 97.596342] CS: 0010 DS: ES: CR0: 80050033 [ 97.596342] CR2: 0010 CR3: 0001d307f000 CR4: 06f0 [ 97.596342] Stack: [ 97.596342] 8801d52f9c28 8801d5e41800 [ 97.596342] 8801d52f9c68 a00e914a 8801d3267000 [ 97.596342] a01ec790 8801d52f9ca8 8174d8ed [ 97.596342] Call Trace: [ 97.596342] [a00e914a] rpc_pipefs_event+0x6a/0xb0 [sunrpc] [ 97.596342] [8174d8ed] notifier_call_chain+0x4d/0x70 [ 97.596342] [8108ec18] __blocking_notifier_call_chain+0x58/0x80 [ 97.596342] [8108ec56] blocking_notifier_call_chain+0x16/0x20 [ 97.596342] [a0103f8c] rpc_fill_super+0xfc/0x1b0 [sunrpc] [ 97.596342] [811b7110] ? set_anon_super+0x30/0x30 [ 97.596342] [a0103e90] ? rpc_kill_sb+0x110/0x110 [sunrpc] [ 97.596342] [811b80d5] mount_ns+0xa5/0xe0 [ 97.596342] [a0102549] rpc_mount+0x29/0x30 [sunrpc] [ 97.596342] [811b8d33] mount_fs+0x43/0x1b0 [ 97.596342] [811d29d3] ? find_filesystem+0x63/0x80 [ 97.596342] [811d3cb6] vfs_kern_mount+0x76/0x120 [ 97.596342] [811d5344] do_new_mount+0xa4/0x1f0 [ 97.596342] [811d69a6] do_mount+0x1e6/0x230 [ 97.596342] [811d6a80] SyS_mount+0x90/0xe0 [ 97.596342] [8175241d] system_call_fastpath+0x1a/0x1f [ 97.596342] Code: 01 00 00 48 89 de 48 8b 50 20 e8 54 f5 ff ff 48 85 c0 0f 84 8c 00 00 00 48 3d 00 f0 ff ff 77 5e 48 8b 7b 58 48 89 83 b0 00 00 00 48 8b 47 10 48 8b 50 38 31 c0 48 85 d2 74 44 ff d2 85 c0 74 3e [ 97.596342] RIP [a00e8618] __rpc_clnt_handle_event+0x48/0xd0 [sunrpc] [ 97.596342] RSP 8801d52f9c18 [ 97.596342] CR2: 0010 [ 97.677800] ---[ end trace 8421fc045141f7a0 ]--- [ 98.213048] init: idmapd main process (784) terminated with status 1 [ 98.214932] init: idmapd main process ended, respawning The disk drive for /usr/global is not ready yet or not present. Continue to wait, or Press S to skip mounting or M for manual recovery -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1310350] Re: Oops __rpc_clnt_handle_event+0x48/0xd0
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: linux-lts-saucy (Ubuntu) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1310350 Title: Oops __rpc_clnt_handle_event+0x48/0xd0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1310350/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs