[Kernel-packages] [Bug 1668356] Re: Hard lockup after 4 hours uptime
Hi, We can confirm that this issue still persists as of 4.4.0-103-generic #126-Ubuntu SMP Mon Dec 4 16:23:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux this update. It only happens on a soft reboot, i.e. 'sudo reboot'. Not sure if there is a work around for it. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1668356 Title: Hard lockup after 4 hours uptime Status in linux package in Ubuntu: Triaged Status in linux source package in Xenial: Triaged Bug description: We have recently deployed intel NUC6i5 devices in store for POS display and are encountering strange inexplicable freezes on several devices. What is very strange is that all devices are freezing exactly 4h after boot. We have had this exact same issue on more than 20 devices (over 100), with parts from different batches, and all did freeze exactly 4h after boot (but it's not reproducible, it won't freeze every day). Some devices are playing MPV videos, while other run chromium. They are running non-stop, but are all rebooting daily at 05h05. Looks like it might be related to https://lkml.org/lkml/2015/6/11/787 That seem to have been fixed and backported already https://lkml.org/lkml/2015/10/17/259 ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-64-generic 4.4.0-64.85 ProcVersionSignature: Ubuntu 4.4.0-64.85-generic 4.4.44 Uname: Linux 4.4.0-64-generic x86_64 ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 Date: Mon Feb 27 21:06:44 2017 InstallationDate: Installed on 2016-06-08 (264 days ago) InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1) Lsusb: Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 8087:0a2b Intel Corp. Bus 001 Device 002: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-64-generic.efi.signed root=UUID=1f60f9c9-bbf1-45df-bdc3-9b4da883839e ro quiet splash net.ifnames=0 vt.handoff=7 SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.board.name: NUC6i5SYB To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1668356/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1668356] Re: Hard lockup after 4 hours uptime
Hard to tell regarding updates, I'd say it started early january (but we did not really pay attention at first). Our players have unattended security upgrades so I'd say some kernel upgrade landing in january might have introduced a regression. As far as I know we never encountered this issue in 2016. Do you think using 4.10 would be considered safe in production? I'm a bit afraid to (further) break production machine. Thanks! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1668356 Title: Hard lockup after 4 hours uptime Status in linux package in Ubuntu: Triaged Status in linux source package in Xenial: Triaged Bug description: We have recently deployed intel NUC6i5 devices in store for POS display and are encountering strange inexplicable freezes on several devices. What is very strange is that all devices are freezing exactly 4h after boot. We have had this exact same issue on more than 20 devices (over 100), with parts from different batches, and all did freeze exactly 4h after boot (but it's not reproducible, it won't freeze every day). Some devices are playing MPV videos, while other run chromium. They are running non-stop, but are all rebooting daily at 05h05. Looks like it might be related to https://lkml.org/lkml/2015/6/11/787 That seem to have been fixed and backported already https://lkml.org/lkml/2015/10/17/259 ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-64-generic 4.4.0-64.85 ProcVersionSignature: Ubuntu 4.4.0-64.85-generic 4.4.44 Uname: Linux 4.4.0-64-generic x86_64 ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 Date: Mon Feb 27 21:06:44 2017 InstallationDate: Installed on 2016-06-08 (264 days ago) InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1) Lsusb: Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 8087:0a2b Intel Corp. Bus 001 Device 002: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-64-generic.efi.signed root=UUID=1f60f9c9-bbf1-45df-bdc3-9b4da883839e ro quiet splash net.ifnames=0 vt.handoff=7 SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.board.name: NUC6i5SYB To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1668356/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1668356] Re: Hard lockup after 4 hours uptime
Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.10 kernel[0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'. Once testing of the upstream kernel is complete, please mark this bug as "Confirmed". Thanks in advance. [0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.10 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1668356 Title: Hard lockup after 4 hours uptime Status in linux package in Ubuntu: Triaged Status in linux source package in Xenial: Triaged Bug description: We have recently deployed intel NUC6i5 devices in store for POS display and are encountering strange inexplicable freezes on several devices. What is very strange is that all devices are freezing exactly 4h after boot. We have had this exact same issue on more than 20 devices (over 100), with parts from different batches, and all did freeze exactly 4h after boot (but it's not reproducible, it won't freeze every day). Some devices are playing MPV videos, while other run chromium. They are running non-stop, but are all rebooting daily at 05h05. Looks like it might be related to https://lkml.org/lkml/2015/6/11/787 That seem to have been fixed and backported already https://lkml.org/lkml/2015/10/17/259 ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-64-generic 4.4.0-64.85 ProcVersionSignature: Ubuntu 4.4.0-64.85-generic 4.4.44 Uname: Linux 4.4.0-64-generic x86_64 ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 Date: Mon Feb 27 21:06:44 2017 InstallationDate: Installed on 2016-06-08 (264 days ago) InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1) Lsusb: Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 8087:0a2b Intel Corp. Bus 001 Device 002: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-64-generic.efi.signed root=UUID=1f60f9c9-bbf1-45df-bdc3-9b4da883839e ro quiet splash net.ifnames=0 vt.handoff=7 SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.board.name: NUC6i5SYB To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1668356/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1668356] Re: Hard lockup after 4 hours uptime
The lkml tread you referenced in the bug description was for commit 37b12910dd11d9ab969f2c310dc9160b7f3e3405. That commit landed upstream in v4.3.rc1, so it is already in the 4.4 based Xenial kernel. Did this issue start happening after a recent upgrade, or after applying updates? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1668356 Title: Hard lockup after 4 hours uptime Status in linux package in Ubuntu: Triaged Status in linux source package in Xenial: Triaged Bug description: We have recently deployed intel NUC6i5 devices in store for POS display and are encountering strange inexplicable freezes on several devices. What is very strange is that all devices are freezing exactly 4h after boot. We have had this exact same issue on more than 20 devices (over 100), with parts from different batches, and all did freeze exactly 4h after boot (but it's not reproducible, it won't freeze every day). Some devices are playing MPV videos, while other run chromium. They are running non-stop, but are all rebooting daily at 05h05. Looks like it might be related to https://lkml.org/lkml/2015/6/11/787 That seem to have been fixed and backported already https://lkml.org/lkml/2015/10/17/259 ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-64-generic 4.4.0-64.85 ProcVersionSignature: Ubuntu 4.4.0-64.85-generic 4.4.44 Uname: Linux 4.4.0-64-generic x86_64 ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 Date: Mon Feb 27 21:06:44 2017 InstallationDate: Installed on 2016-06-08 (264 days ago) InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1) Lsusb: Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 8087:0a2b Intel Corp. Bus 001 Device 002: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-64-generic.efi.signed root=UUID=1f60f9c9-bbf1-45df-bdc3-9b4da883839e ro quiet splash net.ifnames=0 vt.handoff=7 SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.board.name: NUC6i5SYB To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1668356/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1668356] Re: Hard lockup after 4 hours uptime
** Changed in: linux (Ubuntu) Importance: Undecided => Medium ** Also affects: linux (Ubuntu Xenial) Importance: Undecided Status: New ** Changed in: linux (Ubuntu) Importance: Medium => High ** Changed in: linux (Ubuntu Xenial) Importance: Undecided => High ** Changed in: linux (Ubuntu Xenial) Status: New => Triaged ** Changed in: linux (Ubuntu) Status: Confirmed => Triaged ** Tags added: kernel-da-key -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1668356 Title: Hard lockup after 4 hours uptime Status in linux package in Ubuntu: Triaged Status in linux source package in Xenial: Triaged Bug description: We have recently deployed intel NUC6i5 devices in store for POS display and are encountering strange inexplicable freezes on several devices. What is very strange is that all devices are freezing exactly 4h after boot. We have had this exact same issue on more than 20 devices (over 100), with parts from different batches, and all did freeze exactly 4h after boot (but it's not reproducible, it won't freeze every day). Some devices are playing MPV videos, while other run chromium. They are running non-stop, but are all rebooting daily at 05h05. Looks like it might be related to https://lkml.org/lkml/2015/6/11/787 That seem to have been fixed and backported already https://lkml.org/lkml/2015/10/17/259 ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-64-generic 4.4.0-64.85 ProcVersionSignature: Ubuntu 4.4.0-64.85-generic 4.4.44 Uname: Linux 4.4.0-64-generic x86_64 ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 Date: Mon Feb 27 21:06:44 2017 InstallationDate: Installed on 2016-06-08 (264 days ago) InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1) Lsusb: Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 8087:0a2b Intel Corp. Bus 001 Device 002: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-64-generic.efi.signed root=UUID=1f60f9c9-bbf1-45df-bdc3-9b4da883839e ro quiet splash net.ifnames=0 vt.handoff=7 SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.board.name: NUC6i5SYB To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1668356/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1668356] Re: Hard lockup after 4 hours uptime
DHCPREQUEST of 10.34.242.77 on eth0 to 10.32.65.65 port 67 (xid=0x30fdf923) BUG: unable to handle kernel NULL pointer dereference at (null) IP: [] timecounter_read+0x13/0x60 PGD 0 Oops: [#1] SMP Modules linked in: rfcomm ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_multiport xfrm_user xfrm_algo iptable_nat nf_nat_ipv4 br_netfilter bridge stp llc aufs pl2303 usbserial bnep arc4 snd_hda_codec_hdmi snd_soc_skl snd_soc_skl_ipc snd_hda_ext_core snd_soc_sst_ipc snd_hda_codec_realtek snd_soc_sst_dsp snd_hda_codec_generic nls_iso8859_1 snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine dw_dmac_core snd_hda_intel iwlmvm snd_hda_codec snd_hda_core intel_rapl 8250_dw snd_hwdep mac80211 x86_pkg_temp_thermal intel_powerclamp coretemp snd_pcm kvm_intel kvm snd_seq_midi snd_seq_midi_event irqbypass crct10dif_pclmul crc32_pclmul iwlwifi ghash_clmulni_intel snd_rawmidi aesni_intel snd_seq aes_x86_64 lrw gf128mul glue_helper cfg80211 snd_seq_device ablk_helper cryptd snd_timer snd soundcore idma64 virt_dma shpchp ir_lirc_codec ir_xmp_decoder lirc_dev ir_mce_kbd_decoder ir_sharp_decoder intel_lpss_pci ir_sanyo_decoder btusb ir_sony_decoder hci_uart btrtl ir_jvc_decoder ir_rc6_decoder btbcm btqca ir_rc5_decoder btintel ir_nec_decoder bluetooth mei_me rc_rc6_mce ite_cir rc_core intel_lpss_acpi intel_lpss mei acpi_pad mac_hid acpi_als kfifo_buf industrialio ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_tcpudp xt_addrtype nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables parport_pc ppdev sunrpc lp parport autofs4 i915_bpo intel_ips i2c_algo_bit drm_kms_helper syscopyarea e1000e sysfillrect sysimgblt fb_sys_fops ptp sdhci_pci ahci drm pps_core sdhci libahci video pinctrl_sunrisepoint i2c_hid pinctrl_intel hid fjes CPU: 3 PID: 15471 Comm: kworker/3:0 Not tainted 4.4.0-64-generic #85-Ubuntu Hardware name: /NUC6i5SYB, BIOS SYSKLi35.86A.0051.2016.0804.1114 08/04/2016 Workqueue: events e1000e_systim_overflow_work [e1000e] task: 880031f32d00 ti: 8800350e8000 task.ti: 8800350e8000 RIP: 0010:[] [] timecounter_read+0x13/0x60 RSP: 0018:8800350ebdb0 EFLAGS: 00010046 RAX: RBX: 8800353ab7a0 RCX: 0001 RDX: 0001 RSI: 8800350ebdf8 RDI: RBP: 8800350ebdb8 R08: 88016ed965c0 R09: R10: 00010035 R11: 0001 R12: 8800353ab780 R13: 8800350ebdf8 R14: 0246 R15: 8800353ab6d0 FS: () GS:88016ed8() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: CR3: 02e0a000 CR4: 003406e0 Stack: 8800353ab7d0 8800350ebde8 c014d36e 8800353ab6d0 88016ed965c0 88016ed9af00 00c0 8800350ebe18 c014d521 81837e26 88016ed9af00 a91221c0 Call Trace: [] e1000e_phc_gettime+0x2e/0x60 [e1000e] [] e1000e_systim_overflow_work+0x31/0xa0 [e1000e] [] ? __schedule+0x3b6/0xa30 [] process_one_work+0x165/0x480 [] worker_thread+0x4b/0x4c0 [] ? process_one_work+0x480/0x480 [] ? process_one_work+0x480/0x480 [] kthread+0xd8/0xf0 [] ? kthread_create_on_node+0x1e0/0x1e0 [] ret_from_fork+0x3f/0x70 [] ? kthread_create_on_node+0x1e0/0x1e0 Code: 00 48 d3 e0 48 83 e8 01 48 89 43 18 5b 41 5c 41 5d 5d c3 0f 1f 44 00 00 0f 1f 44 00 00 55 48 89 e5 53 48 8b 07 48 89 fb 48 89 c7 10 48 8b 33 48 89 c2 48 2b 53 08 8b 4e 10 48 23 56 08 48 0f RIP [] timecounter_read+0x13/0x60 RSP CR2: ---[ end trace 7d024538180dff79 ]--- BUG: unable to handle kernel paging request at ffd8 IP: [] kthread_data+0x10/0x20 PGD 2e0d067 PUD 2e0f067 PMD 0 Oops: [#2] SMP Modules linked in: rfcomm ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_multiport xfrm_user xfrm_algo iptable_nat nf_nat_ipv4 br_netfilter bridge stp llc aufs pl2303 usbserial bnep arc4 snd_hda_codec_hdmi snd_soc_skl snd_soc_skl_ipc snd_hda_ext_core snd_soc_sst_ipc snd_hda_codec_realtek snd_soc_sst_dsp snd_hda_codec_generic nls_iso8859_1 snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine dw_dmac_core snd_hda_intel iwlmvm snd_hda_codec snd_hda_core intel_rapl 8250_dw snd_hwdep mac80211 x86_pkg_temp_thermal intel_powerclamp coretemp snd_pcm kvm_intel kvm snd_seq_midi snd_seq_midi_event irqbypass crct10dif_pclmul crc32_pclmul iwlwifi ghash_clmulni_intel snd_rawmidi aesni_intel snd_seq aes_x86_64 lrw gf128mul glue_helper cfg80211 snd_seq_device ablk_helper cryptd snd_timer snd soundcore idma64 virt_dma shpchp ir_lirc_codec ir_xmp_decoder lirc_dev ir_mce_kbd_decoder ir_sharp_decoder intel_lpss_pci ir_sanyo_decoder btusb ir_sony_decoder hci_uart btrtl ir_jvc_decoder ir_rc6_decoder btbcm btqca
