[Ocfs2-users] invalid opcode bug in dlmglue?
We've gotten a couple of dumps likes this in the last couple of days while migrating some new users to our mail store which involves untarring/moving large quantities of files. We've gracefully rebooted the node after every instance and it seems to do fine with normal mail operations. I'm wondering if you have any thoughts on the messages? Running in ESX 3.5. The kernel is Debian 2.6.30 based. Storage backend is iSCSI EqualLogic. Only one node currently has the FS mounted. Thanks, Brian Feb 4 09:34:41 iris kernel: [528465.151651] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Feb 4 09:34:41 iris kernel: [528465.151722] Process rm (pid: 32114, ti=dfad4000 task=e17b9610 task.ti=dfad4000) Feb 4 09:34:41 iris kernel: [528465.147544] [ cut here ] Feb 4 09:34:41 iris kernel: [528465.148706] kernel BUG at fs/ocfs2/dlmglue.c:2470! Feb 4 09:34:41 iris kernel: [528465.148818] invalid opcode: [#1] SMP Feb 4 09:34:41 iris kernel: [528465.148983] last sysfs file: /sys/devices/system/clocksource/clocksource0/available_clocksource Feb 4 09:34:41 iris kernel: [528465.149113] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_stack_o2cb ocfs2_stackglue netconsole vmsync vmmemctl ocfs2_dlmfs ocfs2_dlm ocfs2_nodemanager configfs usbhid hid uhci_hcd ohci_hcd eh ci_hcd usbcore psmouse evdev serio_raw parport_pc parport snd_pcsp snd_pcm snd_timer snd soundcore snd_page_alloc container button ac i2c_piix4 processor i2c_core intel_agp shpchp agpgart pci_hotplug ext3 jbd mbcache dm_mirror dm_region_ha sh dm_log dm_snapshot dm_mod sd_mod crc_t10dif ide_cd_mod cdrom ata_generic libata ide_pci_generic floppy mptspi mptscsih mptbase scsi_transport_spi scsi_mod vmxnet piix ide_core thermal fan thermal_sys [last unloaded: scsi_wait_scan] Feb 4 09:34:41 iris kernel: [528465.150763] Feb 4 09:34:41 iris kernel: [528465.150945] Pid: 32114, comm: rm Not tainted (2.6.30-vmwareguest-smp-64g.20090711 #1) VMware Virtual Platform Feb 4 09:34:41 iris kernel: [528465.151104] EIP: 0060:[f887783d] EFLAGS: 00010246 CPU: 2 Feb 4 09:34:41 iris kernel: [528465.151446] EIP is at ocfs2_dentry_lock+0x26/0xf7 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.151520] EAX: f6548800 EBX: c8c53c6c ECX: EDX: Feb 4 09:34:41 iris kernel: [528465.151586] ESI: 17395beb EDI: f6538000 EBP: 0005 ESP: dfad5e88 Feb 4 09:34:41 iris kernel: [528465.151651] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Feb 4 09:34:41 iris kernel: [528465.151722] Process rm (pid: 32114, ti=dfad4000 task=e17b9610 task.ti=dfad4000) Feb 4 09:34:41 iris kernel: [528465.151814] Stack: Feb 4 09:34:41 iris kernel: [528465.151886] 0001 c75fe8dc c8c53c6c 17395beb c8c53c6c f888fdcc Feb 4 09:34:41 iris kernel: [528465.152084] 17395beb f88925c9 f6538000 f0987900 c75fe940 c75fe5c0 Feb 4 09:34:41 iris kernel: [528465.152303] e6472e10 Feb 4 09:34:41 iris kernel: [528465.152566] Call Trace: Feb 4 09:34:41 iris kernel: [528465.152580] [f888fdcc] ? ocfs2_remote_dentry_delete+0xe/0x95 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.152872] [f88925c9] ? ocfs2_unlink+0x3fe/0xa26 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.152960] [c019ab62] ? vfs_unlink+0x5c/0x95 Feb 4 09:34:41 iris kernel: [528465.153165] [c019be31] ? do_unlinkat+0x93/0xfc Feb 4 09:34:41 iris kernel: [528465.153240] [c0114001] ? smp_reschedule_interrupt+0x13/0x1c Feb 4 09:34:41 iris kernel: [528465.153336] [c0107eda] ? reschedule_interrupt+0x2a/0x30 Feb 4 09:34:41 iris kernel: [528465.153413] [c01077d4] ? sysenter_do_call+0x12/0x28 Feb 4 09:34:41 iris kernel: [528465.153538] Code: e9 19 fe ff ff 55 57 56 53 83 ec 0c 83 fa 01 8b 50 58 19 ed 83 e5 fe 83 c5 05 89 54 24 04 8b 40 54 85 d2 8b b8 98 01 00 00 75 04 0f 0b eb fe 8d 9f 9c 00 00 00 89 d8 e8 2a 1e ab c7 8b 87 a4 00 Feb 4 09:34:41 iris kernel: [528465.154876] EIP: [f887783d] ocfs2_dentry_lock+0x26/0xf7 [ocfs2] SS:ESP 0068:dfad5e88 Feb 4 09:34:41 iris kernel: [528465.155307] ---[ end trace 62c828cac153c25f ]--- signature.asc Description: Digital signature ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] invalid opcode bug in dlmglue?
Excellent. Thanks for the quick response. Also, any idea when the tools might support indexed dirs? I suspect we'll have some downtime coming up in a couple of months and am wondering if we can use the opportunity to turn on that feature for quicker lookup times. Thanks, Brian Sunil Mushran sunil.mush...@oracle.com 2010-02-04 09:16: Fixed. http://oss.oracle.com/bugzilla/show_bug.cgi?id=1137 You probably already have this patch. If not, add it. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=a5a0a630922a2f6a774b6dac19f70cb5abd86bb0 You are definitely missing this patch. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=a1b08e75dff3dc18a88444803753e667bb1d126e Brian Kroth wrote: We've gotten a couple of dumps likes this in the last couple of days while migrating some new users to our mail store which involves untarring/moving large quantities of files. We've gracefully rebooted the node after every instance and it seems to do fine with normal mail operations. I'm wondering if you have any thoughts on the messages? Running in ESX 3.5. The kernel is Debian 2.6.30 based. Storage backend is iSCSI EqualLogic. Only one node currently has the FS mounted. Thanks, Brian Feb 4 09:34:41 iris kernel: [528465.151651] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Feb 4 09:34:41 iris kernel: [528465.151722] Process rm (pid: 32114, ti=dfad4000 task=e17b9610 task.ti=dfad4000) Feb 4 09:34:41 iris kernel: [528465.147544] [ cut here ] Feb 4 09:34:41 iris kernel: [528465.148706] kernel BUG at fs/ocfs2/dlmglue.c:2470! Feb 4 09:34:41 iris kernel: [528465.148818] invalid opcode: [#1] SMP Feb 4 09:34:41 iris kernel: [528465.148983] last sysfs file: /sys/devices/system/clocksource/clocksource0/available_clocksource Feb 4 09:34:41 iris kernel: [528465.149113] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_stack_o2cb ocfs2_stackglue netconsole vmsync vmmemctl ocfs2_dlmfs ocfs2_dlm ocfs2_nodemanager configfs usbhid hid uhci_hcd ohci_hcd eh ci_hcd usbcore psmouse evdev serio_raw parport_pc parport snd_pcsp snd_pcm snd_timer snd soundcore snd_page_alloc container button ac i2c_piix4 processor i2c_core intel_agp shpchp agpgart pci_hotplug ext3 jbd mbcache dm_mirror dm_region_ha sh dm_log dm_snapshot dm_mod sd_mod crc_t10dif ide_cd_mod cdrom ata_generic libata ide_pci_generic floppy mptspi mptscsih mptbase scsi_transport_spi scsi_mod vmxnet piix ide_core thermal fan thermal_sys [last unloaded: scsi_wait_scan] Feb 4 09:34:41 iris kernel: [528465.150763] Feb 4 09:34:41 iris kernel: [528465.150945] Pid: 32114, comm: rm Not tainted (2.6.30-vmwareguest-smp-64g.20090711 #1) VMware Virtual Platform Feb 4 09:34:41 iris kernel: [528465.151104] EIP: 0060:[f887783d] EFLAGS: 00010246 CPU: 2 Feb 4 09:34:41 iris kernel: [528465.151446] EIP is at ocfs2_dentry_lock+0x26/0xf7 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.151520] EAX: f6548800 EBX: c8c53c6c ECX: EDX: Feb 4 09:34:41 iris kernel: [528465.151586] ESI: 17395beb EDI: f6538000 EBP: 0005 ESP: dfad5e88 Feb 4 09:34:41 iris kernel: [528465.151651] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Feb 4 09:34:41 iris kernel: [528465.151722] Process rm (pid: 32114, ti=dfad4000 task=e17b9610 task.ti=dfad4000) Feb 4 09:34:41 iris kernel: [528465.151814] Stack: Feb 4 09:34:41 iris kernel: [528465.151886] 0001 c75fe8dc c8c53c6c 17395beb c8c53c6c f888fdcc Feb 4 09:34:41 iris kernel: [528465.152084] 17395beb f88925c9 f6538000 f0987900 c75fe940 c75fe5c0 Feb 4 09:34:41 iris kernel: [528465.152303] e6472e10 Feb 4 09:34:41 iris kernel: [528465.152566] Call Trace: Feb 4 09:34:41 iris kernel: [528465.152580] [f888fdcc] ? ocfs2_remote_dentry_delete+0xe/0x95 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.152872] [f88925c9] ? ocfs2_unlink+0x3fe/0xa26 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.152960] [c019ab62] ? vfs_unlink+0x5c/0x95 Feb 4 09:34:41 iris kernel: [528465.153165] [c019be31] ? do_unlinkat+0x93/0xfc Feb 4 09:34:41 iris kernel: [528465.153240] [c0114001] ? smp_reschedule_interrupt+0x13/0x1c Feb 4 09:34:41 iris kernel: [528465.153336] [c0107eda] ? reschedule_interrupt+0x2a/0x30 Feb 4 09:34:41 iris kernel: [528465.153413] [c01077d4] ? sysenter_do_call+0x12/0x28 Feb 4 09:34:41 iris kernel: [528465.153538] Code: e9 19 fe ff ff 55 57 56 53 83 ec 0c 83 fa 01 8b 50 58 19 ed 83 e5 fe 83 c5 05 89 54 24 04 8b 40 54 85 d2 8b b8 98 01 00 00 75 04 0f 0b eb fe 8d 9f 9c 00 00 00 89 d8 e8 2a 1e ab c7 8b 87 a4 00 Feb 4 09:34:41 iris kernel: [528465.154876] EIP: [f887783d] ocfs2_dentry_lock+0x26/0xf7 [ocfs2] SS:ESP 0068:dfad5e88 Feb 4 09:34:41 iris kernel: [528465.155307] ---[ end trace 62c828cac153c25f ]--- signature.asc Description: Digital
Re: [Ocfs2-users] invalid opcode bug in dlmglue?
http://oss.oracle.com/pipermail/ocfs2-tools-devel/2010-January/thread.html The dx_dir patches are in the review cycle. Brian Kroth wrote: Excellent. Thanks for the quick response. Also, any idea when the tools might support indexed dirs? I suspect we'll have some downtime coming up in a couple of months and am wondering if we can use the opportunity to turn on that feature for quicker lookup times. Thanks, Brian Sunil Mushran sunil.mush...@oracle.com 2010-02-04 09:16: Fixed. http://oss.oracle.com/bugzilla/show_bug.cgi?id=1137 You probably already have this patch. If not, add it. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=a5a0a630922a2f6a774b6dac19f70cb5abd86bb0 You are definitely missing this patch. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=a1b08e75dff3dc18a88444803753e667bb1d126e Brian Kroth wrote: We've gotten a couple of dumps likes this in the last couple of days while migrating some new users to our mail store which involves untarring/moving large quantities of files. We've gracefully rebooted the node after every instance and it seems to do fine with normal mail operations. I'm wondering if you have any thoughts on the messages? Running in ESX 3.5. The kernel is Debian 2.6.30 based. Storage backend is iSCSI EqualLogic. Only one node currently has the FS mounted. Thanks, Brian Feb 4 09:34:41 iris kernel: [528465.151651] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Feb 4 09:34:41 iris kernel: [528465.151722] Process rm (pid: 32114, ti=dfad4000 task=e17b9610 task.ti=dfad4000) Feb 4 09:34:41 iris kernel: [528465.147544] [ cut here ] Feb 4 09:34:41 iris kernel: [528465.148706] kernel BUG at fs/ocfs2/dlmglue.c:2470! Feb 4 09:34:41 iris kernel: [528465.148818] invalid opcode: [#1] SMP Feb 4 09:34:41 iris kernel: [528465.148983] last sysfs file: /sys/devices/system/clocksource/clocksource0/available_clocksource Feb 4 09:34:41 iris kernel: [528465.149113] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_stack_o2cb ocfs2_stackglue netconsole vmsync vmmemctl ocfs2_dlmfs ocfs2_dlm ocfs2_nodemanager configfs usbhid hid uhci_hcd ohci_hcd eh ci_hcd usbcore psmouse evdev serio_raw parport_pc parport snd_pcsp snd_pcm snd_timer snd soundcore snd_page_alloc container button ac i2c_piix4 processor i2c_core intel_agp shpchp agpgart pci_hotplug ext3 jbd mbcache dm_mirror dm_region_ha sh dm_log dm_snapshot dm_mod sd_mod crc_t10dif ide_cd_mod cdrom ata_generic libata ide_pci_generic floppy mptspi mptscsih mptbase scsi_transport_spi scsi_mod vmxnet piix ide_core thermal fan thermal_sys [last unloaded: scsi_wait_scan] Feb 4 09:34:41 iris kernel: [528465.150763] Feb 4 09:34:41 iris kernel: [528465.150945] Pid: 32114, comm: rm Not tainted (2.6.30-vmwareguest-smp-64g.20090711 #1) VMware Virtual Platform Feb 4 09:34:41 iris kernel: [528465.151104] EIP: 0060:[f887783d] EFLAGS: 00010246 CPU: 2 Feb 4 09:34:41 iris kernel: [528465.151446] EIP is at ocfs2_dentry_lock+0x26/0xf7 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.151520] EAX: f6548800 EBX: c8c53c6c ECX: EDX: Feb 4 09:34:41 iris kernel: [528465.151586] ESI: 17395beb EDI: f6538000 EBP: 0005 ESP: dfad5e88 Feb 4 09:34:41 iris kernel: [528465.151651] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Feb 4 09:34:41 iris kernel: [528465.151722] Process rm (pid: 32114, ti=dfad4000 task=e17b9610 task.ti=dfad4000) Feb 4 09:34:41 iris kernel: [528465.151814] Stack: Feb 4 09:34:41 iris kernel: [528465.151886] 0001 c75fe8dc c8c53c6c 17395beb c8c53c6c f888fdcc Feb 4 09:34:41 iris kernel: [528465.152084] 17395beb f88925c9 f6538000 f0987900 c75fe940 c75fe5c0 Feb 4 09:34:41 iris kernel: [528465.152303] e6472e10 Feb 4 09:34:41 iris kernel: [528465.152566] Call Trace: Feb 4 09:34:41 iris kernel: [528465.152580] [f888fdcc] ? ocfs2_remote_dentry_delete+0xe/0x95 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.152872] [f88925c9] ? ocfs2_unlink+0x3fe/0xa26 [ocfs2] Feb 4 09:34:41 iris kernel: [528465.152960] [c019ab62] ? vfs_unlink+0x5c/0x95 Feb 4 09:34:41 iris kernel: [528465.153165] [c019be31] ? do_unlinkat+0x93/0xfc Feb 4 09:34:41 iris kernel: [528465.153240] [c0114001] ? smp_reschedule_interrupt+0x13/0x1c Feb 4 09:34:41 iris kernel: [528465.153336] [c0107eda] ? reschedule_interrupt+0x2a/0x30 Feb 4 09:34:41 iris kernel: [528465.153413] [c01077d4] ? sysenter_do_call+0x12/0x28 Feb 4 09:34:41 iris kernel: [528465.153538] Code: e9 19 fe ff ff 55 57 56 53 83 ec 0c 83 fa 01 8b 50 58 19 ed 83 e5 fe 83 c5 05 89 54 24 04 8b 40 54 85 d2 8b b8 98 01 00 00 75 04 0f 0b eb fe 8d 9f 9c 00 00 00 89 d8 e8 2a 1e ab c7 8b 87 a4 00 Feb 4 09:34:41 iris kernel: [528465.154876] EIP: [f887783d] ocfs2_dentry_lock+0x26/0xf7