Re: Kernel trace, how to debug?
On Sun, Apr 03, 2011 at 08:41:51PM +0200, David Sastre wrote: > On Sun, Apr 03, 2011 at 07:20:08PM +0200, David Sastre wrote: > > Hello, > > > > I got a kernel trace related to some I/O error. > > I'd like to debug it further, but don't not how to proceed. > > I have some basic gdb skills, but the pkg involved is the kernel, so > > I can't just run 'gdb executable trace'... > > > > The trace looks like this: > > > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554092] [ cut > > here ] > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554096] WARNING: at > > /build/buildd-linux-2.6_2.6.32-30-i386-UYhWt7/linux-2.6-2.6.32/debian/build/source_i386_none/fs/buffer.c:1160 > > mark_buffer_dirty+0x20/0x7a() > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554098] Hardware name: > > P55-US3L > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554099] Modules linked > > in: tun ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables > > dahdi_echocan_oslec echo wcfxo dahdi crc_ccitt ppdev lp kvm_intel kvm > > binfmt_misc fuse bridge stp loop saa7134_alsa tda10048 saa7134_dvb > > videobuf_dvb dvb_core snd_hda_codec_atihdmi snd_hda_codec_realtek tda18271 > > tda8290 joydev tuner usbhid hid snd_hda_intel saa7134 sg snd_hda_codec > > ir_common sr_mod v4l2_common videodev snd_hwdep snd_pcm_oss v4l1_compat > > videobuf_dma_sg snd_seq_midi snd_mixer_oss videobuf_core snd_rawmidi > > snd_pcm r8169 snd_seq_midi_event cdrom usb_storage snd_seq ata_generic > > uhci_hcd snd_timer ehci_hcd mii tulip tveeprom ata_piix usbcore > > snd_seq_device ahci fglrx(P) snd serio_raw pata_jmicron soundcore > > snd_page_alloc nls_base pcspkr i2c_i801 i2c_core evdev processor button > > parport_pc parport ext3 jbd mbcache dm_mod raid1 md_mod sd_mod crc_t10dif > > sata_via libata scsi_mod thermal thermal_sys > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554140] Pid: 18351, comm: > > amarok Tainted: P 2.6.32-5-686-bigmem #1 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554142] Call Trace: > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554146] [] ? > > warn_slowpath_common+0x5e/0x8a > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554148] [] ? > > warn_slowpath_null+0xa/0xc > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554150] [] ? > > mark_buffer_dirty+0x20/0x7a > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554157] [] ? > > ext3_commit_super+0x40/0x56 [ext3] > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554162] [] ? > > ext3_handle_error+0x6e/0x8e [ext3] > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554166] [] ? > > ext3_error+0x3a/0x40 [ext3] > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554172] [] ? > > ext3_free_branches+0xa3/0x1e9 [ext3] > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554178] [] ? > > ext3_truncate+0x4ec/0x70c [ext3] > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554182] [] ? > > journal_stop+0x254/0x260 [jbd] > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554187] [] ? > > ext3_write_begin+0x1ac/0x1d2 [ext3] > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554191] [] ? > > generic_file_buffered_write+0xd0/0x21d > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554194] [] ? > > __generic_file_aio_write+0x265/0x28d > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554197] [] ? > > touch_atime+0x69/0xd9 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554199] [] ? > > generic_file_aio_write+0x51/0x93 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554202] [] ? > > do_sync_write+0xc0/0x107 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554205] [] ? > > cp_new_stat64+0xf7/0x109 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554208] [] ? > > autoremove_wake_function+0x0/0x2d > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554211] [] ? > > sys_fstat64+0x1e/0x23 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554214] [] ? > > security_file_permission+0xc/0xd > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554216] [] ? > > do_sync_write+0x0/0x107 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554217] [] ? > > vfs_write+0x7e/0xd6 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554219] [] ? > > sys_write+0x3c/0x63 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554222] [] ? > > sysenter_do_call+0x12/0x28 > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554223] ---[ end trace > > 20f4b55a0560b754 ]--- > > > > FWIW, I have dbg symbols installed for this kernel: > > > > ii linux-image-2.6.32-5-686-bigmem 2.6.32-30 > > Linux 2.6.32 for PCs with 4GB+ RAM > > ii linux-image-2.6.32-5-686-bigmem-dbg 2.6.32-31 > > Debugging infos for Linux 2.6.32-5-686-bigmem > > Can anybody tell me how to properly enable ddebug on a 2.6.32-5 kernel? > > I've been able to find some info regarding 2.6.38¹, but that doesn't
Re: Kernel trace, how to debug?
On Sun, Apr 03, 2011 at 07:20:08PM +0200, David Sastre wrote: > Hello, > > I got a kernel trace related to some I/O error. > I'd like to debug it further, but don't not how to proceed. > I have some basic gdb skills, but the pkg involved is the kernel, so > I can't just run 'gdb executable trace'... > > The trace looks like this: > > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554092] [ cut > here ] > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554096] WARNING: at > /build/buildd-linux-2.6_2.6.32-30-i386-UYhWt7/linux-2.6-2.6.32/debian/build/source_i386_none/fs/buffer.c:1160 > mark_buffer_dirty+0x20/0x7a() > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554098] Hardware name: > P55-US3L > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554099] Modules linked in: > tun ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables > dahdi_echocan_oslec echo wcfxo dahdi crc_ccitt ppdev lp kvm_intel kvm > binfmt_misc fuse bridge stp loop saa7134_alsa tda10048 saa7134_dvb > videobuf_dvb dvb_core snd_hda_codec_atihdmi snd_hda_codec_realtek tda18271 > tda8290 joydev tuner usbhid hid snd_hda_intel saa7134 sg snd_hda_codec > ir_common sr_mod v4l2_common videodev snd_hwdep snd_pcm_oss v4l1_compat > videobuf_dma_sg snd_seq_midi snd_mixer_oss videobuf_core snd_rawmidi snd_pcm > r8169 snd_seq_midi_event cdrom usb_storage snd_seq ata_generic uhci_hcd > snd_timer ehci_hcd mii tulip tveeprom ata_piix usbcore snd_seq_device ahci > fglrx(P) snd serio_raw pata_jmicron soundcore snd_page_alloc nls_base pcspkr > i2c_i801 i2c_core evdev processor button parport_pc parport ext3 jbd mbcache > dm_mod raid1 md_mod sd_mod crc_t10dif sata_via libata scsi_mod thermal > thermal_sys > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554140] Pid: 18351, comm: > amarok Tainted: P 2.6.32-5-686-bigmem #1 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554142] Call Trace: > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554146] [] ? > warn_slowpath_common+0x5e/0x8a > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554148] [] ? > warn_slowpath_null+0xa/0xc > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554150] [] ? > mark_buffer_dirty+0x20/0x7a > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554157] [] ? > ext3_commit_super+0x40/0x56 [ext3] > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554162] [] ? > ext3_handle_error+0x6e/0x8e [ext3] > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554166] [] ? > ext3_error+0x3a/0x40 [ext3] > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554172] [] ? > ext3_free_branches+0xa3/0x1e9 [ext3] > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554178] [] ? > ext3_truncate+0x4ec/0x70c [ext3] > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554182] [] ? > journal_stop+0x254/0x260 [jbd] > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554187] [] ? > ext3_write_begin+0x1ac/0x1d2 [ext3] > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554191] [] ? > generic_file_buffered_write+0xd0/0x21d > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554194] [] ? > __generic_file_aio_write+0x265/0x28d > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554197] [] ? > touch_atime+0x69/0xd9 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554199] [] ? > generic_file_aio_write+0x51/0x93 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554202] [] ? > do_sync_write+0xc0/0x107 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554205] [] ? > cp_new_stat64+0xf7/0x109 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554208] [] ? > autoremove_wake_function+0x0/0x2d > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554211] [] ? > sys_fstat64+0x1e/0x23 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554214] [] ? > security_file_permission+0xc/0xd > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554216] [] ? > do_sync_write+0x0/0x107 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554217] [] ? > vfs_write+0x7e/0xd6 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554219] [] ? > sys_write+0x3c/0x63 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554222] [] ? > sysenter_do_call+0x12/0x28 > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554223] ---[ end trace > 20f4b55a0560b754 ]--- > > FWIW, I have dbg symbols installed for this kernel: > > ii linux-image-2.6.32-5-686-bigmem 2.6.32-30 > Linux 2.6.32 for PCs with 4GB+ RAM > ii linux-image-2.6.32-5-686-bigmem-dbg 2.6.32-31 > Debugging infos for Linux 2.6.32-5-686-bigmem Can anybody tell me how to properly enable ddebug on a 2.6.32-5 kernel? I've been able to find some info regarding 2.6.38¹, but that doesn't seem to apply to 2.6.32, as there is no /sys/kernel/debug/dynamic_debug directory after mounting the debufgs: # mount -t debugfs none /sys/kernel/debug/ # ll /sys/kernel/debug/ total 0 drwxr-xr-x 40 root root 0 abr 3 19:58 bdi drwxr-xr-x 4 root
Kernel trace, how to debug?
Hello, I got a kernel trace related to some I/O error. I'd like to debug it further, but don't not how to proceed. I have some basic gdb skills, but the pkg involved is the kernel, so I can't just run 'gdb executable trace'... The trace looks like this: 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554092] [ cut here ] 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554096] WARNING: at /build/buildd-linux-2.6_2.6.32-30-i386-UYhWt7/linux-2.6-2.6.32/debian/build/source_i386_none/fs/buffer.c:1160 mark_buffer_dirty+0x20/0x7a() 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554098] Hardware name: P55-US3L 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554099] Modules linked in: tun ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables dahdi_echocan_oslec echo wcfxo dahdi crc_ccitt ppdev lp kvm_intel kvm binfmt_misc fuse bridge stp loop saa7134_alsa tda10048 saa7134_dvb videobuf_dvb dvb_core snd_hda_codec_atihdmi snd_hda_codec_realtek tda18271 tda8290 joydev tuner usbhid hid snd_hda_intel saa7134 sg snd_hda_codec ir_common sr_mod v4l2_common videodev snd_hwdep snd_pcm_oss v4l1_compat videobuf_dma_sg snd_seq_midi snd_mixer_oss videobuf_core snd_rawmidi snd_pcm r8169 snd_seq_midi_event cdrom usb_storage snd_seq ata_generic uhci_hcd snd_timer ehci_hcd mii tulip tveeprom ata_piix usbcore snd_seq_device ahci fglrx(P) snd serio_raw pata_jmicron soundcore snd_page_alloc nls_base pcspkr i2c_i801 i2c_core evdev processor button parport_pc parport ext3 jbd mbcache dm_mod raid1 md_mod sd_mod crc_t10dif sata_via libata scsi_mod thermal thermal_sys 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554140] Pid: 18351, comm: amarok Tainted: P 2.6.32-5-686-bigmem #1 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554142] Call Trace: 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554146] [] ? warn_slowpath_common+0x5e/0x8a 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554148] [] ? warn_slowpath_null+0xa/0xc 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554150] [] ? mark_buffer_dirty+0x20/0x7a 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554157] [] ? ext3_commit_super+0x40/0x56 [ext3] 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554162] [] ? ext3_handle_error+0x6e/0x8e [ext3] 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554166] [] ? ext3_error+0x3a/0x40 [ext3] 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554172] [] ? ext3_free_branches+0xa3/0x1e9 [ext3] 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554178] [] ? ext3_truncate+0x4ec/0x70c [ext3] 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554182] [] ? journal_stop+0x254/0x260 [jbd] 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554187] [] ? ext3_write_begin+0x1ac/0x1d2 [ext3] 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554191] [] ? generic_file_buffered_write+0xd0/0x21d 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554194] [] ? __generic_file_aio_write+0x265/0x28d 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554197] [] ? touch_atime+0x69/0xd9 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554199] [] ? generic_file_aio_write+0x51/0x93 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554202] [] ? do_sync_write+0xc0/0x107 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554205] [] ? cp_new_stat64+0xf7/0x109 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554208] [] ? autoremove_wake_function+0x0/0x2d 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554211] [] ? sys_fstat64+0x1e/0x23 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554214] [] ? security_file_permission+0xc/0xd 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554216] [] ? do_sync_write+0x0/0x107 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554217] [] ? vfs_write+0x7e/0xd6 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554219] [] ? sys_write+0x3c/0x63 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554222] [] ? sysenter_do_call+0x12/0x28 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554223] ---[ end trace 20f4b55a0560b754 ]--- FWIW, I have dbg symbols installed for this kernel: ii linux-image-2.6.32-5-686-bigmem 2.6.32-30 Linux 2.6.32 for PCs with 4GB+ RAM ii linux-image-2.6.32-5-686-bigmem-dbg 2.6.32-31 Debugging infos for Linux 2.6.32-5-686-bigmem Any pointers highly appreciated. -- Huella de clave primaria: 0FDA C36F F110 54F4 D42B D0EB 617D 396C 448B 31EB signature.asc Description: Digital signature
