Re: [BUG] Hang (deadlock?) inside iscsi_if_rx
On 01/24/2012 08:30 AM, Sasha Levin wrote:
Hi all,
I got the following hang when running trinity under KVM tool:
[ 2520.824250] INFO: task trinity:31986 blocked for more than 120 seconds.
[ 2520.825513] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables
this message.
[ 2520.826983] trinity D 5504 31986 2767
0x0004
[ 2520.828416] 88000a21ba98 0086 88000a3f8000
88000a3f8000
[ 2520.829892] 001d4340 88000a21bfd8 88000a21a000
001d4340
[ 2520.831380] 001d4340 001d4340 88000a21bfd8
001d4340
[ 2520.832838] Call Trace:
[ 2520.837089] [8265297a] schedule+0x3a/0x50
[ 2520.837983] [82651049] __mutex_lock_common+0x209/0x5b0
[ 2520.839181] [81af64f3] ? iscsi_if_rx+0x23/0xa00
[ 2520.840232] [81053943] ? sched_clock+0x13/0x20
[ 2520.841242] [81af64f3] ? iscsi_if_rx+0x23/0xa00
[ 2520.842271] [82651430] mutex_lock_nested+0x40/0x50
[ 2520.843353] [81af64f3] iscsi_if_rx+0x23/0xa00
[ 2520.844461] [810dfffd] ? sub_preempt_count+0x9d/0xd0
[ 2520.845576] [82654130] ? _raw_read_unlock+0x30/0x60
[ 2520.846705] [8221632e] netlink_unicast+0x1ae/0x1f0
[ 2520.847790] [82216ae7] netlink_sendmsg+0x227/0x350
[ 2520.848884] [821a490d] ? sock_update_netprioidx+0xdd/0x1b0
[ 2520.850105] [821a4882] ? sock_update_netprioidx+0x52/0x1b0
[ 2520.851282] [8219e376] sock_aio_write+0x166/0x180
[ 2520.852317] [810dfe51] ? get_parent_ip+0x11/0x50
[ 2520.853342] [811d0b1a] do_sync_write+0xda/0x120
[ 2520.854347] [81109c42] ? lock_release_holdtime+0xb2/0x160
[ 2520.855533] [810dfe51] ? get_parent_ip+0x11/0x50
[ 2520.856569] [817914e7] ? security_file_permission+0x27/0xb0
[ 2520.857756] [811d0ccc] vfs_write+0x16c/0x180
[ 2520.858713] [811d0dcf] sys_write+0x4f/0xa0
[ 2520.859680] [826553f9] system_call_fastpath+0x16/0x1b
[ 2520.873039] 1 lock held by trinity/31986:
[ 2520.873801] #0: (rx_queue_mutex){+.+...}, at: [81af64f3]
iscsi_if_rx+0x23/0xa00
[ 2520.875587] Kernel panic - not syncing: hung_task: blocked tasks
From looking at the logs, there were two instances of the fuzzer blocked on a
write() to a netlink socket. There are no iSCSI targets anywhere on the
network.
What does the trinity test do? Does it write random junk to netlink
sockets to test the kernel handlers or were you actually stressing the
creation of targets. If the latter what driver were you trying to create
targets with (iscsi_tcp, bnx2i, etc)?
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To post to this group, send email to open-iscsi@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
Re: [BUG] Hang (deadlock?) inside iscsi_if_rx
On 01/24/2012 10:29 AM, Mike Christie wrote:
On 01/24/2012 08:30 AM, Sasha Levin wrote:
Hi all,
I got the following hang when running trinity under KVM tool:
[ 2520.824250] INFO: task trinity:31986 blocked for more than 120 seconds.
[ 2520.825513] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables
this message.
[ 2520.826983] trinity D 5504 31986 2767
0x0004
[ 2520.828416] 88000a21ba98 0086 88000a3f8000
88000a3f8000
[ 2520.829892] 001d4340 88000a21bfd8 88000a21a000
001d4340
[ 2520.831380] 001d4340 001d4340 88000a21bfd8
001d4340
[ 2520.832838] Call Trace:
[ 2520.837089] [8265297a] schedule+0x3a/0x50
[ 2520.837983] [82651049] __mutex_lock_common+0x209/0x5b0
[ 2520.839181] [81af64f3] ? iscsi_if_rx+0x23/0xa00
[ 2520.840232] [81053943] ? sched_clock+0x13/0x20
[ 2520.841242] [81af64f3] ? iscsi_if_rx+0x23/0xa00
[ 2520.842271] [82651430] mutex_lock_nested+0x40/0x50
[ 2520.843353] [81af64f3] iscsi_if_rx+0x23/0xa00
[ 2520.844461] [810dfffd] ? sub_preempt_count+0x9d/0xd0
[ 2520.845576] [82654130] ? _raw_read_unlock+0x30/0x60
[ 2520.846705] [8221632e] netlink_unicast+0x1ae/0x1f0
[ 2520.847790] [82216ae7] netlink_sendmsg+0x227/0x350
[ 2520.848884] [821a490d] ? sock_update_netprioidx+0xdd/0x1b0
[ 2520.850105] [821a4882] ? sock_update_netprioidx+0x52/0x1b0
[ 2520.851282] [8219e376] sock_aio_write+0x166/0x180
[ 2520.852317] [810dfe51] ? get_parent_ip+0x11/0x50
[ 2520.853342] [811d0b1a] do_sync_write+0xda/0x120
[ 2520.854347] [81109c42] ? lock_release_holdtime+0xb2/0x160
[ 2520.855533] [810dfe51] ? get_parent_ip+0x11/0x50
[ 2520.856569] [817914e7] ? security_file_permission+0x27/0xb0
[ 2520.857756] [811d0ccc] vfs_write+0x16c/0x180
[ 2520.858713] [811d0dcf] sys_write+0x4f/0xa0
[ 2520.859680] [826553f9] system_call_fastpath+0x16/0x1b
[ 2520.873039] 1 lock held by trinity/31986:
[ 2520.873801] #0: (rx_queue_mutex){+.+...}, at: [81af64f3]
iscsi_if_rx+0x23/0xa00
[ 2520.875587] Kernel panic - not syncing: hung_task: blocked tasks
From looking at the logs, there were two instances of the fuzzer blocked on
a write() to a netlink socket. There are no iSCSI targets anywhere on the
network.
What does the trinity test do? Does it write random junk to netlink
sockets to test the kernel handlers or were you actually stressing the
creation of targets. If the latter what driver were you trying to create
targets with (iscsi_tcp, bnx2i, etc)?
Oh yeah, for the random type of test, how much data does userspace write?
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To post to this group, send email to open-iscsi@googlegroups.com.
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
