[
https://issues.apache.org/jira/browse/HDFS-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531873#comment-16531873
]
Wei-Chiu Chuang commented on HDFS-13614:
----------------------------------------
The impalad had dozens threads blocked on SelectorPool monitor:
{noformat}
"Thread-588336" #591562 prio=5 os_prio=0 tid=0x00007f904b9ad000 nid=0x1b2fb6
waiting for monitor entry [0x00007f8fb1f6b000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.release(SocketIOWithTimeout.java:428)
- waiting to lock <0x0000000080697830> (a
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
at
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:373)
at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.readChannelFully(PacketReceiver.java:258)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:209)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:171)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:102)
at
org.apache.hadoop.hdfs.RemoteBlockReader2.readNextPacket(RemoteBlockReader2.java:207)
at
org.apache.hadoop.hdfs.RemoteBlockReader2.read(RemoteBlockReader2.java:156)
- locked <0x0000000589375ce0> (a
org.apache.hadoop.hdfs.RemoteBlockReader2)
at
org.apache.hadoop.hdfs.DFSInputStream$ByteArrayStrategy.doRead(DFSInputStream.java:788)
at
org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:844)
- locked <0x000000058931c560> (a org.apache.hadoop.hdfs.DFSInputStream)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:904)
- locked <0x000000058931c560> (a org.apache.hadoop.hdfs.DFSInputStream)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:954)
- locked <0x000000058931c560> (a org.apache.hadoop.hdfs.DFSInputStream)
at java.io.DataInputStream.read(DataInputStream.java:149)
{noformat}
It seems the after the NPE was thrown, the object's monitor didn't get released
somehow.
> DN failed to connect with NN because of NPE in SocketIOWithTimeout
> ------------------------------------------------------------------
>
> Key: HDFS-13614
> URL: https://issues.apache.org/jira/browse/HDFS-13614
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.7.2
> Reporter: Surendra Singh Lilhore
> Priority: Major
>
> {{LinkedList$ListItr.next()}} is throwing NPE in {{SocketIOWithTimeout}}.
> Because of this socket connections are failing. It may be java bug also..
> [https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8133715]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
