[
https://issues.apache.org/jira/browse/HDFS-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630935#comment-16630935
]
Ayush Saxena commented on HDFS-13945:
-------------------------------------
Thanx [~elgoiri] for the comment.
Regarding the overview : IIUC The test was basically aiming that if we fail a
volume in a datanode containing a replica of block the block would get into the
state of UnderReplicated and this we were checking whether it is
underReplicated or not in the assertion.The reason for it failing here if we
analyze by the logs was the since this file creation was above the assertion
check.By the time this file is created and we wait for it to get its 3
replicas.In that time this under replicated gets replicated and this is the
catch.Now when we check the assertion it says no underReplicated blocks since
that block got replicated.So to counter this I placed this creation of file
below this check to eliminate the time to get the block replicated.
* InstanceStorageDir() works perfect in most cases but I encountered a
scenario in my local not here in jenkins that that the block was not in storage
dir that we failed. So to make sure that the volume that we fail is for sure
the volume that has the replica.I moved to the alternative used here.
* File Creation was done earlier too.One Context I thought was it was to make
the disk report error and get the block reports and identify the block that we
are aiming as underReplicated.For sure it can be removed.I didn't remove it
completely just because of one doubt that isn't there any other purpose being
checked by this file creation like whether we are able add file after volume
failure or not.Though it seems to be unrelated to the scenario which we are
checking . If agreed that it has now in the present situation no relation with
the test we can remove it. :)
> TestDataNodeVolumeFailure is Flaky
> ----------------------------------
>
> Key: HDFS-13945
> URL: https://issues.apache.org/jira/browse/HDFS-13945
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Major
> Attachments: HDFS-13945-01.patch
>
>
> The test is failing in trunk since long.
> Reference -
> [https://builds.apache.org/job/PreCommit-HDFS-Build/25140/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestDataNodeVolumeFailure/testUnderReplicationAfterVolFailure/]
>
> [https://builds.apache.org/job/PreCommit-HDFS-Build/25135/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestDataNodeVolumeFailure/testUnderReplicationAfterVolFailure/]
>
> [https://builds.apache.org/job/PreCommit-HDFS-Build/25133/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestDataNodeVolumeFailure/testUnderReplicationAfterVolFailure/]
>
> [https://builds.apache.org/job/PreCommit-HDFS-Build/25104/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestDataNodeVolumeFailure/testUnderReplicationAfterVolFailure/]
>
>
> Stack Trace -
>
> Timed out waiting for condition. Thread diagnostics: Timestamp: 2018-09-26
> 03:32:07,162 "IPC Server handler 2 on 33471" daemon prio=5 tid=2931
> timed_waiting java.lang.Thread.State: TIMED_WAITING at
> sun.misc.Unsafe.park(Native Method) at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:288) at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2668) "IPC Server
> handler 3 on 34285" daemon prio=5 tid=2646 timed_waiting
> java.lang.Thread.State: TIMED_WAITING at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:288) at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2668)
> "org.apache.hadoop.util.JvmPauseMonitor$Monitor@1d2ee4cd" daemon prio=5
> tid=2633 timed_waiting java.lang.Thread.State: TIMED_WAITING at
> java.lang.Thread.sleep(Native Method) at
> org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192)
> at java.lang.Thread.run(Thread.java:748) "IPC Server Responder" daemon prio=5
> tid=2766 runnable java.lang.Thread.State: RUNNABLE at
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at
> org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1334) at
> org.apache.hadoop.ipc.Server$Responder.run(Server.java:1317)
> "org.eclipse.jetty.server.session.HashSessionManager@1287fc65Timer" daemon
> prio=5 tid=2492 timed_waiting java.lang.Thread.State: TIMED_WAITING at
> sun.misc.Unsafe.park(Native Method) at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) "qtp548667392-2533" daemon prio=5
> tid=2533 timed_waiting java.lang.Thread.State: TIMED_WAITING at
> sun.misc.Unsafe.park(Native Method) at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
> at java.lang.Thread.run(Thread.java:748)
> "org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor@63818b03"
> daemon prio=5 tid=2521 timed_waiting java.lang.Thread.State: TIMED_WAITING
> at java.lang.Thread.sleep(Native Method) at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor.run(FSNamesystem.java:4045)
> at java.lang.Thread.run(Thread.java:748)
> "BP-1973654218-172.17.0.2-1537975830395 heartbeating to
> localhost/127.0.0.1:43522" daemon prio=5 tid=2640 timed_waiting
> java.lang.Thread.State: TIMED_WAITING at java.lang.Object.wait(Native Method)
> at
> org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158)
> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:717)
> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:844)
> at java.lang.Thread.run(Thread.java:748)
> "AsyncAppender-Dispatcher-Thread-38" daemon prio=5 tid=70 in Object.wait()
> java.lang.Thread.State: WAITING (on object monitor) at
> java.lang.Object.wait(Native Method) at
> java.lang.Object.wait(Object.java:502) at
> org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:548) at
> java.lang.Thread.run(Thread.java:748) "IPC Server listener on 43522" daemon
> prio=5 tid=2499 runnable java.lang.Thread.State: RUNNABLE at
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at
> org.apache.hadoop.ipc.Server$Listener.run(Server.java:1159) "IPC Server
> handler 8 on 34285" daemon prio=5 tid=2651 timed_waiting
> java.lang.Thread.State: TIMED_WAITING at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:288) at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2668)
> "nioEventLoopGroup-30-1" prio=10 tid=2824 runnable java.lang.Thread.State:
> RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at
> io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
> at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:754) at
> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:410) at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> at java.lang.Thread.run(Thread.java:748) "IPC Server handler 7 on 43522"
> daemon prio=5 tid=2515 timed_waiting java.lang.Thread.State: TIMED_WAITING at
> sun.misc.Unsafe.park(Native Method) at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:288) at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2668)
> "qtp1255640117-2483" daemon prio=5 tid=2483 runnable java.lang.Thread.State:
> RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
> at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
> at java.lang.Thread.run(Thread.java:748)
> "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@2b2f4b0a" daemon
> prio=5 tid=2815 runnable java.lang.Thread.State: RUNNABLE at
> sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
> at sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:100) at
> org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) at
> org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145)
> at java.lang.Thread.run(Thread.java:748) "IPC Server handler 8 on 37367"
> daemon prio=5 tid=2779 timed_waiting
>
> Timed out waiting for condition. Thread diagnostics: Timestamp: 2018-09-26
> 03:32:07,162 "IPC Server handler 2 on 33471" daemon prio=5 tid=2931
> timed_waiting java.lang.Thread.State: TIMED_WAITING at
> sun.misc.Unsafe.park(Native Method) at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:288) at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2668) "IPC Server
> handler 3 on 34285" daemon prio=5 tid=2646 timed_waiting
> java.lang.Thread.State: TIMED_WAITING at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:288) at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2668)
> "org.apache.hadoop.util.JvmPauseMonitor$Monitor@1d2ee4cd" daemon prio=5
> tid=2633 timed_waiting java.lang.Thread.State: TIMED_WAITING at
> java.lang.Thread.sleep(Native Method) at
> org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192)
> at java.lang.Thread.run(Thread.java:748) "IPC Server Responder" daemon prio=5
> tid=2766 runnable java.lang.Thread.State: RUNNABLE at
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at
> org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1334) at
> org.apache.hadoop.ipc.Server$Responder.run(Server.java:1317)
> "org.eclipse.jetty.server.session.HashSessionManager@1287fc65Timer" daemon
> prio=5 tid=2492 timed_waiting java.lang.Thread.State: TIMED_WAITING at
> sun.misc.Unsafe.park(Native Method) at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) "qtp548667392-2533" daemon prio=5
> tid=2533 timed_waiting java.lang.Thread.State: TIMED_WAITING at
> sun.misc.Unsafe.park(Native Method) at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
> at java.lang.Thread.run(Thread.java:748)
> "org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor@63818b03"
> daemon prio=5 tid=2521 timed_waiting java.lang.Thread.State: TIMED_WAITING
> at java.lang.Thread.sleep(Native Method) at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor.run(FSNamesystem.java:4045)
> at java.lang.Thread.run(Thread.java:748)
> "BP-1973654218-172.17.0.2-1537975830395 heartbeating to
> localhost/127.0.0.1:43522" daemon prio=5 tid=2640 timed_waiting
> java.lang.Thread.State: TIMED_WAITING at java.lang.Object.wait(Native Method)
> at
> org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158)
> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:717)
> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:844)
> at java.lang.Thread.run(Thread.java:748)
> "AsyncAppender-Dispatcher-Thread-38" daemon prio=5 tid=70 in Object.wait()
> java.lang.Thread.State: WAITING (on object monitor) at
> java.lang.Object.wait(Native Method) at
> java.lang.Object.wait(Object.java:502) at
> org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:548) at
> java.lang.Thread.run(Thread.java:748) "IPC Server listener on 43522" daemon
> prio=5 tid=2499 runnable java.lang.Thread.State: RUNNABLE at
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at
> org.apache.hadoop.ipc.Server$Listener.run(Server.java:1159) "IPC Server
> handler 8 on 34285" daemon prio=5 tid=2651 timed_waiting
> java.lang.Thread.State: TIMED_WAITING at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:288) at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2668)
> "nioEventLoopGroup-30-1" prio=10 tid=2824 runnable java.lang.Thread.State:
> RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at
> io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
> at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:754) at
> io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:410) at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> at java.lang.Thread.run(Thread.java:748) "IPC Server handler 7 on 43522"
> daemon prio=5 tid=2515 timed_waiting java.lang.Thread.State: TIMED_WAITING at
> sun.misc.Unsafe.park(Native Method) at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:288) at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2668)
> "qtp1255640117-2483" daemon prio=5 tid=2483 runnable java.lang.Thread.State:
> RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
> at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
> at java.lang.Thread.run(Thread.java:748)
> "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@2b2f4b0a" daemon
> prio=5 tid=2815 runnable java.lang.Thread.State: RUNNABLE at
> sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
> at sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:100) at
> org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) at
> org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145)
> at java.lang.Thread.run(Thread.java:748) "IPC Server handler 8 on 37367"
> daemon prio=5 tid=2779 timed_waiting
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
