[
https://issues.apache.org/jira/browse/HDFS-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13867699#comment-13867699
]
Steve Loughran commented on HDFS-5755:
--------------------------------------
Scenario
# client app tries to talk to a namenode
# namenode service (entire VM is not running)
# RPC client starts spinng, sleeping, retrying
# control-C issued by hand to exit program
# system halting messages printed
# RPC client keeps spinning
# control-C issued again -no obvious effect
# kill -QUIT issued to show trace
# eventually, when the retry count is completed, the process exits
{code}
2014-01-10 10:34:49.549 java[2540:1903] Unable to load realm info from
SCDynamicStore
14/01/10 10:34:49 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/01/10 10:34:50 WARN hdfs.BlockReaderLocal: The short-circuit local reads
feature cannot be used because libhadoop cannot be loaded.
14/01/10 10:34:50 INFO client.RMProxy: Connecting to ResourceManager at
sandbox.hortonworks.com/192.168.1.134:8050
14/01/10 10:35:10 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 0 time(s);
maxRetries=45
14/01/10 10:35:11 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 0 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:12 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 1 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:13 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 2 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:14 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 3 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:15 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 4 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:35 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 1 time(s);
maxRetries=45
14/01/10 10:35:36 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 5 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:37 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 6 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:38 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 7 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:39 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 8 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:40 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 9 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:35:41 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 10 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:01 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 2 time(s);
maxRetries=45
14/01/10 10:36:02 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 11 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:03 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 12 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:04 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 13 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:05 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 14 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:06 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 15 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:07 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 16 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:08 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 17 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:09 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 18 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:10 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 19 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:11 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 20 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:12 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 21 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:13 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 22 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:33 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 3 time(s);
maxRetries=45
14/01/10 10:36:34 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 23 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:35 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 24 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:36 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 25 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:37 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 26 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:38 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 27 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:36:58 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 4 time(s);
maxRetries=45
14/01/10 10:36:59 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 28 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:00 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 29 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:01 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 30 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:02 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 31 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:03 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 32 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
^C14/01/10 10:37:06 INFO launcher.ServiceLauncher: Halting service
14/01/10 10:37:06 INFO util.ExitUtil: Exiting with status 0
^C14/01/10 10:37:07 INFO launcher.ServiceLauncher: Halting service
14/01/10 10:37:07 INFO util.ExitUtil: Exiting with status 0
14/01/10 10:37:23 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 5 time(s);
maxRetries=45
14/01/10 10:37:24 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 33 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:25 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 34 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:26 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 35 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:27 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 36 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:28 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 37 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
2014-01-10 10:37:47
Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.45-b08 mixed mode):
"SIGINT handler" daemon prio=5 tid=0x00007feed2404000 nid=0x3b13 waiting for
monitor entry [0x00000001189df000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.Shutdown.exit(Shutdown.java:212)
- waiting to lock <0x00000007fed4a298> (a java.lang.Class for
java.lang.Shutdown)
at java.lang.Runtime.exit(Runtime.java:109)
at java.lang.System.exit(System.java:962)
at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:133)
at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:192)
at
org.apache.hadoop.yarn.service.launcher.ServiceLauncher.exit(ServiceLauncher.java:270)
at
org.apache.hadoop.yarn.service.launcher.ServiceLauncher.interrupted(ServiceLauncher.java:257)
at
org.apache.hadoop.yarn.service.launcher.IrqHandler.handle(IrqHandler.java:70)
at sun.misc.Signal$1.run(Signal.java:212)
at java.lang.Thread.run(Thread.java:744)
"Thread-0" prio=5 tid=0x00007feed32ed800 nid=0x5707 waiting on condition
[0x000000011a45d000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.ipc.Client.stop(Client.java:1173)
at org.apache.hadoop.ipc.ClientCache.stopClient(ClientCache.java:100)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.close(ProtobufRpcEngine.java:251)
at org.apache.hadoop.ipc.RPC.stopProxy(RPC.java:626)
at
org.apache.hadoop.io.retry.DefaultFailoverProxyProvider.close(DefaultFailoverProxyProvider.java:57)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.close(RetryInvocationHandler.java:206)
at org.apache.hadoop.ipc.RPC.stopProxy(RPC.java:626)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.close(ClientNamenodeProtocolTranslatorPB.java:174)
at org.apache.hadoop.ipc.RPC.stopProxy(RPC.java:621)
at
org.apache.hadoop.hdfs.DFSClient.closeConnectionToNamenode(DFSClient.java:738)
at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:794)
- locked <0x00000007fec26860> (a org.apache.hadoop.hdfs.DFSClient)
at
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:847)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2524)
- locked <0x00000007fec254e0> (a org.apache.hadoop.fs.FileSystem$Cache)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2541)
- locked <0x00000007fec254f8> (a
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
"SIGINT handler" daemon prio=5 tid=0x00007feed2403800 nid=0x3d13 in
Object.wait() [0x0000000117a14000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000007fed58110> (a
org.apache.hadoop.util.ShutdownHookManager$1)
at java.lang.Thread.join(Thread.java:1280)
- locked <0x00000007fed58110> (a
org.apache.hadoop.util.ShutdownHookManager$1)
at java.lang.Thread.join(Thread.java:1354)
at
java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:106)
at
java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
at java.lang.Shutdown.runHooks(Shutdown.java:123)
at java.lang.Shutdown.sequence(Shutdown.java:167)
at java.lang.Shutdown.exit(Shutdown.java:212)
- locked <0x00000007fed4a298> (a java.lang.Class for java.lang.Shutdown)
at java.lang.Runtime.exit(Runtime.java:109)
at java.lang.System.exit(System.java:962)
at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:133)
at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:192)
at
org.apache.hadoop.yarn.service.launcher.ServiceLauncher.exit(ServiceLauncher.java:270)
at
org.apache.hadoop.yarn.service.launcher.ServiceLauncher.interrupted(ServiceLauncher.java:257)
at
org.apache.hadoop.yarn.service.launcher.IrqHandler.handle(IrqHandler.java:70)
at sun.misc.Signal$1.run(Signal.java:212)
at java.lang.Thread.run(Thread.java:744)
"Service Thread" daemon prio=5 tid=0x00007feed3807000 nid=0x5303 runnable
[0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" daemon prio=5 tid=0x00007feed2804800 nid=0x5103 waiting on
condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" daemon prio=5 tid=0x00007feed2801000 nid=0x4f03 waiting on
condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=5 tid=0x00007feed3017000 nid=0x4d03 waiting on
condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=5 tid=0x00007feed3802800 nid=0x3903 in Object.wait()
[0x00000001178cb000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000007feb006c0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
- locked <0x00000007feb006c0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
"Reference Handler" daemon prio=5 tid=0x00007feed3001000 nid=0x3703 in
Object.wait() [0x00000001177c8000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000007feb08d40> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0x00000007feb08d40> (a java.lang.ref.Reference$Lock)
"main" prio=5 tid=0x00007feed2019800 nid=0x1903 runnable [0x0000000111152000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x00000007fb6aeee8> (a sun.nio.ch.Util$2)
- locked <0x00000007fb6aee68> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000007fb6aece0> (a sun.nio.ch.KQueueSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:203)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493)
at
org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:547)
- locked <0x00000007fb5ceea8> (a
org.apache.hadoop.ipc.Client$Connection)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:642)
- locked <0x00000007fb5ceea8> (a
org.apache.hadoop.ipc.Client$Connection)
at org.apache.hadoop.ipc.Client$Connection.access$2600(Client.java:314)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1399)
at org.apache.hadoop.ipc.Client.call(Client.java:1318)
at org.apache.hadoop.ipc.Client.call(Client.java:1300)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy13.getApplications(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getApplications(ApplicationClientProtocolPBClientImpl.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy14.getApplications(Unknown Source)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplications(YarnClientImpl.java:237)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplications(YarnClientImpl.java:221)
at
org.apache.hadoop.hoya.yarn.client.HoyaYarnClientImpl.listHoyaInstances(HoyaYarnClientImpl.java:75)
at
org.apache.hadoop.hoya.yarn.client.HoyaClient.listHoyaInstances(HoyaClient.java:1167)
at
org.apache.hadoop.hoya.yarn.client.HoyaClient.findInstance(HoyaClient.java:1291)
at
org.apache.hadoop.hoya.yarn.client.HoyaClient.findInstance(HoyaClient.java:1276)
at
org.apache.hadoop.hoya.yarn.client.HoyaClient.bondToCluster(HoyaClient.java:1770)
at
org.apache.hadoop.hoya.yarn.client.HoyaClient.createClusterOperations(HoyaClient.java:1787)
at
org.apache.hadoop.hoya.yarn.client.HoyaClient.getClusterDescription(HoyaClient.java:1681)
at
org.apache.hadoop.hoya.yarn.client.HoyaClient.actionStatus(HoyaClient.java:1358)
at
org.apache.hadoop.hoya.yarn.client.HoyaClient.runService(HoyaClient.java:237)
at
org.apache.hadoop.yarn.service.launcher.ServiceLauncher.launchService(ServiceLauncher.java:176)
at
org.apache.hadoop.yarn.service.launcher.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:385)
at
org.apache.hadoop.yarn.service.launcher.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:314)
at
org.apache.hadoop.yarn.service.launcher.ServiceLauncher.serviceMain(ServiceLauncher.java:530)
at org.apache.hadoop.hoya.Hoya.main(Hoya.java:49)
"VM Thread" prio=5 tid=0x00007feed2066800 nid=0x3503 runnable
"GC task thread#0 (ParallelGC)" prio=5 tid=0x00007feed2014000 nid=0x2503
runnable
"GC task thread#1 (ParallelGC)" prio=5 tid=0x00007feed2027000 nid=0x2703
runnable
"GC task thread#2 (ParallelGC)" prio=5 tid=0x00007feed2027800 nid=0x2903
runnable
"GC task thread#3 (ParallelGC)" prio=5 tid=0x00007feed2028800 nid=0x2b03
runnable
"GC task thread#4 (ParallelGC)" prio=5 tid=0x00007feed2029000 nid=0x2d03
runnable
"GC task thread#5 (ParallelGC)" prio=5 tid=0x00007feed2029800 nid=0x2f03
runnable
"GC task thread#6 (ParallelGC)" prio=5 tid=0x00007feed202a000 nid=0x3103
runnable
"GC task thread#7 (ParallelGC)" prio=5 tid=0x00007feed202b000 nid=0x3303
runnable
"VM Periodic Task Thread" prio=5 tid=0x00007feed280d000 nid=0x5503 waiting on
condition
JNI global references: 161
Heap
PSYoungGen total 76800K, used 27515K [0x00000007faa80000,
0x0000000800000000, 0x0000000800000000)
eden space 66048K, 26% used
[0x00000007faa80000,0x00000007fbbd1420,0x00000007feb00000)
from space 10752K, 90% used
[0x00000007feb00000,0x00000007ff48dbb8,0x00000007ff580000)
to space 10752K, 0% used
[0x00000007ff580000,0x00000007ff580000,0x0000000800000000)
ParOldGen total 174592K, used 72K [0x00000007f0000000,
0x00000007faa80000, 0x00000007faa80000)
object space 174592K, 0% used
[0x00000007f0000000,0x00000007f0012010,0x00000007faa80000)
PSPermGen total 21504K, used 15641K [0x00000007eae00000,
0x00000007ec300000, 0x00000007f0000000)
object space 21504K, 72% used
[0x00000007eae00000,0x00000007ebd464b8,0x00000007ec300000)
14/01/10 10:37:48 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 6 time(s);
maxRetries=45
14/01/10 10:37:49 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 38 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:50 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 39 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:51 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 40 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:52 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 41 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:37:53 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 42 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:38:13 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 7 time(s);
maxRetries=45
14/01/10 10:38:14 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 43 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:38:15 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 44 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:38:16 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 45 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:38:17 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 46 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:38:18 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 47 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:38:38 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 8 time(s);
maxRetries=45
14/01/10 10:38:39 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 48 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
14/01/10 10:38:40 INFO ipc.Client: Retrying connect to server:
sandbox.hortonworks.com/192.168.1.134:8050. Already tried 49 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1
SECONDS)
(git:feature/BUG-12091:4fe977c △ ) hoya-0.10.0-SNAPSHOT $
{code}
> Filesystem close blocks SystemExit if a DFS client is trying to talk to a
> nonexistent NN
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-5755
> URL: https://issues.apache.org/jira/browse/HDFS-5755
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Steve Loughran
> Priority: Minor
>
> If a DFS client instance is spinning in a connection refused cycle, and you
> try to stop the process via System.exit(), the exit call does not complete
> until the client has actually given up.
> That is: you can't exit a process with a standard kill while a DFS client
> operation is blocking.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
