[
https://issues.apache.org/jira/browse/HDFS-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136610#comment-16136610
]
Mukul Kumar Singh commented on HDFS-12216:
------------------------------------------
Looked into the recent failures after the patch and it seems that the request
to get the key is being sent even before XceiverServer on datanode is up.
{code}
HW13605:ozone_review msingh$ cat
/Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports/org.apache.hadoop.ozone.web.client.TestKeys-output.txt
| egrep "51619|Exception getting XceiverClient"
2017-08-22 15:29:53,204 [BP-1861535517-10.200.5.245-1503395989416 heartbeating
to localhost/127.0.0.1:51613] INFO server.XceiverServer
(XceiverServer.java:<init>(75)) - Found a free port for the server : 51619
2017-08-22 15:29:53,966 [nioEventLoopGroup-4-1] INFO logging.LoggingHandler
(Slf4JLogger.java:info(101)) - [id: 0xd118274a] BIND(0.0.0.0/0.0.0.0:51619)
2017-08-22 15:29:53,966 [nioEventLoopGroup-4-1] INFO logging.LoggingHandler
(Slf4JLogger.java:info(101)) - [id: 0xd118274a, /0.0.0.0:51619] ACTIVE
2017-08-22 15:29:58,240 [nioEventLoopGroup-4-1] INFO logging.LoggingHandler
(Slf4JLogger.java:info(101)) - [id: 0xd118274a, /0.0.0.0:51619] RECEIVED: [id:
0xccf8c3db, /127.0.0.1:51633 => /127.0.0.1:51619]
2017-08-22 15:29:59,553 [nioEventLoopGroup-4-1] INFO logging.LoggingHandler
(Slf4JLogger.java:info(101)) - [id: 0xd118274a, /0.0.0.0:51619] UNREGISTERED
2017-08-22 15:30:00,042 [Thread-378] INFO exceptions.OzoneExceptionMapper
(OzoneExceptionMapper.java:toResponse(39)) ozone
ea73188d-e1f0-43a3-8d0e-4a6b13ffba95/af614819-4b18-4f49-96d8-8e8117ff7d98/8a1f6102-bf6c-4b0d-a124-0803c9950b2b
hdfs b3ad0f07-3daa-406b-bf28-438efbd772f6 - Returning exception. ex:
{"httpCode":500,"shortMessage":"internalServerError","resource":"hdfs","message":"Exception
getting
XceiverClient.","requestID":"b3ad0f07-3daa-406b-bf28-438efbd772f6","hostName":"hw13605.local"}
2017-08-22 15:30:00,047 [nioEventLoopGroup-10-1] INFO logging.LoggingHandler
(Slf4JLogger.java:info(101)) - [id: 0xbb9534b5] BIND(0.0.0.0/0.0.0.0:51619)
2017-08-22 15:30:00,047 [nioEventLoopGroup-10-1] INFO logging.LoggingHandler
(Slf4JLogger.java:info(101)) - [id: 0xbb9534b5, /0.0.0.0:51619] ACTIVE
2017-08-22 15:30:00,071 [nioEventLoopGroup-10-1] INFO logging.LoggingHandler
(Slf4JLogger.java:info(101)) - [id: 0xbb9534b5, /0.0.0.0:51619] UNREGISTERED
{code}
> Ozone: TestKeys is failing consistently
> ---------------------------------------
>
> Key: HDFS-12216
> URL: https://issues.apache.org/jira/browse/HDFS-12216
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Mukul Kumar Singh
> Assignee: Mukul Kumar Singh
> Fix For: HDFS-7240
>
> Attachments: HDFS-12216-HDFS-7240.001.patch,
> HDFS-12216-HDFS-7240.002.patch, HDFS-12216-HDFS-7240.003.patch,
> HDFS-12216-HDFS-7240.004.patch, HDFS-12216-HDFS-7240.005.patch,
> HDFS-12216-HDFS-7240.006.patch
>
>
> TestKeys and TestKeysRatis are failing consistently as noted in test logs for
> HDFS-12183
> TestKeysRatis is failing because of the following error
> {code}
> 2017-07-28 23:11:28,783 [StateMachineUpdater-127.0.0.1:55793] ERROR
> impl.StateMachineUpdater (ExitUtils.java:terminate(80)) - Terminating with
> exit status 2: StateMachineUpdater-127.0.0.1:55793: the StateMachineUpdater
> hits Throwable
> org.iq80.leveldb.DBException: Closed
> at org.fusesource.leveldbjni.internal.JniDB.put(JniDB.java:123)
> at org.apache.hadoop.utils.LevelDBStore.put(LevelDBStore.java:98)
> at
> org.apache.hadoop.ozone.container.common.impl.KeyManagerImpl.putKey(KeyManagerImpl.java:90)
> at
> org.apache.hadoop.ozone.container.common.impl.Dispatcher.handlePutKey(Dispatcher.java:547)
> at
> org.apache.hadoop.ozone.container.common.impl.Dispatcher.keyProcessHandler(Dispatcher.java:206)
> at
> org.apache.hadoop.ozone.container.common.impl.Dispatcher.dispatch(Dispatcher.java:110)
> at
> org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.dispatch(ContainerStateMachine.java:94)
> at
> org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.applyTransaction(ContainerStateMachine.java:81)
> at
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:913)
> at
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:142)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> where as TestKeys is failing because of
> {code}
> 2017-07-28 23:14:20,889 [Thread-486] INFO scm.XceiverClientManager
> (XceiverClientManager.java:getClient(158)) - exception
> java.util.concurrent.ExecutionException: java.net.ConnectException:
> Connection refused: /127.0.0.1:55914
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
