[
https://issues.apache.org/jira/browse/HDFS-12337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138668#comment-16138668
]
Anu Engineer commented on HDFS-12337:
-------------------------------------
+1, I will commit this shortly.
> Ozone: Concurrent RocksDB open calls fail because of "No locks available"
> -------------------------------------------------------------------------
>
> Key: HDFS-12337
> URL: https://issues.apache.org/jira/browse/HDFS-12337
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Mukul Kumar Singh
> Assignee: Mukul Kumar Singh
> Fix For: HDFS-7240
>
> Attachments: HDFS-12337-HDFS-7240.001.patch
>
>
> HDFS-12216 fixes the issue with the static container cache and re-using the
> same container port on datanode restart. However TestKeys still fails after
> HDFS-12216 is fixed.
> The test is now failing because Concurrent RocksDB open calls fail. In the
> current code BlockDeleting service and Dispatcher tries to open the db
> concurrently.
> This jira will also fix the keepPort property for Ratis Container port and
> set the reuse address property for XceiverServerRatis correctly.
> {code}
> 2017-08-22 16:51:34,453 [BlockDeletingService#1] INFO utils.RocksDBStore
> (RocksDBStore.java:<init>(64)) - opening db file
> /Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/dn0_data0/containers/621a3b15-b9fc-4d49-a6e6-29d4c40cc91f/metadata/container.db
> 2017-08-22 16:51:34,460 [nioEventLoopGroup-9-1] INFO logging.LoggingHandler
> (Slf4JLogger.java:info(101)) - [id: 0x8822cd3d, /0.0.0.0:57044] RECEIVED:
> [id: 0x61367e6f, /127.0.0.1:57173 => /127.0.0
> .1:57044]
> 2017-08-22 16:51:34,461 [nioEventLoopGroup-10-1] INFO utils.RocksDBStore
> (RocksDBStore.java:<init>(64)) - opening db file
> /Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/dn0_data0/containers/621a3b15-b9fc-4d49-a6e6-29d4c40cc91f/metadata/container.db
> 2017-08-22 16:51:34,465 [nioEventLoopGroup-10-1] INFO utils.RocksDBStore
> (RocksDBStore.java:<init>(67)) - Failed init RocksDB, db path :
> /Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/dn0_data0/containers/621a3b15-b9fc-4d49-a6e6-29d4c40cc91f/metadata/container.dbexception
> org.rocksdb.RocksDBException: lock
> /Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/dn0_data0/containers/621a3b15-b9fc-4d49-a6e6-29d4c40cc91f/metadata/container.db/LOCK:
> No locks available
> 2017-08-22 16:51:34,465 [BlockDeletingService#1] INFO
> background.BlockDeletingService (BlockDeletingService.java:call(203)) -
> The elapsed time of task@70a576ee for deleting blocks: 12ms.
> 2017-08-22 16:51:34,474 [nioEventLoopGroup-10-1] INFO impl.Dispatcher
> (ContainerUtils.java:logAndReturnError(129)) - Operation: GetKey : Trace
> ID: 73f19131-f63b-459a-8f09-9a3db893a296 : Message:
> 621a3b15-b9fc-4d49-a6e6-29d4c40cc91f : Result: UNABLE_TO_READ_METADATA_DB
> 2017-08-22 16:51:34,475 [Thread-382] INFO exceptions.OzoneExceptionMapper
> (OzoneExceptionMapper.java:toResponse(39)) ozone
> c2a23759-c76f-49ea-b574-f0802a4e5b75/c0df3a48-f75b-4b5e-b1bd-c189ce698056/13b3d486-3d7a-49e4-bc9d-1ef63e674548
> hdfs 73f19131-f63b-459a-8f09-9a3db893a296 - Returning exception. ex:
> {"httpCode":500,"shortMessage":"internalServerError","resource":"hdfs","message":"621a3b15-b9fc-4d49-a6e6-29d4c40cc91f","requestID":"73f19131-f63b-459a-8f09-9a3db893a296","hostName":"hw13605.local"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
