[
https://issues.apache.org/jira/browse/HDDS-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16849974#comment-16849974
]
Xiaoyu Yao commented on HDDS-1604:
----------------------------------
With the debug log added to ContainerCache, we can see the leak from
ContainerReader.initializeUsedBytes easily.
{code}
/Users/xyao/github/hadoop/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-b08b80e5-1b9c-436e-b236-35679090d88d/datanode-0/data/containers/hdds/4c414084-b9c3-4774-8c08-3fad261af8df/current/containerDir0/1/metadata/1-dn-container.db
to refCnt 2
java.lang.Exception
at
org.apache.hadoop.ozone.container.common.utils.ContainerCache$ReferenceCountedDB.incrementReference(ContainerCache.java:210)
at
org.apache.hadoop.ozone.container.common.utils.ContainerCache$ReferenceCountedDB.access$100(ContainerCache.java:193)
at
org.apache.hadoop.ozone.container.common.utils.ContainerCache.getDB(ContainerCache.java:158)
at
org.apache.hadoop.ozone.container.keyvalue.helpers.BlockUtils.getDB(BlockUtils.java:77)
at
org.apache.hadoop.ozone.container.keyvalue.KeyValueBlockIterator.<init>(KeyValueBlockIterator.java:97)
at
org.apache.hadoop.ozone.container.keyvalue.KeyValueBlockIterator.<init>(KeyValueBlockIterator.java:75)
at
org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.initializeUsedBytes(ContainerReader.java:242)
at
org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.verifyAndFixupContainerData(ContainerReader.java:220)
at
org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.verifyContainerFile(ContainerReader.java:168)
at
org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.readVolume(ContainerReader.java:146)
at
org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.run(ContainerReader.java:101)
at java.lang.Thread.run(Thread.java:748)
{code}
> ContainerReader#initializeUsedBytes leaks DB reference
> ------------------------------------------------------
>
> Key: HDDS-1604
> URL: https://issues.apache.org/jira/browse/HDDS-1604
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Xiaoyu Yao
> Assignee: Xiaoyu Yao
> Priority: Major
>
> This was caught by the New ContainerCache with reference counting from
> HDDS-1449. The root cause is an unclosed KeyValueBlockIterator from
> ContainerReader#initializeUsedBytes.
> I will post a patch shortly, which will fix some UT failures exposed by
> HDDS-1449.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
