[
https://issues.apache.org/jira/browse/HDFS-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458184#comment-17458184
]
JiangHua Zhu commented on HDFS-16228:
-------------------------------------
Sorry, I am late.
Thank you [~shv] for your comment.
I try to add some patches related to FGL, and use NNThroughputBenchmark to
verify mkdirs or create functions, for example:
./bin/hadoop org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark -fs
hdfs://xxxx -op mkdirs -threads 50 -dirs 2000000
Executing this command many times, sometimes there are errors.
When I stop updating the data and compare the metadata size displayed by Active
NmaeNode and Standby NameNode, we will find that there is an error at this time.
Active NN
!screenshot-1.png!
Standby NN
!screenshot-2.png!
There are some data sizes related to INodeMap#Size, such as filesTotal.
I tried to check PartitionedGSet, because at the same time, multiple
PartitionEntry operations are allowed, and there will be situations where the
size operation is uncertain.
When I changed the type of size to AtomicInteger, this didn't happen.
> [FGL]Improve safer PartitionedGSet#size
> ---------------------------------------
>
> Key: HDFS-16228
> URL: https://issues.apache.org/jira/browse/HDFS-16228
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs, namenode
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Labels: pull-request-available
> Attachments: screenshot-1.png, screenshot-2.png
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> When multiple PartitionedEntry is working at the same time, there may be
> inconsistencies in the operation PartitionedGSet#size.
> For example, there are some size++ or size-- operations in PartitionedGSet.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
