[
https://issues.apache.org/jira/browse/HDFS-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16227704#comment-16227704
]
Nanda kumar commented on HDFS-12474:
------------------------------------
Thanks [~linyiqun] & [~xyao] for the review. Uploaded patch v001 addressing the
review comments.
>> line 230: Why not use {{float}} type value for the threshold-percentage
>> setting?
Changed.
>> line 375: {{(containerInfo.getUsed() / containerInfo.getSize())}} should
>> be...
Fixed.
>> line 369: ContainerInfo.lastUsed doesn't be set here.
We are setting lastUsed in {{ContainerInfo#Constructor}}, line 58
{{this.lastUsed = Time.monotonicNow()}}. We don't expose {{lastUsed}} through
constructor argument or setter, this is to avoid someone setting it to past
values. Updating of lastUsed is always done with {{Time.monotonicNow()}}.
>> Need to update the reference to BlockContainerInfo since the class is
>> removed with the patch. Otherwise, maven failed to build.
Thanks for the catch, I don't know how it got missed.
>> NIT: Line 27-33 unused imports
I don't see any unused import in ContainerMapping.java, please let me know if
I'm missing something.
>> if we always update the allocated based on the reported usage, should we
>> restrict the size to be smaller than The maximum container size (5gb)?
This is done in container report processing because in case of SCM crash the
allocated size will not be properly updated by {{ContainerStateManager}}
\[ContainerStateManager#close - will create a jira for this\], during next
start-up of SCM we don't want the allocated size to be very old (previous
successful shutdown of SCM). This update is done as a fallback mechanism in
case SCM crashes without properly updating allocated size, this is not a
correct allocated size.
In theory allocated size can exceed the container size, so I don't think we
should restrict the size here. For that matter even used size might exceed the
container size if we don't receive container reports in time (containers will
be closed while processing container reports).
>> document the new configuration key for
>> ozone.scm.container.close.threshold.percentage in ozone-default.xml?
Done.
> Ozone: SCM: Handling container report with key count and container usage.
> -------------------------------------------------------------------------
>
> Key: HDFS-12474
> URL: https://issues.apache.org/jira/browse/HDFS-12474
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: HDFS-7240
> Reporter: Xiaoyu Yao
> Assignee: Nanda kumar
> Labels: ozoneMerge
> Attachments: HDFS-12474-HDFS-7240.000.patch,
> HDFS-12474-HDFS-7240.001.patch
>
>
> Currently, the container report only contains the # of reports sent to SCM.
> We will need to provide the key count and the usage of each individual
> containers to update the SCM container state maintained by
> ContainerStateManager. This has a dependency on HDFS-12387.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
