[
https://issues.apache.org/jira/browse/HDFS-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16282358#comment-16282358
]
Chen Liang commented on HDFS-12751:
-----------------------------------
Thanks [~nandakumar131] for the followup!
bq. We do not update allocatedBytes here
But looks to me it does update allocated bytes, as in the follow code from
{{ContainerStateManager#updateContainerState}}. The {{info}} variable is the
new container info passed in from {{updateContainerState}}.
{code}
ContainerInfo containerInfo = new ContainerInfo.Builder()
.setContainerName(info.getContainerName())
.setState(newState)
.setPipeline(info.getPipeline())
.setAllocatedBytes(info.getAllocatedBytes())
.setUsedBytes(info.getUsedBytes())
.setNumberOfKeys(info.getNumberOfKeys())
.setStateEnterTime(Time.monotonicNow())
.setOwner(info.getOwner())
.build();
{code}
bq. Not exactly. Whenever we allocate block...
I did miss this part earlier, thanks for point out! But appears to me that this
is what BlockManager perceives as the bytes that might get allocated, not
necessarily the actual allocated bytes on the container. e.g. client may then
terminate before talking to container etc. While the allocatedBytes we got from
block report seems to be the more accurate number, precisely the number of
bytes the container sees when sending the report. And block report is the
trigger of the {{updateContainerState}} code path. Namely, I feel that
persisting the number we got from updateContainerState is already the better
thing.
Additionally, I'm under the impression that {{ContainerMapping}} is the class
that interacts the container store, while {{ContainerStateManager}} is purely
an in-memory state representation that (currently) does not read/write to
container meta store at all. Seems to me that we should keep this abstraction,
leave {{ContainerStateManager}} away from container.db and only let
{{ContainerMapping}} do the container metadata management.
So although we are indeed missing the {{allocatedSize}} from allocateBlock code
path, I would prefer to leave this part as-is. What do you think?
Nonetheless, containerStateManager.close() not being does look like a bug, will
upload a patch later.
> Ozone: SCM: update container allocated size to container db for all the open
> containers in ContainerStateManager#close
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-12751
> URL: https://issues.apache.org/jira/browse/HDFS-12751
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Nanda kumar
> Assignee: Chen Liang
>
> Container allocated size is maintained in memory by
> {{ContainerStateManager}}, this has to be updated in container db when we
> shutdown SCM. {{ContainerStateManager#close}} will be called during SCM
> shutdown, so updating allocated size for all the open containers should be
> done here.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
