[ 
https://issues.apache.org/jira/browse/YARN-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048129#comment-14048129
 ] 

Tsuyoshi OZAWA commented on YARN-2229:
--------------------------------------

I took some time to think about the new format. IMHO, if it's necessary for us 
to assure backward compatibility, we need to use current format and deal with 
overflow. Current idea is as follows:
* ContainerId will have epoch as 64 bit field based on the value stored on 
ZKRMStateStore.
* {{ContainerId#compareTo}}, {{ContainerId#equals}} use the epoch field to deal 
with the overflow.
* {{ContainerId#toString}} will show the 64 bit epoch as suffix of current 
container id. {{ConverterUtils#toContainerId}} will be updated to parse epoch.

64 bits are used like this:
{code}
|54bits|10bits|
|0|0| // inital value 
|0|1023| // before overflow
|1|1024==0| // overflowed. toString shows only lower 10 bits for backward 
compatibility.
|1|1|
|2|1024==0|
{code}

Old {{ConverterUtil#toContainerId}} can still parse the new format of 
{{ContainerId#toString}}. One problem is that the result of old 
{{ConverterUtil#toContainerId}} cannot care the overflow of epoch. [~jianhe], 
[~bikassaha], what do you think?

> Making ContainerId long type
> ----------------------------
>
>                 Key: YARN-2229
>                 URL: https://issues.apache.org/jira/browse/YARN-2229
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>
> On YARN-2052, we changed containerId format: upper 10 bits are for epoch, 
> lower 22 bits are for sequence number of Ids. This is for preserving 
> semantics of {{ContainerId#getId()}}, {{ContainerId#toString()}}, 
> {{ContainerId#compareTo()}}, {{ContainerId#equals}}, and 
> {{ConverterUtils#toContainerId}}. One concern is epoch can overflow after RM 
> restarts 1024 times.
> To avoid the problem, its better to make containerId long. We need to define 
> the new format of container Id with preserving backward compatibility on this 
> JIRA.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to