Tsuyoshi OZAWA commented on YARN-2229:

Thanks for the comment, Jian and Anubhav. I took same time to understand what 
we're thinking.

If we'll have a choice to preserve cluster-level compatibility for rolling 
updates, the current design(v10) is not acceptable. We should choose the first 
design as Sid and Zhijie mentioned in this case. However, before starting to 
think cluster-level compatibility at container id level, we don't support 
rolling update because ContainerToken don't have compatibility. Jian's 
suggestion cares it - we don't need to preserve cluster-level compatibility 
currently because we cannot support rolling update. Therefore, I think the 
current design(v10), is acceptable. [~sseth], [~zjshen], [~adhoot], [~jianhe], 
do you agree with v10 design based on these discussion?

> ContainerId can overflow with RM restart
> ----------------------------------------
>                 Key: YARN-2229
>                 URL: https://issues.apache.org/jira/browse/YARN-2229
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-2229.1.patch, YARN-2229.10.patch, 
> YARN-2229.10.patch, YARN-2229.2.patch, YARN-2229.2.patch, YARN-2229.3.patch, 
> YARN-2229.4.patch, YARN-2229.5.patch, YARN-2229.6.patch, YARN-2229.7.patch, 
> YARN-2229.8.patch, YARN-2229.9.patch
> On YARN-2052, we changed containerId format: upper 10 bits are for epoch, 
> lower 22 bits are for sequence number of Ids. This is for preserving 
> semantics of {{ContainerId#getId()}}, {{ContainerId#toString()}}, 
> {{ContainerId#compareTo()}}, {{ContainerId#equals}}, and 
> {{ConverterUtils#toContainerId}}. One concern is epoch can overflow after RM 
> restarts 1024 times.
> To avoid the problem, its better to make containerId long. We need to define 
> the new format of container Id with preserving backward compatibility on this 

This message was sent by Atlassian JIRA

Reply via email to