[
https://issues.apache.org/jira/browse/YARN-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625613#comment-13625613
]
Zhijie Shen commented on YARN-555:
----------------------------------
Check the code in ContainerLaunchContextPBImpl:
{code}
@Override
public Map<String, LocalResource> getLocalResources() {
initLocalResources();
return this.localResources;
}
private void initLocalResources() {
if (this.localResources != null) {
return;
}
ContainerLaunchContextProtoOrBuilder p = viaProto ? proto : builder;
List<StringLocalResourceMapProto> list = p.getLocalResourcesList();
this.localResources = new HashMap<String, LocalResource>();
for (StringLocalResourceMapProto c : list) {
this.localResources.put(c.getKey(), convertFromProtoFormat(c.getValue()));
}
}
@Override
public void setLocalResources(
final Map<String, LocalResource> localResources) {
if (localResources == null)
return;
initLocalResources();
this.localResources.clear();
this.localResources.putAll(localResources);
}
{code}
In your second case, localResources is the same object of
appContext.getAMContainerSpec().localResources. When setLocalResources is
called, the instance is cleaned up first, and then appends itself, which
already contains no items.
The buggy part is that setters don't really set the collections but append them
to the existing ones. It's not the only problem with
ContainerLaunchContextPBImpl, but with all XXXXPBImpls that set collections
(see YARN-489). A special issue with ContainerLaunchContextPBImpl is that
appending will be done after cleaning up.
> ContainerLaunchContext is buggy when it comes to setter methods on a new
> instance
> ---------------------------------------------------------------------------------
>
> Key: YARN-555
> URL: https://issues.apache.org/jira/browse/YARN-555
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
> Affects Versions: 2.0.3-alpha
> Reporter: Harsh J
> Priority: Minor
>
> If you look at the API of ContainerLaunchContext, its got setter methods,
> such as for setResource, setCommands, etc…:
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/api/records/ContainerLaunchContext.html#setCommands(java.util.List)
> However, there's certain things broken in its use here that am trying to
> understand. Let me explain with some code context:
> 1. I initialize a proper CLC for an ApplicationSubmissionContext (appContext).
> {code}
> ContainerLaunchContext appMasterLaunchContext =
> Records.newRecord(ContainerLaunchContext.class);
> appContext.setAMContainerSpec(appMasterLaunchContext);
> {code}
> 2. I create a resource request of 130 MB, as applicationMasterResource, and
> try to set it into the CLC via:
> {code}
> appContext.getAMContainerSpec().setResource(applicationMasterResource);
> {code}
> 3. This works OK. If I query it back now, it returns 130 for a
> {{getMemory()}} call.
> 4. So I attempt to do the same with setCommands/setEnvironment/etc., all of
> which fail to mutate cause the check in CLC's implementation class disregards
> whatever I try to set for some reason.
> Edit: It seems like the issue is that when I do a
> appContext.getAMContainerSpec().getLocalResources() or similar call to get
> existing initialized data structures to populate further on, what I really
> get underneath is a silently non-mutative data structure that I can call .put
> or .add on, but it won't really reflect it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
