[ 
https://issues.apache.org/jira/browse/YARN-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013263#comment-15013263
 ] 

Weiwei Yang commented on YARN-3556:
-----------------------------------

I just spent some time looking at this problem. This is not just for an empty 
entry in environment, any tag in xml if it is empty, you will get NPE. Yarn 
leverages protobuf to serialize data, this logic is everywhere, I don't see a 
good option to fix though I admit the error message is not quite user friendly.

> Yarn REST to submit application fails with NPE if the <environment> element 
> has empty entries
> ---------------------------------------------------------------------------------------------
>
>                 Key: YARN-3556
>                 URL: https://issues.apache.org/jira/browse/YARN-3556
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>    Affects Versions: 2.6.0
>            Reporter: Rajesh Kartha
>            Assignee: Weiwei Yang
>            Priority: Minor
>         Attachments: wc-no-cp-NPE.xml
>
>
> I was trying out the YARN REST api to submit applications and ran into a NPE 
> where the <environment> element has empty <entry> tags
> example:
>  <am-container-spec>      
>               <environment>                   
>                       <entry>                   
>                       </entry>
>               </environment>
>               <commands>
>                       <command>hadoop jar 
> /usr/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar wordcount 
> /tmp/zkSmoke.out /tmp/REST-WC-4</command>
>               </commands>
>     </am-container-spec>
> The Exception:
> 2015-04-28 09:34:41,725 WARN  resourcemanager.RMAuditLogger 
> (RMAuditLogger.java:logFailure(263)) - USER=dr.who  OPERATION=Application 
> Finished - Failed 
> TARGET=RMAppManagerRESULT=FAILURE  DESCRIPTION=App failed with state: FAILED  
>      PERMISSIONS=Application application_1430176860750_0024 failed 
> 2 times due to Error launching appattempt_1430176860750_0024_000002. 
> Got exception: java.lang.NullPointerException
>         at 
> org.apache.hadoop.yarn.proto.YarnProtos$StringStringMapProto$Builder.setKey(YarnProtos.java:44696)
>         at 
> org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl$3$1.next(ContainerLaunchContextPBImpl.java:376)
>         at 
> org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl$3$1.next(ContainerLaunchContextPBImpl.java:364)
>         at 
> com.google.protobuf.AbstractMessageLite$Builder.checkForNullValues(AbstractMessageLite.java:336)
>         at 
> com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:323)
>         at 
> org.apache.hadoop.yarn.proto.YarnProtos$ContainerLaunchContextProto$Builder.addAllEnvironment(YarnProtos.java:39623)
>         at 
> org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl.addEnvToProto(ContainerLaunchContextPBImpl.java:387)
>         at 
> org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl.mergeLocalToBuilder(ContainerLaunchContextPBImpl.java:115)
>         at 
> org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl.mergeLocalToProto(ContainerLaunchContextPBImpl.java:128)
>         at 
> org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl.getProto(ContainerLaunchContextPBImpl.java:70)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl.convertToProtoFormat(StartContainerRequestPBImpl.java:156)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl.mergeLocalToBuilder(StartContainerRequestPBImpl.java:85)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl.mergeLocalToProto(StartContainerRequestPBImpl.java:95)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl.getProto(StartContainerRequestPBImpl.java:57)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.convertToProtoFormat(StartContainersRequestPBImpl.java:137)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.addLocalRequestsToProto(StartContainersRequestPBImpl.java:97)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.mergeLocalToBuilder(StartContainersRequestPBImpl.java:79)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.mergeLocalToProto(StartContainersRequestPBImpl.java:72)
>         at 
> org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.getProto(StartContainersRequestPBImpl.java:48)
>         at 
> org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:93)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:119)
>         at 
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:254)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> . Failing the application.      APPID=application_1430176860750_0024
> Would be good to continue the job instead of the NPE



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to