Jason Lowe commented on YARN-3143:

I think this is caused by this code in the RMWebServices.java:

    final ConcurrentMap<ApplicationId, RMApp> apps =
    AppsInfo allApps = new AppsInfo();
    for (ApplicationReport report : appReports) {
      RMApp rmapp = apps.get(report.getApplicationId());

      if (finalStatusQuery != null && !finalStatusQuery.isEmpty()) {
        if (!rmapp.getFinalApplicationStatus().toString()
            .equalsIgnoreCase(finalStatusQuery)) {

      AppInfo app = new AppInfo(rmapp, hasAccess(rmapp, hsr),

The problem here is that we're trying to look at a pool of applications 
asynchronously, therefore it is possible that we could get an app report for an 
application that is gone by the time we call apps.get(report.getApplicationId). 
 That means rmapp can be null, and note that AppInfo's constructor effectively 
does nothing if the RMApp argument is null.  That explains the lack of all 
sorts of fields by the time it gets to JSON.

So there are a couple of problems here:

# We should not be adding AppInfos if rmapp is null
# We can blindly dereference rmapp and generate an NPE if there's a final 
status query

> RM Apps REST API can return NPE or entries missing id and other fields
> ----------------------------------------------------------------------
>                 Key: YARN-3143
>                 URL: https://issues.apache.org/jira/browse/YARN-3143
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: webapp
>    Affects Versions: 2.5.2
>            Reporter: Kendall Thrapp
>            Assignee: Jason Lowe
> I'm seeing intermittent null pointer exceptions being returned by
> the YARN Apps REST API.
> For example:
> {code}
> http://{cluster}:{port}/ws/v1/cluster/apps?finalStatus=UNDEFINED
> {code}
> JSON Response was:
> {code}
> {"RemoteException":{"exception":"NullPointerException","javaClassName":"java.lang.NullPointerException"}}
> {code}
> At a glance appears to be only when we query for unfinished apps (i.e. 
> finalStatus=UNDEFINED).  
> Possibly related, when I do get back a list of apps, sometimes one or more of 
> the apps will be missing most of the fields, like id, name, user, etc., and 
> the fields that are present all have zero for the value.  
> For example:
> {code}
> {"progress":0.0,"clusterId":0,"applicationTags":"","startedTime":0,"finishedTime":0,"elapsedTime":0,"allocatedMB":0,"allocatedVCores":0,"runningContainers":0,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0}
> {code}
> Let me know if there's any other information I can provide to help debug.

This message was sent by Atlassian JIRA

Reply via email to