[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936026#comment-14936026
]
Varun Saxena commented on YARN-4210:
------------------------------------
[~gtCarrera9],
bq. From my end to end test I can see there were exceptions converting Integer
to Long in FlowRunEntityReader#parseEntity. Specifically, I changed the code to
firstly converting startTime and endTime to Number then get their long values
to get rid of the problem. Not sure why we're getting integers here.
This has to do with GenericObjectMapper. In readResults we use it. It will
automatically interpret a numeric value as an Integer if its less than max
value of int i.e. .2,147,483,647.
Current seconds since epoch will be a value lower than max value of int. Hence,
the exception.
As these fields are taken as info, we can merely typecast them into a Number
and take a long value to resolve the problem. Just as you did.
Will fix it as part of this JIRA.
bq. I'm not sure why I always need to add the ?userid= part to query for the
flows launched by myself?
You do not need to if user in request is same as the one in the records present
in flow run table.
Can you print whats coming as user in the request i.e. in
TimelineReaderWebServices ? I suspect this might be a user different from what
is present in the table.
I had tried querying without userid. And user is taken from request's caller
UGI. hope I am not missing something here.
bq. Also, w.r.t application queries, let's prioritize it since with the current
set of REST APIs, we cannot query anything related to applications unless we
know its id. We need some ways to list all apps in a flow run to finish a full
story. Hopefully this can happen in parallel with this JIRA to speed the whole
thing up.
Yes, already started this. You can watch YARN-3864 for updates. Need to update
the JIRA title though.
> HBase reader throws NPE if Get returns no rows
> ----------------------------------------------
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
