[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369810#comment-15369810
]
Hudson commented on YARN-4210:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #10074 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10074/])
YARN-4210. HBase reader throws NPE if Get returns no rows (Varun Saxena (sjlee:
rev 708fa8b1ae85b6efda318368bc0c0ba02d4958c8)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/FlowRunEntityReader.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineEntityReader.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/ApplicationEntityReader.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorage.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/FlowActivityEntityReader.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/GenericEntityReader.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesFlowRun.java
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Fix For: YARN-2928
>
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> This JIRA handles multiple issues.
> * If HBase Get does not fetch any rows for the query, we still try to parse
> the result and read fields. This leads to NPE while reading metrics. We
> should not attempt to read anything if no row is returned i.e. result is
> empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
> * ResultScanner is not closed in HBase Reader.
> * Exception encountered while reading start and end time in
> FlowRunEntityReader
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939596#comment-14939596
]
Varun Saxena commented on YARN-4210:
Thanks [~vrushalic] for the review and commit. And Li and Sangjin for the
reviews
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Fix For: YARN-2928
>
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> This JIRA handles multiple issues.
> * If HBase Get does not fetch any rows for the query, we still try to parse
> the result and read fields. This leads to NPE while reading metrics. We
> should not attempt to read anything if no row is returned i.e. result is
> empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
> * ResultScanner is not closed in HBase Reader.
> * Exception encountered while reading start and end time in
> FlowRunEntityReader
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938858#comment-14938858
]
Vrushali C commented on YARN-4210:
--
Committed patch v3 in, thanks [~varun_saxena] for the jira and the patch and
[~gtCarrera], [~sjlee0] for the reviews.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Fix For: YARN-2928
>
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> This JIRA handles multiple issues.
> * If HBase Get does not fetch any rows for the query, we still try to parse
> the result and read fields. This leads to NPE while reading metrics. We
> should not attempt to read anything if no row is returned i.e. result is
> empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
> * ResultScanner is not closed in HBase Reader.
> * Exception encountered while reading start and end time in
> FlowRunEntityReader
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938217#comment-14938217
]
Vrushali C commented on YARN-4210:
--
+ 1 from me too on patch v3. Will commit this in after committing in YARN-4203.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938185#comment-14938185
]
Li Lu commented on YARN-4210:
-
+1.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936495#comment-14936495
]
Varun Saxena commented on YARN-4210:
Have updated a patch on YARN-4203.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936413#comment-14936413
]
Li Lu commented on YARN-4210:
-
Of course we should net only report user. I believe detailed user information
is also important when the URL does not contain an user parameter. I'm fine
with a fix in YARN-4203 though. Thanks for the work!
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936407#comment-14936407
]
Varun Saxena commented on YARN-4210:
[~gtCarrera9],
Entity may not be found for reasons other than user as well. So printing only
user doesnt seem complete to me.
But from a debugging point of view this info should be available somewhere.
In YARN-4203 we will be adding request/response logging. Maybe I can print
request user while printing "Received URL " message. If URL does not
contain user we can assume while debugging that request user was taken.
Thoughts ?
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936328#comment-14936328
]
Li Lu commented on YARN-4210:
-
Yes, LGTM overall. My only reservation is about the logging message.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936258#comment-14936258
]
Sangjin Lee commented on YARN-4210:
---
I took a quick look at the patch, and the changes LGTM for the most part, with
the understanding that the query that returns all apps for a given flow run
will be done in YARN-3864.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936251#comment-14936251
]
Sangjin Lee commented on YARN-4210:
---
Regarding the flow run start/end time, yes, it is because
{{GenericObjectMapper}} converts numbers within the integer range into an
{{Integer}}. What Varun did in the v.3 patch is the right thing to do.
Apparently we missed it through our unit test because we happened to use
artificial start/end time that were larger than the integer max.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936113#comment-14936113
]
Hadoop QA commented on YARN-4210:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 20m 23s | Pre-patch YARN-2928 compilation
is healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 8m 20s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 0s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 30s | The applied patch generated 1
new checkstyle issues (total was 33, now 34). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 41s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 52s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 2m 54s | Tests passed in
hadoop-yarn-server-timelineservice. |
| | | 45m 44s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12764335/YARN-4210-YARN-2928.03.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / def22b9 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9301/artifact/patchprocess/diffcheckstylehadoop-yarn-server-timelineservice.txt
|
| hadoop-yarn-server-timelineservice test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9301/artifact/patchprocess/testrun_hadoop-yarn-server-timelineservice.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9301/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9301/console |
This message was automatically generated.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936045#comment-14936045
]
Li Lu commented on YARN-4210:
-
bq. Can you print whats coming as user in the request i.e. in
TimelineReaderWebServices ? I suspect this might be a user different from what
is present in the table.
Verified and this part LGTM. It would be helpful to report the user in the not
found exception though?
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch, YARN-4210-YARN-2928.03.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936026#comment-14936026
]
Varun Saxena commented on YARN-4210:
[~gtCarrera9],
bq. From my end to end test I can see there were exceptions converting Integer
to Long in FlowRunEntityReader#parseEntity. Specifically, I changed the code to
firstly converting startTime and endTime to Number then get their long values
to get rid of the problem. Not sure why we're getting integers here.
This has to do with GenericObjectMapper. In readResults we use it. It will
automatically interpret a numeric value as an Integer if its less than max
value of int i.e. .2,147,483,647.
Current seconds since epoch will be a value lower than max value of int. Hence,
the exception.
As these fields are taken as info, we can merely typecast them into a Number
and take a long value to resolve the problem. Just as you did.
Will fix it as part of this JIRA.
bq. I'm not sure why I always need to add the ?userid= part to query for the
flows launched by myself?
You do not need to if user in request is same as the one in the records present
in flow run table.
Can you print whats coming as user in the request i.e. in
TimelineReaderWebServices ? I suspect this might be a user different from what
is present in the table.
I had tried querying without userid. And user is taken from request's caller
UGI. hope I am not missing something here.
bq. Also, w.r.t application queries, let's prioritize it since with the current
set of REST APIs, we cannot query anything related to applications unless we
know its id. We need some ways to list all apps in a flow run to finish a full
story. Hopefully this can happen in parallel with this JIRA to speed the whole
thing up.
Yes, already started this. You can watch YARN-3864 for updates. Need to update
the JIRA title though.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935989#comment-14935989
]
Li Lu commented on YARN-4210:
-
Also, w.r.t application queries, let's prioritize it since with the current set
of REST APIs, we cannot query anything related to applications unless we know
its id. We need some ways to list all apps in a flow run to finish a full
story. Hopefully this can happen in parallel with this JIRA to speed the whole
thing up.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935976#comment-14935976
]
Li Lu commented on YARN-4210:
-
Applied the patch and I can see the NPE problem is gone. Digging a little bit
more I thing two quick changes would make this a more comprehensive fix the the
flowrun inquiry:
- I'm not sure why I always need to add the {{?userid=}} part to query for the
flows launched by myself? Are we missing anything there in parseUser? Why can't
we directly use callerUGI if there's no userid parameter?
- From my end to end test I can see there were exceptions converting Integer to
Long in FlowRunEntityReader#parseEntity. Specifically, I changed the code to
firstly converting startTime and endTime to Number then get their long values
to get rid of the problem. Not sure why we're getting integers here.
cc/[~vrushalic][~sjlee0].
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935870#comment-14935870
]
Sangjin Lee commented on YARN-4210:
---
No worries. We can modify the description of YARN-3864 and do it there. Let me
know if you need my help.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935747#comment-14935747
]
Varun Saxena commented on YARN-4210:
[~sjlee0], at the time by aggregated entities we meant querying flows, apps,
etc. The name is a little generic. Will change the title and do part about
querying apps there i.e. from REST to changes in HBase reader.
If you say, I can raise a separate JIRA and leave 3864 as it is.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935737#comment-14935737
]
Sangjin Lee commented on YARN-4210:
---
But this has nothing to do with aggregation, right? This is to support queries
on the application table.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935724#comment-14935724
]
Varun Saxena commented on YARN-4210:
[~sjlee0], will do this is as part of YARN-3864. That has been earmarked for
app side query interface.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935706#comment-14935706
]
Sangjin Lee commented on YARN-4210:
---
Thanks for the patch [~varun_saxena]. I'll take a look at the current patch,
and will get back to you with comments.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935704#comment-14935704
]
Sangjin Lee commented on YARN-4210:
---
That's the current behavior, but I can see that it would be useful if we can
query multiple applications for a given flow run. Would you like to take care
of that as part of this JIRA? If not, I can take a look at it as a separate
JIRA. Let me know what you want to do. Note that the validation also needs to
change to support it.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935700#comment-14935700
]
Sangjin Lee commented on YARN-4210:
---
That's right. {{FlowActivityEntityReader}} does it a little differently because
it uses {{PageFilter}}, and can rely on the strict ordering of results.
Good find on {{ResultScanner}} not being closed! They should be closed once the
query is done.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935500#comment-14935500
]
Hadoop QA commented on YARN-4210:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 14s | Findbugs (version ) appears to
be broken on YARN-2928. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 8m 11s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 17s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 26s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 16s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 42s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 40s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 56s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 3m 22s | Tests passed in
hadoop-yarn-server-timelineservice. |
| | | 42m 10s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12764263/YARN-4210-YARN-2928.02.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / def22b9 |
| hadoop-yarn-server-timelineservice test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9297/artifact/patchprocess/testrun_hadoop-yarn-server-timelineservice.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9297/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9297/console |
This message was automatically generated.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935365#comment-14935365
]
Varun Saxena commented on YARN-4210:
Found that we do not support getting multiple apps from application table. We
wont need to query something like list all apps for a flow run ?
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch,
> YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935313#comment-14935313
]
Varun Saxena commented on YARN-4210:
Point 2 is invalid.
I was under the impression that we will use PageFilter everywhere.
But just checked we will be fetching all entities in GenericEntityReader.
Makes sense too. As entity id can be anything for generic entity table and
lexicographical ordering of row key may not always match with created time.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935165#comment-14935165
]
Varun Saxena commented on YARN-4210:
Moreover, there are couple of other issues I noticed while going through the
code.
# I cannot see ResultScanner#close being called in xxxEntityReader classes.
Depending on number of queries going on in parallel, this may become an issue
at server side. Server will expire the scanner after a configured lease
interval if close is not called explicitly.
# Also is FlowActivityEntityReader#readEntities required ?
TimelineEntityReader#readEntities should suffice. In
TimelineEntityReader#readEntities though we can check when we reach the limit
instead of going over limit and then calling pollLast. In short follow same
approach as FlowActivityEntityReader,
cc [~sjlee0], [~vrushalic], [~jrottinghuis].
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
[
https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934981#comment-14934981
]
Hadoop QA commented on YARN-4210:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 15s | Findbugs (version ) appears to
be broken on YARN-2928. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 3s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 12s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 16s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 42s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 55s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 3m 17s | Tests passed in
hadoop-yarn-server-timelineservice. |
| | | 41m 45s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12764215/YARN-4210-YARN-2928.01.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / def22b9 |
| hadoop-yarn-server-timelineservice test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9292/artifact/patchprocess/testrun_hadoop-yarn-server-timelineservice.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9292/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9292/console |
This message was automatically generated.
> HBase reader throws NPE if Get returns no rows
> --
>
> Key: YARN-4210
> URL: https://issues.apache.org/jira/browse/YARN-4210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4210-YARN-2928.01.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the
> result and read fields. This leads to NPE while reading metrics. We should
> not attempt to read anything if no row is returned i.e. result is empty.
> Found during web UI poc testing.
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0]
> reader.TimelineReaderWebServices
> (TimelineReaderWebServices.java:handleException(199)) - Error while
> processing REST request
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
