[
https://issues.apache.org/jira/browse/EAGLE-381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huizhi Lu updated EAGLE-381:
----------------------------
Description:
About the attempt_id, assuming attempt_id starting from 1 may not be completely
correct. This will not work in HDP Sandbox 2.4. Please see the attempt id in
sandbox. Spark version is 1.6.0.
I am confused why the attempt id are in different naming rule.
We need to fix this issue. I will take care of it.
hdp-2.4
[ {
"id" : "application_1468625664674_0003",
"name" : "Spark Pi",
"attempts" : [ {
"attemptId" : "appattempt_1468625664674_0003_000001",
"startTime" : "2016-07-16T00:13:16.320GMT",
"endTime" : "2016-07-16T00:13:24.279GMT",
"sparkUser" : "spark",
"completed" : true
} ]
}, {
"id" : "application_1468625664674_0002",
"name" : "Spark Pi",
"attempts" : [ {
"startTime" : "2016-07-16T00:12:01.386GMT",
"endTime" : "2016-07-16T00:12:18.181GMT",
"sparkUser" : "spark",
"completed" : true
} ]
cluster.
"id" : "application_1464382345557_264985",
"name" : "xxxx",
"attempts" : [ {
"attemptId" : "1",
"startTime" : "2016-07-15T09:01:25.032GMT",
"endTime" : "2016-07-15T09:58:29.941GMT",
"sparkUser" : "xxx",
"completed" : true
} ]
solution:
get resource manager version from api/conf and set attempt id depending on the
version .
was:
About the attempt_id, assuming attempt_id starting from 1 may not be completely
correct. This will not work in HDP Sandbox 2.4. Please see the attempt id in
sandbox. Spark version is 1.6.0.
I am confused why the attempt id are in different naming rule.
We need to fix this issue. I will take care of it.
hdp-2.4
[ {
"id" : "application_1468625664674_0003",
"name" : "Spark Pi",
"attempts" : [ {
"attemptId" : "appattempt_1468625664674_0003_000001",
"startTime" : "2016-07-16T00:13:16.320GMT",
"endTime" : "2016-07-16T00:13:24.279GMT",
"sparkUser" : "spark",
"completed" : true
} ]
}, {
"id" : "application_1468625664674_0002",
"name" : "Spark Pi",
"attempts" : [ {
"startTime" : "2016-07-16T00:12:01.386GMT",
"endTime" : "2016-07-16T00:12:18.181GMT",
"sparkUser" : "spark",
"completed" : true
} ]
Ares.
"id" : "application_1464382345557_264985",
"name" : "xxxx",
"attempts" : [ {
"attemptId" : "1",
"startTime" : "2016-07-15T09:01:25.032GMT",
"endTime" : "2016-07-15T09:58:29.941GMT",
"sparkUser" : "xxx",
"completed" : true
} ]
> Get resource manager version and improve app attempt id fetcher for spark job
> feeder
> ------------------------------------------------------------------------------------
>
> Key: EAGLE-381
> URL: https://issues.apache.org/jira/browse/EAGLE-381
> Project: Eagle
> Issue Type: Bug
> Affects Versions: v0.5.0
> Reporter: Huizhi Lu
> Assignee: Huizhi Lu
> Fix For: v0.5.0
>
>
> About the attempt_id, assuming attempt_id starting from 1 may not be
> completely correct. This will not work in HDP Sandbox 2.4. Please see the
> attempt id in sandbox. Spark version is 1.6.0.
> I am confused why the attempt id are in different naming rule.
> We need to fix this issue. I will take care of it.
> hdp-2.4
> [ {
> "id" : "application_1468625664674_0003",
> "name" : "Spark Pi",
> "attempts" : [ {
> "attemptId" : "appattempt_1468625664674_0003_000001",
> "startTime" : "2016-07-16T00:13:16.320GMT",
> "endTime" : "2016-07-16T00:13:24.279GMT",
> "sparkUser" : "spark",
> "completed" : true
> } ]
> }, {
> "id" : "application_1468625664674_0002",
> "name" : "Spark Pi",
> "attempts" : [ {
> "startTime" : "2016-07-16T00:12:01.386GMT",
> "endTime" : "2016-07-16T00:12:18.181GMT",
> "sparkUser" : "spark",
> "completed" : true
> } ]
> cluster.
> "id" : "application_1464382345557_264985",
> "name" : "xxxx",
> "attempts" : [ {
> "attemptId" : "1",
> "startTime" : "2016-07-15T09:01:25.032GMT",
> "endTime" : "2016-07-15T09:58:29.941GMT",
> "sparkUser" : "xxx",
> "completed" : true
> } ]
> solution:
> get resource manager version from api/conf and set attempt id depending on
> the version .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
