[jira] [Commented] (TEZ-4192) Attach task specific info in scheduler and retrieve in communicator
[
https://issues.apache.org/jira/browse/TEZ-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121466#comment-17121466
]
TezQA commented on TEZ-4192:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
39s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
13s{color} | {color:red} tez-ext-service-tests in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
18s{color} | {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
12s{color} | {color:red} tez-ext-service-tests in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 18s{color}
| {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 12s{color}
| {color:red} tez-ext-service-tests in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 22s{color} | {color:orange} tez-dag: The patch generated 9 new + 206
unchanged - 2 fixed = 215 total (was 208) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 9s{color} | {color:orange} tez-ext-service-tests: The patch generated 3 new
+ 11 unchanged - 0 fixed = 14 total (was 11) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} tez-ext-service-tests in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
45s{color} | {color:green} tez-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 19s{color}
| {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 13s{color}
| {color:red} tez-ext-service-tests in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.11 Server=19.03.11 base:
https://builds.apache.org/job/PreCommit-TEZ-Build/458/artifact/out/Dockerfile |
| JIRA Issue |
[jira] [Updated] (TEZ-4192) Attach task specific info in scheduler and retrieve in communicator
[ https://issues.apache.org/jira/browse/TEZ-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated TEZ-4192: -- Attachment: HIVE-23589.patch > Attach task specific info in scheduler and retrieve in communicator > --- > > Key: TEZ-4192 > URL: https://issues.apache.org/jira/browse/TEZ-4192 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.9.2 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Attachments: TEZ-4192.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > There is no way to pass information from scheduler to communicator. Sometimes > we want to let communicator know about extra properties of the task other > than its spec. One example is if a task is preemptable. > When scheduler receives a task, its taskspec is already constructed. It is a > nice immutable object at this point. We do not want to modify and put more > info in it at that stage. So we pass a new piece of information to > TaskSchedulerContext#taskAllocated along with the allocation. This info is > passed to the communicator untouched. It is up to the communicator > implementation to make use of it. Communicator implementation needs to > override a new registerRunningTaskAttempt that accepts this additional object > along with all the other arguments in the original registerRunningTaskAttempt > method. > The new registerRunningTaskAttempt method has a default implementation that > falls back to the old method. So if the user does not override the new > method, the same old behavior is preserved. Similarly on scheduler side, the > user needs to call new taskAllocated method to pass the additional info > object. If they use the old taskAllocated method, old behavior is preserved. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4192) Attach task specific info in scheduler and retrieve in communicator
[ https://issues.apache.org/jira/browse/TEZ-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated TEZ-4192: -- Attachment: (was: HIVE-23589.patch) > Attach task specific info in scheduler and retrieve in communicator > --- > > Key: TEZ-4192 > URL: https://issues.apache.org/jira/browse/TEZ-4192 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.9.2 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Attachments: TEZ-4192.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > There is no way to pass information from scheduler to communicator. Sometimes > we want to let communicator know about extra properties of the task other > than its spec. One example is if a task is preemptable. > When scheduler receives a task, its taskspec is already constructed. It is a > nice immutable object at this point. We do not want to modify and put more > info in it at that stage. So we pass a new piece of information to > TaskSchedulerContext#taskAllocated along with the allocation. This info is > passed to the communicator untouched. It is up to the communicator > implementation to make use of it. Communicator implementation needs to > override a new registerRunningTaskAttempt that accepts this additional object > along with all the other arguments in the original registerRunningTaskAttempt > method. > The new registerRunningTaskAttempt method has a default implementation that > falls back to the old method. So if the user does not override the new > method, the same old behavior is preserved. Similarly on scheduler side, the > user needs to call new taskAllocated method to pass the additional info > object. If they use the old taskAllocated method, old behavior is preserved. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (TEZ-4192) Attach task specific info in scheduler and retrieve in communicator
Mustafa Iman created TEZ-4192: - Summary: Attach task specific info in scheduler and retrieve in communicator Key: TEZ-4192 URL: https://issues.apache.org/jira/browse/TEZ-4192 Project: Apache Tez Issue Type: Improvement Affects Versions: 0.9.2 Reporter: Mustafa Iman Assignee: Mustafa Iman There is no way to pass information from scheduler to communicator. Sometimes we want to let communicator know about extra properties of the task other than its spec. One example is if a task is preemptable. When scheduler receives a task, its taskspec is already constructed. It is a nice immutable object at this point. We do not want to modify and put more info in it at that stage. So we pass a new piece of information to TaskSchedulerContext#taskAllocated along with the allocation. This info is passed to the communicator untouched. It is up to the communicator implementation to make use of it. Communicator implementation needs to override a new registerRunningTaskAttempt that accepts this additional object along with all the other arguments in the original registerRunningTaskAttempt method. The new registerRunningTaskAttempt method has a default implementation that falls back to the old method. So if the user does not override the new method, the same old behavior is preserved. Similarly on scheduler side, the user needs to call new taskAllocated method to pass the additional info object. If they use the old taskAllocated method, old behavior is preserved. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4105) Tez job-analyzer tool to support proto logging history
[
https://issues.apache.org/jira/browse/TEZ-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121349#comment-17121349
]
TezQA commented on TEZ-4105:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 5m
22s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
32s{color} | {color:red} tez-tools/analyzers/job-analyzer in master has 4
extant findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
17s{color} | {color:red} root in master has 3 extant findbugs warnings. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch passed checkstyle in
tez-protobuf-history-plugin {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} tez-plugins/tez-history-parser: The patch generated
0 new + 66 unchanged - 1 fixed = 66 total (was 67) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} The patch passed checkstyle in job-analyzer {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} root: The patch generated 0 new + 170 unchanged - 1
fixed = 170 total (was 171) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} tez-protobuf-history-plugin in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
15s{color} | {color:green} tez-history-parser in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
25s{color} | {color:green} job-analyzer in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 59m
29s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} |
[jira] [Commented] (TEZ-4172) Let tasks be killed after too many overall attempts
[ https://issues.apache.org/jira/browse/TEZ-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121332#comment-17121332 ] Ashutosh Chauhan commented on TEZ-4172: --- [~rajesh.balamohan] can you please help review this? > Let tasks be killed after too many overall attempts > --- > > Key: TEZ-4172 > URL: https://issues.apache.org/jira/browse/TEZ-4172 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4172.01.patch > > > Currently, TaskImpl doesn't consider failing a task if there are too many > overall attempts. In case of LLAP, the number of preempted task attempts -> > overall task attempts [can grow in a > linkedhashmap|https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/TaskImpl.java#L127]. > In an edge case, where an upstream application (Hive LLAP) cannot cope with a > problematic query, this can also lead to OOM in the AM, due the very high > number of TaskAttemptImpl objects. > It would be beneficial to have the chance to limit the overall number of task > attempts, regardless of they have been failed or killed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4105) Tez job-analyzer tool to support proto logging history
[ https://issues.apache.org/jira/browse/TEZ-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121279#comment-17121279 ] László Bodor commented on TEZ-4105: --- rebased, uploaded [^TEZ-4105.11.patch] to double-check patch's compatibility with master > Tez job-analyzer tool to support proto logging history > -- > > Key: TEZ-4105 > URL: https://issues.apache.org/jira/browse/TEZ-4105 > Project: Apache Tez > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4105.01.patch, TEZ-4105.02.patch, TEZ-4105.03.patch, > TEZ-4105.04.patch, TEZ-4105.05.patch, TEZ-4105.06.patch, TEZ-4105.07.patch, > TEZ-4105.08.patch, TEZ-4105.09.patch, TEZ-4105.10.patch, TEZ-4105.11.patch, > dag_1583980529217__18_1, dag_1583980529217__18_1_1 > > > Currently analyzers in tez-tools can only work with output of ats (zipped > json files) and simple history logging (plain text json file) files. It would > be nice to have a parser that can create the needed info for analyzers from a > dag protobuf file. In order to achieve this, we need at least a converter > which can convert HistoryProtoEvent instances to a format which can be read > by TezAnalyzerBase (+ a new parser is needed, similarly to > SimpleHistoryParser) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4105) Tez job-analyzer tool to support proto logging history
[ https://issues.apache.org/jira/browse/TEZ-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4105: -- Attachment: TEZ-4105.11.patch > Tez job-analyzer tool to support proto logging history > -- > > Key: TEZ-4105 > URL: https://issues.apache.org/jira/browse/TEZ-4105 > Project: Apache Tez > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4105.01.patch, TEZ-4105.02.patch, TEZ-4105.03.patch, > TEZ-4105.04.patch, TEZ-4105.05.patch, TEZ-4105.06.patch, TEZ-4105.07.patch, > TEZ-4105.08.patch, TEZ-4105.09.patch, TEZ-4105.10.patch, TEZ-4105.11.patch, > dag_1583980529217__18_1, dag_1583980529217__18_1_1 > > > Currently analyzers in tez-tools can only work with output of ats (zipped > json files) and simple history logging (plain text json file) files. It would > be nice to have a parser that can create the needed info for analyzers from a > dag protobuf file. In order to achieve this, we need at least a converter > which can convert HistoryProtoEvent instances to a format which can be read > by TezAnalyzerBase (+ a new parser is needed, similarly to > SimpleHistoryParser) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}
{code:java}
public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
timelineClient = TimelineClient.createTimelineClient();
timelineClient.init(conf);
}
else
{
this.timelineClient = null;
...
}
...
}
{code}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else { this.timelineClient = null; ... }
}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else
{
this.timelineClient = null;
...
}
}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else
{ this.timelineClient = null; ... }
}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else
{
this.timelineClient = null; ... }
}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else {
this.timelineClient = null;
...
}
}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else
{
this.timelineClient = null;
...
}
}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else { this.timelineClient = null; ... }
}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
{quote}public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else
{
this.timelineClient = null;
...
}
}
{quote}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.io.IOException: Timeline V1 client is not properly configured.
Either timeline service is not enabled or version is not set to 1.x
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ...
14 more
2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#ff}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#ff}_timelineClient.init(conf);_ {color}
}
else
{ this.timelineClient = null; ... }
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error
starting DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173) at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960) at
org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630) at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error
starting DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173) at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960) at
org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630) at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)Caused by:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
... 14 more2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED))
{
{color:#FF}_timelineClient = TimelineClient.createTimelineClient();_ {color}
{color:#FF}_timelineClient.init(conf);_ {color}
}
else
{
this.timelineClient = null;
...
}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error
starting DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173) at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960) at
org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630) at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)Caused
[jira] [Updated] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
[
https://issues.apache.org/jira/browse/TEZ-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
aimahou updated TEZ-4191:
-
Description:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error
starting DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173) at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960) at
org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630) at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)Caused by:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
... 14 more2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
*Related code snippet*
ATSHistoryLoggingService.java
public void serviceInit(Configuration conf) throws Exception {
...
if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED)) {
timelineClient = TimelineClient.createTimelineClient();
timelineClient.init(conf);
} else {
this.timelineClient = null;
...
}
was:
h3. *Issue:*
Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f
in property yarn.timeline-service.versions
*Property in yarn-site.xml:*
yarn.webapp.api-service.enable
true
yarn.timeline-service.version
2.0f
yarn.timeline-service.versions
2.0f
*Specific exception:*
2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error
starting DAGAppMasterorg.apache.hadoop.service.ServiceStateException:
java.io.IOException: Timeline V1 client is not properly configured. Either
timeline service is not enabled or version is not set to 1.x at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173) at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960) at
org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630) at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at
org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)Caused by:
java.io.IOException: Timeline V1 client is not
[jira] [Created] (TEZ-4191) Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions
aimahou created TEZ-4191: Summary: Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions Key: TEZ-4191 URL: https://issues.apache.org/jira/browse/TEZ-4191 Project: Apache Tez Issue Type: Bug Affects Versions: 0.9.1 Reporter: aimahou h3. *Issue:* Tez ATS plugin init failed when enable yarn.timeline-service without set 1.5f in property yarn.timeline-service.versions *Property in yarn-site.xml:* yarn.webapp.api-service.enable true yarn.timeline-service.version 2.0f yarn.timeline-service.versions 2.0f *Specific exception:* 2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting DAGAppMaster2020-05-11 16:54:39,518 [ERROR] [main] |app.DAGAppMaster|: Error starting DAGAppMasterorg.apache.hadoop.service.ServiceStateException: java.io.IOException: Timeline V1 client is not properly configured. Either timeline service is not enabled or version is not set to 1.x at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173) at org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:110) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108) at org.apache.tez.dag.history.HistoryEventHandler.serviceInit(HistoryEventHandler.java:105) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.tez.dag.app.DAGAppMaster.initServices(DAGAppMaster.java:1960) at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:630) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659) at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)Caused by: java.io.IOException: Timeline V1 client is not properly configured. Either timeline service is not enabled or version is not set to 1.x at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:100) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) ... 14 more2020-05-11 16:54:39,520 [INFO] [shutdown-hook-0] |app.DAGAppMaster|: DAGAppMasterShutdownHook invoked -- This message was sent by Atlassian Jira (v8.3.4#803005)
