[
https://issues.apache.org/jira/browse/HIVE-29477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-29477:
--------------------------------
Description:
Given the exception described in TEZ-4686:
https://issues.apache.org/jira/secure/attachment/13080838/hs2_stacktrace.txt
{code}
Caused by: java.lang.NullPointerException: Cannot invoke
"org.apache.tez.client.registry.AMRecord.getApplicationId()" because
"this.amRecord" is null
at
org.apache.tez.client.registry.zookeeper.ZkFrameworkClient.createApplication(ZkFrameworkClient.java:114)
at
org.apache.tez.client.TezClient.createApplication(TezClient.java:1103)
at org.apache.tez.client.TezClient.start(TezClient.java:399)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:488)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternalUnsafe(TezSessionState.java:406)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:297)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.openInternal(TezSessionPoolSession.java:122)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:250)
at
org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:481)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:232)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
{code}
this is related to the assumption made in TEZ-4007:
https://github.com/apache/tez/blob/17546aa680e6f9a52411fe6a66c7a26de76e53a6/tez-api/src/main/java/org/apache/tez/client/registry/zookeeper/ZkFrameworkClient.java#L91
So the point of this issue is: *how to acquire an application id*, and this is
closely related to the standalone zookeeper mode in Tez.
What actually happens in Tez Yarn world
was:
Given the exception described in TEZ-4686:
https://issues.apache.org/jira/secure/attachment/13080838/hs2_stacktrace.txt
{code}
Caused by: java.lang.NullPointerException: Cannot invoke
"org.apache.tez.client.registry.AMRecord.getApplicationId()" because
"this.amRecord" is null
at
org.apache.tez.client.registry.zookeeper.ZkFrameworkClient.createApplication(ZkFrameworkClient.java:114)
at
org.apache.tez.client.TezClient.createApplication(TezClient.java:1103)
at org.apache.tez.client.TezClient.start(TezClient.java:399)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:488)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternalUnsafe(TezSessionState.java:406)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:297)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.openInternal(TezSessionPoolSession.java:122)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:250)
at
org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:481)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:232)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
{code}
> Introduce codepath for Tez external sessions discovered by Zookeeper
> --------------------------------------------------------------------
>
> Key: HIVE-29477
> URL: https://issues.apache.org/jira/browse/HIVE-29477
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Priority: Major
>
> Given the exception described in TEZ-4686:
> https://issues.apache.org/jira/secure/attachment/13080838/hs2_stacktrace.txt
> {code}
> Caused by: java.lang.NullPointerException: Cannot invoke
> "org.apache.tez.client.registry.AMRecord.getApplicationId()" because
> "this.amRecord" is null
> at
> org.apache.tez.client.registry.zookeeper.ZkFrameworkClient.createApplication(ZkFrameworkClient.java:114)
> at
> org.apache.tez.client.TezClient.createApplication(TezClient.java:1103)
> at org.apache.tez.client.TezClient.start(TezClient.java:399)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:488)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternalUnsafe(TezSessionState.java:406)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:297)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.openInternal(TezSessionPoolSession.java:122)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:250)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:481)
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:232)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> {code}
> this is related to the assumption made in TEZ-4007:
> https://github.com/apache/tez/blob/17546aa680e6f9a52411fe6a66c7a26de76e53a6/tez-api/src/main/java/org/apache/tez/client/registry/zookeeper/ZkFrameworkClient.java#L91
> So the point of this issue is: *how to acquire an application id*, and this
> is closely related to the standalone zookeeper mode in Tez.
> What actually happens in Tez Yarn world
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
