Andrii Tkach created AMBARI-10129:
-------------------------------------
Summary: Pig Client needs Tez Client on same host, ambari web need
to update HostRoles it sends
Key: AMBARI-10129
URL: https://issues.apache.org/jira/browse/AMBARI-10129
Project: Ambari
Issue Type: Bug
Components: ambari-web
Affects Versions: 2.0.0
Reporter: Andrii Tkach
Assignee: Andrii Tkach
Fix For: 2.1.0
Pig service check fails if the Tez client is not installed on the same host
that contains the Pig client.
If during the Install Wizard, Pig and Tez are added at the same time, then this
guarantees that whatever hosts are selected to contain the clients will have
both services, which works.
However, if Tez is added first (plus it dependencies on MR2, Yarn) on say host
3, and then Pig service is added, and the client is only installed on host 1,
then the Pig service check will fail.
I tried adding this to PIG's metainfo.xml
{code}
<dependencies>
<dependency>
<name>TEZ/TEZ_CLIENT</name>
<scope>host</scope>
<auto-deploy>
<enabled>true</enabled>
</auto-deploy>
</dependency>
<dependency>
<name>MAPREDUCE2/MAPREDUCE2_CLIENT</name>
<scope>host</scope>
<auto-deploy>
<enabled>true</enabled>
</auto-deploy>
</dependency>
</dependencies>
{code}
However, it was not sufficient for Tez Client to be installed on the same hosts.
When I installed Pig, I saw that ambari-web only sent the request as,
{code}
POST http://c6401.ambari.apache.org:8080/api/v1/clusters/c1/hosts
{"RequestInfo":{"query":"Hosts/host_name:
c6401.ambari.apache.org"},"Body":{"host_components":[{"HostRoles":{"component_name":"PIG"}}]}}
{code}
*Notice that the HostRoles only contains PIG; it should also be adding
TEZ_CLIENT and MAPREDUCE2_CLIENT, as per the dependencies shown above*
This is the failure that Pig's service check generates,
{code}
15/02/11 20:17:28 INFO pig.ExecTypeProvider: Picked TEZ as the ExecType
...
2015-02-11 20:17:35,966 [PigTezLauncher-0] ERROR
org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG
org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez
jars, tez.lib.uris is not defined in the configuration
at
org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:162)
at
org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:721)
at org.apache.tez.client.TezClient.start(TezClient.java:298)
at
org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:95)
at
org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:195)
at
org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:159)
at
org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:167)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
