Re: Review Request 51121: [PREVIEW] Oozie Hive actions fail when Atlas is installed since Atlas Hive Hooks need to be copied to Oozie Share Lib in HDFS

2016-08-16 Thread Venkat Ranganathan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51121/#review145866
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/hive-site.xml
 (line 24)


Why do we need this file?


- Venkat Ranganathan


On Aug. 15, 2016, 5:40 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51121/
> ---
> 
> (Updated Aug. 15, 2016, 5:40 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty and Venkat Ranganathan.
> 
> 
> Bugs: AMBARI-18151
> https://issues.apache.org/jira/browse/AMBARI-18151
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Whenever Oozie submits Hive actions, they fail and the application logs show 
> "Internal Error: 
> java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook"
> 
> This is because whenever Atlas is installed, the Hive Hook (which is 
> installed via the Oozie RPM too, in /usr/$stack/$version/atlas/hook/hive/) 
> needs to be copied to the Oozie Share lib in HDFS (e.g., 
> hdfs:///user/oozie/share/lib/lib_$timestamp/hive)
> 
> Further, need to save atlas-application.properties used for Hive in 
> /etc/oozie/conf/action-conf/hive
> 
> Lastly, remove the atlas.rest.address property from hive-site.xml
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  dd3a692 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
>  e330cb5 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  db048bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/hive-site.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51121/diff/
> 
> 
> Testing
> ---
> 
> Verified on a live cluster with HDP 2.5 and Oozie, Hive, Atlas
> 
> Python unit tests passed,
> 
> OK
> --
> Total run:1034
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 51121: [PREVIEW] Oozie Hive actions fail when Atlas is installed since Atlas Hive Hooks need to be copied to Oozie Share Lib in HDFS

2016-08-15 Thread Alejandro Fernandez 


> On Aug. 16, 2016, 12:54 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py,
> >  line 381
> > 
> >
> > Will this call need kinit or sudo ever?

Will test in a Kerberized cluster.


> On Aug. 16, 2016, 12:54 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py,
> >  line 370
> > 
> >
> > How do users prevent this? Is it expected that at least on Oozie server 
> > host has atlas installed?

The hive hook lives in /usr/hdp/$version/atlas/hook/hive but the dependency is 
installed via an RPM like 
atlas-metadata_2_5_0_0_1203-hive-plugin-0.7.0.2.5.0.0-1203 which is actually 
brought in by Hive. On a host that only has Oozie server, I need to check that 
it is installed.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51121/#review145821
---


On Aug. 16, 2016, 12:40 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51121/
> ---
> 
> (Updated Aug. 16, 2016, 12:40 a.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty and Venkat Ranganathan.
> 
> 
> Bugs: AMBARI-18151
> https://issues.apache.org/jira/browse/AMBARI-18151
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Whenever Oozie submits Hive actions, they fail and the application logs show 
> "Internal Error: 
> java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook"
> 
> This is because whenever Atlas is installed, the Hive Hook (which is 
> installed via the Oozie RPM too, in /usr/$stack/$version/atlas/hook/hive/) 
> needs to be copied to the Oozie Share lib in HDFS (e.g., 
> hdfs:///user/oozie/share/lib/lib_$timestamp/hive)
> 
> Further, need to save atlas-application.properties used for Hive in 
> /etc/oozie/conf/action-conf/hive
> 
> Lastly, remove the atlas.rest.address property from hive-site.xml
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  dd3a692 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
>  e330cb5 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  db048bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/hive-site.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51121/diff/
> 
> 
> Testing
> ---
> 
> Verified on a live cluster with HDP 2.5 and Oozie, Hive, Atlas
> 
> Python unit tests passed,
> 
> OK
> --
> Total run:1034
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>