-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62457/
-----------------------------------------------------------
(Updated Sept. 21, 2017, 5:39 p.m.)
Review request for atlas, Apoorv Naik, Ashutosh Mestry, Sarath Subramanian, and
Suma Shivaprasad.
Summary (updated)
-----------------
ATLAS-2157: HiveHook error while processing temporary tables
Bugs: ATLAS-2157
https://issues.apache.org/jira/browse/ATLAS-2157
Repository: atlas
Description
-------
As reported in user list, executing Hive SQL that use temprorary tables (for
example insert statement) causes Atlas hook to report the following failure.
The fix is to ignore this error, as it doesn't result in loss of functionality.
17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive: Table
values__tmp__table__1 not found: default.values__tmp__table__1 table not found
17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed due
to error
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.registerProcess()
failed.
at
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701)
at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83)
at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
... 6 more
Caused by: org.apache.atlas.hook.AtlasHookException:
HiveHook.processHiveEntity() failed.
at
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731)
at
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668)
... 12 more
Caused by: org.apache.atlas.hook.AtlasHookException:
HiveHook.createOrUpdateEntities() failed.
at
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597)
at
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711)
... 13 more
Caused by: org.apache.atlas.hook.AtlasHookException:
HiveHook.createOrUpdateEntities() failed.
at
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589)
at
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not
found values__tmp__table__1
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186)
at
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568)
... 15 more
Diffs
-----
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
7dc2e2f1
Diff: https://reviews.apache.org/r/62457/diff/1/
Testing
-------
Verified that the error is not reported in the following cases:
- CREATE TEMPORARY TABLE tmp1(c1 string);
- insert into tmp1 values('1');
Also verified that lineage details are recorded correctly - for both table and
column level lineages:
- CREATE TABLE t1(c1 STRING, c2 STRING);
- CREATE TABLE s1(c1 STRING, c2 STRING);
- CREATE VIEW v_t1_s1 AS SELECT (t.c1 + s.c1) c1, (t.c2 + s.c2) c2 FROM t1 t,
s1 s;
Thanks,
Madhan Neethiraj
