[
https://issues.apache.org/jira/browse/ATLAS-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17391992#comment-17391992
]
ASF subversion and git services commented on ATLAS-3033:
--------------------------------------------------------
Commit ed81639bbc3cc8babec8170be367052e49dc8f84 in atlas's branch
refs/heads/branch-2.0 from Ashutosh Mestry
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=ed81639 ]
ATLAS-4152: Entity correlation for deleted entities.
Signed-off-by: Nikhil Bonte <[email protected]>
(cherry picked from commit c3df22605795b9bddcba547852c04461ee9b8203)
ATLAS-3033: Skip hive temp table while getting Table object details from
hiveContext.
Signed-off-by: Madhan Neethiraj <[email protected]>
(cherry picked from commit b5472ce313dcf24b0bffbe5201f0c95eec025848)
(cherry picked from commit 0323ce3825c63b06d7b0032edd10b73b84078a4a)
> ATLAS Hive hook fails for CTAS query involving UDFs
> ---------------------------------------------------
>
> Key: ATLAS-3033
> URL: https://issues.apache.org/jira/browse/ATLAS-3033
> Project: Atlas
> Issue Type: Bug
> Affects Versions: 0.8.3
> Reporter: Nixon Rodrigues
> Assignee: Nixon Rodrigues
> Priority: Major
> Fix For: 0.8.4, 1.2.0, 2.0.0
>
> Attachments: ATLAS-3033-branch-2.0.patch, ATLAS-3033-master.patch,
> ATLAS-3033.patch
>
>
> ATLAS Hive Hook fails to notify for CTAS queries involving UDFs, Hook fails
> with the exception as below
> {code}
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> _dummy_table
> {code}
> complete stack trace.
> {code}
> 2018-12-26 12:15:22,234 ERROR [HiveServer2-Background-Pool: Thread-339]:
> metadata.Hive (Hive.java:getTable(1220)) - Table _dummy_table not found:
> _dummy_database._
> dummy_table table not found
> 2018-12-26 12:15:22,234 ERROR [HiveServer2-Background-Pool: Thread-339]:
> hook.HiveHook (HiveHook.java:run(160)) - HiveHook.run(): failed to process
> operation CREAT
> ETABLE_AS_SELECT
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> _dummy_table
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1221)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1191)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:225)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:203)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:84)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:59)
> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:157)
> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:52)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1619)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
> at
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2018-12-26 12:15:22,234 INFO [HiveServer2-Background-Pool: Thread-339]:
> log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG
> method=PostHook.org.apache.atlas.hive.hook.HiveHook start=1545826522229
> end=1545826522234 duration=5 from=org.apache.hadoop.hive.ql.Driver>
> {code}
> The problem looks to be appearing as result being stored for UDF in temporary
> table _dummy_database._dummy_table and while creating a Lineage the process
> fails.
> Steps to reproduce is running below SQL.
> {code}
> create table masktest11 as select stack(2, "eddie", "111", "ravi", "222") as
> (name, ssn);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
