[
https://issues.apache.org/jira/browse/ATLAS-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3140:
------------------------------
Description:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
{{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
calling {{BaseHiveEvent.toAtlasEntit}}
was:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
Entity.isDummy() to judge whether the entity is dummy when Hive Hook calling
BaseHiveEvent.toAtlasEntity()
> Hive Hook throws InvalidTableException when select from
> _dummy_database._dummy_table
> ------------------------------------------------------------------------------------
>
> Key: ATLAS-3140
> URL: https://issues.apache.org/jira/browse/ATLAS-3140
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: 1.0.0, 1.1.0
> Reporter: Jiaqi Shan
> Priority: Critical
> Fix For: 1.0.0, 1.1.0
>
>
> When we run sql like 'select 1+1', following error is logged in hive.log:
> {code:java}
> 2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167))
> - HiveHook.run(): failed to process operation QUERY
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> _dummy_table
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
> The exception is caused by getting metadata from a dual table. So we use
> {{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
> calling {{BaseHiveEvent.toAtlasEntit}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
