Sarath Subramanian created ATLAS-3475: -----------------------------------------
Summary: Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable Key: ATLAS-3475 URL: https://issues.apache.org/jira/browse/ATLAS-3475 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.0.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Fix For: 2.1.0 While running CTAS query via spark.sql, Atlas is throwing below error {code:java} org.apache.atlas.exception.AtlasBaseException: Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceableorg.apache.atlas.exception.AtlasBaseException: Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable at org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:103) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:938) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:748) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:279) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$91e799f5.createOrUpdate(<generated>) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:802) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:700) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:521) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:89) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) \{code} The above error comes even in a newly created cluster, where there are (were) no shell entities present. the error message is mostly harmless since the entities are getting created properly. Shell entities too are getting resolved correctly. But in the metrics UI, each CTAS query shows up as a failure under ATLAS_HOOK-0 failed messages Steps to reproduce: # spark-shell # spark.sql("create table t1(name string)") # spark.sql("create table t2 as select * from t1") -- This message was sent by Atlassian Jira (v8.3.4#803005)