[
https://issues.apache.org/jira/browse/ATLAS-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aditya Gupta updated ATLAS-5199:
--------------------------------
Description:
*Issue:*
Whenever NotificationHookConsumer fails to process message, After first retry,
the notification message having EMPTY string in date field, gets changed to
null
*Sample hook message snippet:*
*{*
*"version": {*
*"version": "1.0.0",*
*"versionParts": [*
*1*
*]*
*},*
*"msgCompressionKind": "NONE",*
*"msgSplitIdx": 1,*
*"msgSplitCount": 1,*
*"msgCreationTime": 1758064193365,*
*"spooled": false,*
*"message": {*
*"type": "ENTITY_CREATE_V2",*
*"user": "pbsdwp73",*
*"entities": {*
*"referredEntities": {},*
*"entities": [*
*{*
*"typeName": "impala_process",*
*"attributes": {*
*"recentQueries": [*
*"insert into table
`dl_prj73_raw`.`prj73mssbcomplaintsworkingfolder_prj73_complaints_ccis_tc_combined`
select from
`dl_prj28_raw`.`tc_t360consolidation_dl_complaints_ccis_tc_combined`"*
*],*
*"qualifiedName":
"QUERY:dl_prj28_raw.tc_t360consolidation_dl_complaints_ccis_tc_combined@cm:1757434301000->:INSERT:dl_prj73_raw.prj73mssbcomplaintsworkingfolder_prj73_complaints_ccis_tc_combined@cm:1757478805000",*
*"name":
"QUERY:dl_prj28_raw.tc_t360consolidation_dl_complaints_ccis_tc_combined@cm:1757434301000->:INSERT:dl_prj73_raw.prj73mssbcomplaintsworkingfolder_prj73_complaints_ccis_tc_combined@cm:1757478805000",*
*"queryText": "",*
*"operationType": "QUERY",*
*"startTime": null,*
*"queryPlan": "Not Supported",*
*"endTime": null,*
*"userName": "",*
*"queryId": ""*
*},*
*"guid": "-1817980182010191",*
*"isIncomplete": false,*
*"provenanceType": 0,*
*"version": 0,*
*"relationshipAttributes": {*
*"outputs": [*
*{*
*"typeName": "hive_table",*
*"uniqueAttributes": {*
*"qualifiedName":
"dl_prj73_raw.prj73mssbcomplaintsworkingfolder_prj73_complaints_ccis_tc_combined@cm"*
*}*
*}*
*],*
*"inputs": [*
*{*
*"typeName": "hive_table",*
*"uniqueAttributes": {*
*"qualifiedName":
"dl_prj28_raw.tc_t360consolidation_dl_complaints_ccis_tc_combined@cm"*
*}*
*}*
*]*
*},*
*"proxy": false*
*},*
*Exception:*
2025-09-02 13:57:00,670 WARN - [NotificationHookConsumer thread-0:] ~ Error
handling message (NotificationHookConsumer$HookConsumer:829)
org.apache.atlas.exception.AtlasBaseException: Invalid instance
creation/updation parameters passed : impala_process.startTime: mandatory
attribute value missing in type impala_process
at
org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:109)
at
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:1275)
at
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:1145)
at
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:366)
at
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke(<generated>)
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
at
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:111)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)
at
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$810c18d0.createOrUpdate(<generated>)
at
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:899)
at
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:733)
at
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:557)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
was:
*Issue:*
Whenever NotificationHookConsumer fails to process message, After first retry,
the notification message having EMPTY string in date field, gets changed to
null
> Impala "startTime" attribute changes to null from EMPTY string, after 1 retry
> on NotificationHookConsumer
> ---------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-5199
> URL: https://issues.apache.org/jira/browse/ATLAS-5199
> Project: Atlas
> Issue Type: Bug
> Reporter: Aditya Gupta
> Assignee: Aditya Gupta
> Priority: Major
>
> *Issue:*
> Whenever NotificationHookConsumer fails to process message, After first
> retry, the notification message having EMPTY string in date field, gets
> changed to null
>
> *Sample hook message snippet:*
> *{*
> *"version": {*
> *"version": "1.0.0",*
> *"versionParts": [*
> *1*
> *]*
> *},*
> *"msgCompressionKind": "NONE",*
> *"msgSplitIdx": 1,*
> *"msgSplitCount": 1,*
> *"msgCreationTime": 1758064193365,*
> *"spooled": false,*
> *"message": {*
> *"type": "ENTITY_CREATE_V2",*
> *"user": "pbsdwp73",*
> *"entities": {*
> *"referredEntities": {},*
> *"entities": [*
> *{*
> *"typeName": "impala_process",*
> *"attributes": {*
> *"recentQueries": [*
> *"insert into table
> `dl_prj73_raw`.`prj73mssbcomplaintsworkingfolder_prj73_complaints_ccis_tc_combined`
> select from
> `dl_prj28_raw`.`tc_t360consolidation_dl_complaints_ccis_tc_combined`"*
> *],*
> *"qualifiedName":
> "QUERY:dl_prj28_raw.tc_t360consolidation_dl_complaints_ccis_tc_combined@cm:1757434301000->:INSERT:dl_prj73_raw.prj73mssbcomplaintsworkingfolder_prj73_complaints_ccis_tc_combined@cm:1757478805000",*
> *"name":
> "QUERY:dl_prj28_raw.tc_t360consolidation_dl_complaints_ccis_tc_combined@cm:1757434301000->:INSERT:dl_prj73_raw.prj73mssbcomplaintsworkingfolder_prj73_complaints_ccis_tc_combined@cm:1757478805000",*
> *"queryText": "",*
> *"operationType": "QUERY",*
> *"startTime": null,*
> *"queryPlan": "Not Supported",*
> *"endTime": null,*
> *"userName": "",*
> *"queryId": ""*
> *},*
> *"guid": "-1817980182010191",*
> *"isIncomplete": false,*
> *"provenanceType": 0,*
> *"version": 0,*
> *"relationshipAttributes": {*
> *"outputs": [*
> *{*
> *"typeName": "hive_table",*
> *"uniqueAttributes": {*
> *"qualifiedName":
> "dl_prj73_raw.prj73mssbcomplaintsworkingfolder_prj73_complaints_ccis_tc_combined@cm"*
> *}*
> *}*
> *],*
> *"inputs": [*
> *{*
> *"typeName": "hive_table",*
> *"uniqueAttributes": {*
> *"qualifiedName":
> "dl_prj28_raw.tc_t360consolidation_dl_complaints_ccis_tc_combined@cm"*
> *}*
> *}*
> *]*
> *},*
> *"proxy": false*
> *},*
>
> *Exception:*
> 2025-09-02 13:57:00,670 WARN - [NotificationHookConsumer thread-0:] ~ Error
> handling message (NotificationHookConsumer$HookConsumer:829)
> org.apache.atlas.exception.AtlasBaseException: Invalid instance
> creation/updation parameters passed : impala_process.startTime: mandatory
> attribute value missing in type impala_process
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:109)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:1275)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:1145)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:366)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke(<generated>)
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
> at
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:111)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$810c18d0.createOrUpdate(<generated>)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:899)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:733)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:557)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
