Thanks Madhan... Yes...the good news is that the hook is indeed working. ...and to watch the topic, I am opening up a console listener directly:
./kafka-console-consumer.sh --bootstrap-server localhost:6667 --topic ATLAS_ENTITIES ...thus far, I am not seeing anything here when a new table is created, nor with a more general --from-beginning keyword for the consumer. Thanks in advance for any suggestions on where else to look... Ernie Ernie Ostic Worldwide Technical Sales InfoSphere Information Server IBM Analytics Cell: (617) 331 8238 --------------------------------------------------------------- Apache Atlas Update! https://dsrealtime.wordpress.com/2017/11/16/apache-atlas-update-have-you-been-watching/ Open IGC is here! https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/ From: Madhan Neethiraj <[email protected]> To: "[email protected]" <[email protected]> Date: 03/10/2018 01:43 PM Subject: Re: Not getting notifications when new Entities are created... Ernie, Given the table created in Hive appears in Atlas, the notification from Atlas Hive hook to ATLAS_HOOK topic is working. Else, the table will not be seen in Atlas. Notification to ATLAS_ENTITIES topic is sent by Atlas server for any change to entity (create/update/delete, add/update/remove of classifications). Are you seeing any messages at all in ATLAS_ENTITIES topic? For troubleshooting purposes, I would suggest to listen for messages using kafka-console-consumer.sh. Hope this helps. Madhan From: Ernie Ostic <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Saturday, March 10, 2018 at 10:25 AM To: "[email protected]" <[email protected]> Subject: Not getting notifications when new Entities are created... Hi all... Having difficulties with Atlas notifications. This use case (simplified) is now using a .8 within an HDP Sandbox (2.6.4 from 2018-01) just to be sure I have the mechanics worked out. Hoping to see Entity related messages when I create a new Hive table. The Hive table is created successfully, and the table appears just fine in Atlas, but I never get any messages in the ATLAS_ENTITIES or ATLAS_HOOK topics..... The topics appear to be created just fine, and the port and others settings seem to be correct in the atlas config file..... atlas.notification.create.topics=true atlas.notification.embedded=false atlas.notification.replicas=1 atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES Any suggestions on what to look for? ...or what I'm missing or not setting correctly? Thanks! Ernie Ernie Ostic Worldwide Technical Sales InfoSphere Information Server IBM Analytics Cell: (617) 331 8238 --------------------------------------------------------------- Apache Atlas Update! https://dsrealtime.wordpress.com/2017/11/16/apache-atlas-update-have-you-been-watching/ Open IGC is here! https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
