Thanks again Madhan.... I haven't figured out exactly why it was failing earlier, but when providing more fully qualified parameters on the command line (and after a reboot of the image, which may also have had an impact), things are working!
For future reference, I am now seeing messages immediately, as expected when using this complete syntax: ./kafka-console-consumer.sh --bootstrap-server sandbox-hdp.hortonworks.com:6667 --topic ATLAS_ENTITIES --new-consmer --from-beginning --consumer.config=/usr/hdp/current/kafka-broker/config/consumer.properties Ernie Ernie Ostic Worldwide Technical Sales InfoSphere Information Server IBM Analytics Cell: (617) 331 8238 --------------------------------------------------------------- Apache Atlas Update! https://dsrealtime.wordpress.com/2017/11/16/apache-atlas-update-have-you-been-watching/ Open IGC is here! https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/ From: "Ernie Ostic" <eos...@us.ibm.com> To: user@atlas.apache.org Date: 03/10/2018 02:03 PM Subject: Re: Not getting notifications when new Entities are created... Thanks Madhan... Yes...the good news is that the hook is indeed working. ...and to watch the topic, I am opening up a console listener directly: ./kafka-console-consumer.sh --bootstrap-server localhost:6667 --topic ATLAS_ENTITIES ...thus far, I am not seeing anything here when a new table is created, nor with a more general --from-beginning keyword for the consumer. Thanks in advance for any suggestions on where else to look... Ernie Ernie Ostic Worldwide Technical Sales InfoSphere Information Server IBM Analytics Cell: (617) 331 8238 --------------------------------------------------------------- Apache Atlas Update! https://dsrealtime.wordpress.com/2017/11/16/apache-atlas-update-have-you-been-watching/ Open IGC is here! https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/ Inactive hide details for Madhan Neethiraj ---03/10/2018 01:43:44 PM---Ernie,Madhan Neethiraj ---03/10/2018 01:43:44 PM---Ernie, From: Madhan Neethiraj <mad...@apache.org> To: "user@atlas.apache.org" <user@atlas.apache.org> Date: 03/10/2018 01:43 PM Subject: Re: Not getting notifications when new Entities are created... Ernie, Given the table created in Hive appears in Atlas, the notification from Atlas Hive hook to ATLAS_HOOK topic is working. Else, the table will not be seen in Atlas. Notification to ATLAS_ENTITIES topic is sent by Atlas server for any change to entity (create/update/delete, add/update/remove of classifications). Are you seeing any messages at all in ATLAS_ENTITIES topic? For troubleshooting purposes, I would suggest to listen for messages using kafka-console-consumer.sh. Hope this helps. Madhan From: Ernie Ostic <eos...@us.ibm.com> Reply-To: "user@atlas.apache.org" <user@atlas.apache.org> Date: Saturday, March 10, 2018 at 10:25 AM To: "user@atlas.apache.org" <user@atlas.apache.org> Subject: Not getting notifications when new Entities are created... Hi all... Having difficulties with Atlas notifications. This use case (simplified) is now using a .8 within an HDP Sandbox (2.6.4 from 2018-01) just to be sure I have the mechanics worked out. Hoping to see Entity related messages when I create a new Hive table. The Hive table is created successfully, and the table appears just fine in Atlas, but I never get any messages in the ATLAS_ENTITIES or ATLAS_HOOK topics..... The topics appear to be created just fine, and the port and others settings seem to be correct in the atlas config file..... atlas.notification.create.topics=true atlas.notification.embedded=false atlas.notification.replicas=1 atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES Any suggestions on what to look for? ...or what I'm missing or not setting correctly? Thanks! Ernie Ernie Ostic Worldwide Technical Sales InfoSphere Information Server IBM Analytics Cell: (617) 331 8238 --------------------------------------------------------------- Apache Atlas Update! https://dsrealtime.wordpress.com/2017/11/16/apache-atlas-update-have-you-been-watching/ Open IGC is here! https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
