Hi,
While looking at the logs, I found out that ReportingLineageToAtlas is not
able to construct KafkaProducer.
It throws the following logs -
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at
org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:33
5)
at
org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:18
8)
at
org.apache.atlas.kafka.KafkaNotification.createProducer(KafkaNotification.ja
va:286)
at
org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java
:207)
at
org.apache.atlas.notification.AbstractNotification.send(AbstractNotification
.java:84)
at
org.apache.atlas.hook.AtlasHook.notifyEntitiesInternal(AtlasHook.java:133)
at
org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:118)
at
org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:171)
at
org.apache.nifi.atlas.NiFiAtlasHook.commitMessages(NiFiAtlasHook.java:150)
at
org.apache.nifi.atlas.reporting.ReportLineageToAtlas.lambda$consumeNiFiProve
nanceEvents$6(ReportLineageToAtlas.java:721)
at
org.apache.nifi.reporting.util.provenance.ProvenanceEventConsumer.consumeEve
nts(ProvenanceEventConsumer.java:204)
at
org.apache.nifi.atlas.reporting.ReportLineageToAtlas.consumeNiFiProvenanceEv
ents(ReportLineageToAtlas.java:712)
at
org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineage
ToAtlas.java:664)
at
org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapp
er.java:41)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$
301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Sch
eduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
49)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
24)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: No enum constant
org.apache.kafka.common.protocol.SecurityProtocol.PLAINTEXTSASL
at java.lang.Enum.valueOf(Enum.java:238)
at
org.apache.kafka.common.protocol.SecurityProtocol.valueOf(SecurityProtocol.j
ava:28)
at
org.apache.kafka.common.protocol.SecurityProtocol.forName(SecurityProtocol.j
ava:89)
at
org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:7
9)
at
org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:27
7)
... 20 common frames omitted
Thanks,
Mohit
From: Mohit <[email protected]>
Sent: 25 July 2018 17:46
To: [email protected]
Subject: Unable to see Nifi data lineage in Atlas
Hi all,
I have configured ReportingLineageToAtlas reporting task to send Nifi flow
information to Atlas. Nifi is integrated with Ranger.
I am able to see all the information in the Atlas except the lineage. When I
search for hdfs_path or hive_table, I can only see the hive side
information. I can't figure out anything wrong in the configuration.
Is there something in the Ranger configuration that I'm missing?
Regards,
Mohit
