Hi Mohit, >From the log message, I assume that you are using an existing atlas-application.properties copied from somewhere (most likely from HDP environment) and PLAINTEXTSASL is used in it. PLAINTEXTSASL is not supported by the ReportLineageToAtlas.
As a work-around, please set 'Create Atlas Configuration File' to true and let the reporting task generate atlas-application.properties instead. SASL_PLAINTEXT is identical to PLAINTEXTSASL. You may need to restart NiFi to take effect. Hope this helps, Koji On Thu, Jul 26, 2018 at 7:12 PM, Mohit <[email protected]> wrote: > Hi, > > > > While looking at the logs, I found out that ReportingLineageToAtlas is not > able to construct KafkaProducer. > > It throws the following logs – > > > > org.apache.kafka.common.KafkaException: Failed to construct kafka producer > > at > org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335) > > at > org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188) > > at > org.apache.atlas.kafka.KafkaNotification.createProducer(KafkaNotification.java:286) > > at > org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:207) > > at > org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:84) > > at > org.apache.atlas.hook.AtlasHook.notifyEntitiesInternal(AtlasHook.java:133) > > at > org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:118) > > at > org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:171) > > at > org.apache.nifi.atlas.NiFiAtlasHook.commitMessages(NiFiAtlasHook.java:150) > > at > org.apache.nifi.atlas.reporting.ReportLineageToAtlas.lambda$consumeNiFiProvenanceEvents$6(ReportLineageToAtlas.java:721) > > at > org.apache.nifi.reporting.util.provenance.ProvenanceEventConsumer.consumeEvents(ProvenanceEventConsumer.java:204) > > at > org.apache.nifi.atlas.reporting.ReportLineageToAtlas.consumeNiFiProvenanceEvents(ReportLineageToAtlas.java:712) > > at > org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:664) > > at > org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: java.lang.IllegalArgumentException: No enum constant > org.apache.kafka.common.protocol.SecurityProtocol.PLAINTEXTSASL > > at java.lang.Enum.valueOf(Enum.java:238) > > at > org.apache.kafka.common.protocol.SecurityProtocol.valueOf(SecurityProtocol.java:28) > > at > org.apache.kafka.common.protocol.SecurityProtocol.forName(SecurityProtocol.java:89) > > at > org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:79) > > at > org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:277) > > ... 20 common frames omitted > > > > Thanks, > > Mohit > > > > From: Mohit <[email protected]> > Sent: 25 July 2018 17:46 > To: [email protected] > Subject: Unable to see Nifi data lineage in Atlas > > > > Hi all, > > > > I have configured ReportingLineageToAtlas reporting task to send Nifi flow > information to Atlas. Nifi is integrated with Ranger. > > I am able to see all the information in the Atlas except the lineage. When I > search for hdfs_path or hive_table, I can only see the hive side > information. I can’t figure out anything wrong in the configuration. > > Is there something in the Ranger configuration that I’m missing? > > > > Regards, > > Mohit > > > >
