Hi Koji I am trying to evaluate HDF NIfi from security perspective. I am trying to make sure when HDF Nifi talks to HDP , it does not leak/spill any kind of information on HDP data nodes (i.e. on local disk). I am fine if it is writing it on HDFS.
On Thu, Jun 15, 2017 at 2:35 AM, Koji Kawamura <[email protected]> wrote: > Hi Shashi, > > Sorry for delayed response. I am not aware that NiFi writes any > provenance information on HDP nodes. But if your goal is to expose > NiFi provenance data to HDFS, Hive (or Spark) to analyze provenance > data using those services, then SiteToSiteProvenanceReportingTask > might be helpful. > > SiteToSiteProvenanceReportingTask can sends provenance events in JSON > format. You can send it to a NiFi input port then pass those into HDFS > by PutHDFS processor. > > If not, would you elaborate what you are trying to accomplish? > > Thanks, > Koji > > On Mon, Jun 12, 2017 at 6:25 AM, Shashi Vishwakarma > <[email protected]> wrote: > > Hi > > > > I have HDF cluster with 3 Nifi instance which lunches jobs(Hive/Spark) on > > HDP cluster. Usually nifi writes all information to different > repositories > > available on local machine. > > > > My question is - Does nifi writes any data,provenance information or does > > spilling on HDP nodes (ex. data nodes in HDP cluster) while accessing > > HDFS,Hive or Spark services ? > > > > Thanks > > > > Shashi >
