:40 PM
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>"
mailto:users@nifi.apache.org>>
Subject: Re: Nifi JSON event storage in HDFS
I am exploring to use kite processor to store data into Hadoop. I hope this
lets me change storage engine form hdfs to hive to hbase later. Si
t;> From: Mike Harding
>> Reply-To: "users@nifi.apache.org"
>> Date: Wednesday, 2 March 2016 at 10:33
>> To: "users@nifi.apache.org"
>> Subject: Nifi JSON event storage in HDFS
>>
>> Hi All,
>>
>> I currently have a sm
skin this particular cat
>> though,
>> Conrad
>>
>> From: Mike Harding
>> Reply-To: "users@nifi.apache.org"
>> Date: Wednesday, 2 March 2016 at 10:33
>> To: "users@nifi.apache.org"
>> Subject: Nifi JSON event storage in HDFS
>
ifi.apache.org>"
mailto:users@nifi.apache.org>>
Date: Wednesday, 2 March 2016 at 10:54
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>"
mailto:users@nifi.apache.org>>
Subject: Re: Nifi JSON event storage in HDFS
Hi Conrad,
Thanks for the heads up, I will invest
I've been doing a lot of this recently into both hive and spark.
One thing that will make life a lot easier is to use the JSON record file
format, this is essentially just a JSON document per line of a text file, which
means you can use nifi's MergeContent processor to handle batching into hdfs.
ting existing data too.
> HTH – I’m sure there are a ton of other ways to skin this particular cat
> though,
> Conrad
>
> From: Mike Harding
> Reply-To: "users@nifi.apache.org"
> Date: Wednesday, 2 March 2016 at 10:33
> To: "users@nifi.apache.org"
>
ednesday, 2 March 2016 at 10:33
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>"
mailto:users@nifi.apache.org>>
Subject: Nifi JSON event storage in HDFS
Hi All,
I currently have a small hadoop cluster running with HDFS and Hive. My ultimate
goal is to leverage Ni
Hi All,
I currently have a small hadoop cluster running with HDFS and Hive. My
ultimate goal is to leverage NiFi's ingestion and flow capabilities to
store real-time external JSON formatted event data.
What I am unclear about is what the best strategy/design is for storing
FlowFile data (i.e. JSO
