RE: nifi processor to parse+update the current json on the fly

Hello thx Thad , joe , all For different answers. I understand how to know…. Philippe Best regards De : Thad Guidry [mailto:thadgui...@gmail.com] Envoyé : jeudi 7 avril 2016 17:32 À : users@nifi.apache.org Objet : Re: nifi processor to parse+update the current json on the fly Philippe, I would

ConvertJSONToAvro floating point / double errors

In Nifi 0.6.0 with Java 1.7, the conversion of JSON data to Avro appears to reject the floating point values (both float and double). Using the ConvertJSONToAvro processor and the following examples causes errors. Sample Avro Schema: {"name": " sample_float_avro", "type": "record", "fields":

Re: ExecuteSQL to elasticsearch

Madhu, To achieve what Arpit is saying, you can add a SplitJson processor after your ConvertAvroToJson processor, using $[*] as the JSONPath expression. That should split the array into individual flow files, then you would proceed as Arpit describes. Regards, Matt On Thu, Apr 7, 2016 at 2:18

Re: ExecuteSQL to elasticsearch

Each flowfile should contain a single document that gets written to elastic search. So if each flow file had content like > {"DateTime": "2016-04-07 17:22:00.0", "HostName": "lab1", "CounterName": > "AvgDiskSecTransfer", "InstanceName": "C:", "MetricValue": > 9.60508652497083E-4} I suspect it

Re: ExecuteSQL to elasticsearch

Can you provide a sample JSON output from your ConvertAvroToJson processor? It could help identify the location of any mapping/parser exceptions. Thanks, Matt On Thu, Apr 7, 2016 at 1:31 PM, Madhukar Thota wrote: > I am able to construct the dataflow with the

Re: ExecuteSQL to elasticsearch

I am able to construct the dataflow with the following processors ExecuteSQL--> ConvertAvrotoJson --> Elasticsearch. The problem i seeing is elasticsearch unable to index the data because of the Mapping parser exceptions. 13:27:37 EDT ERROR fc43fc28-215c-469a-9908-73d04d98d4c2

Re: NiFi Supports for Kerberized Kafka Cluster

Yep, just watch the JIRA for PR Sent from my iPhone On Apr 7, 2016, at 12:35, indus well > wrote: Thank you, Oleg. I'll be testing it out once it is available next week. Regards, Indus On Thu, Apr 7, 2016 at 10:48 AM, Oleg Zhurakousky

Re: NiFi Supports for Kerberized Kafka Cluster

Thank you, Oleg. I'll be testing it out once it is available next week. Regards, Indus On Thu, Apr 7, 2016 at 10:48 AM, Oleg Zhurakousky < ozhurakou...@hortonworks.com> wrote: > Not at the moment unless you are using HDP distribution of Kafka broker > 0.8 which has Kerberos support while ASF

Re: NiFi Supports for Kerberized Kafka Cluster

Not at the moment unless you are using HDP distribution of Kafka broker 0.8 which has Kerberos support while ASF version of 0.8 Kafka doesn't. That said, we are working on new pair of Kafka processors that will rely on Kafka 0.9 API and broker where such support is provided. It is slated for

Re: nifi processor to parse+update the current json on the fly

Here is an example of json to json conversion using Groovy with JsonSlurper. http://funnifi.blogspot.com/2016/02/executescript-json-to-json-conversion.html On Thu, Apr 7, 2016 at 11:31 AM, Thad Guidry wrote: > Philippe, > > I would encourage you to just use Groovy with

NiFi Supports for Kerberized Kafka Cluster

Hello All: Does current version of NiFi support kerberized Kafka cluster? I am getting time-out errors trying to use PutKafka and GetKafka processors. Please advise. Thanks, Indus

Re: nifi processor to parse+update the current json on the fly

Philippe As far as I know here is the state of affairs for this: 1) You can use EvaluateJSONPath and ReplaceText in a combination for some cases but it is more awkward and difficult than it should be. 2) You can use the execute script processors to write a groovy, javascript, or other type of

nifi processor to parse+update the current json on the fly

Hello I have a DF with a processor owning a json document that need to be transformed before sent to an ElasticSearch processor I have something like this $.element.attributes[0].value --> not to be changed $.element.attributes[0].type--> not to be changed ... $.element.attributes[10].value

Re: problem with putElasticsearch processor

I think Elasticsearch processor uses transport client not http. So you should use 9300(transport port) port not 9200(http port). On Wed, Apr 6, 2016 at 12:41 PM, wrote: > Hello > > My context nifi 0.6.0 on Ubuntu 14.0 > > my small use case is : > > ‘sending JSON

ExecuteSQL to elasticsearch

Friends, I am exploring ExecuteSQL processor in nifi and my goal to get sql data ingested in Elasticsearch. Can someone share or guide what's the flow looks like? Thanks in Advance.

Re: Approaches to Array in Json with Nifi?

Hi all, I got interested in EvaluateJsonPath, too. So I tried to figure out how to do that, and found there're few obstacles now. Input Json = {"company":"xyz", "rate":[0.02, 0.03, 0.04]} Then, applying following JsonPaths return: $.rate.min() => 0.2 $.rate.length() => 3 $.rate.avg() => 0.03