Each flowfile should contain a single document that gets written to elastic search.
So if each flow file had content like > {"DateTime": "2016-04-07 17:22:00.0", "HostName": "lab1", "CounterName": > "AvgDiskSecTransfer", "InstanceName": "C:", "MetricValue": > 9.60508652497083E-4} I suspect it will go through. Put Elastic search supports batching where multiple flow files will be batched in a single transaction but each flow file should only have one document. -- Arpit > On Apr 7, 2016, at 10:40 AM, Madhukar Thota <madhukar.th...@gmail.com> wrote: > > Here is the sample JSON output from ConvertAvroToJson Processor: > > [{"DateTime": "2016-04-07 17:22:00.0", "HostName": "lab1", "CounterName": > "AvgDiskSecTransfer", "InstanceName": "C:", "MetricValue": > 9.60508652497083E-4}, > {"DateTime": "2016-04-07 17:22:00.0", "HostName": "lab1", "CounterName": > "AvgDiskSecTransfer", "InstanceName": "D:", "MetricValue": > 2.3995189985726E-4}, > {"DateTime": "2016-04-07 17:22:00.0", "HostName": "lab1", "CounterName": > "AvgDiskSecTransfer", "InstanceName": "F:", "MetricValue": > 2.17140579479747E-4}] > > > On Thu, Apr 7, 2016 at 1:34 PM, Matt Burgess <mattyb...@gmail.com > <mailto:mattyb...@gmail.com>> wrote: > Can you provide a sample JSON output from your ConvertAvroToJson processor? > It could help identify the location of any mapping/parser exceptions. > > Thanks, > Matt > > On Thu, Apr 7, 2016 at 1:31 PM, Madhukar Thota <madhukar.th...@gmail.com > <mailto:madhukar.th...@gmail.com>> wrote: > I am able to construct the dataflow with the following processors > > ExecuteSQL--> ConvertAvrotoJson --> Elasticsearch. > > The problem i seeing is elasticsearch unable to index the data because of the > Mapping parser exceptions. > > 13:27:37 EDTERRORfc43fc28-215c-469a-9908-73d04d98d4c2 > PutElasticsearch[id=fc43fc28-215c-469a-9908-73d04d98d4c2] Failed to insert > StandardFlowFileRecord[uuid=02af852b-bdf7-452f-a320-b23753c13389,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1460050039787-4636, > container=default, section=540], offset=0, > length=697677],offset=0,name=1386348391725491,size=697677] into Elasticsearch > due to MapperParsingException[failed to parse]; nested: > NotSerializableExceptionWrapper[not_x_content_exception: Compressor detection > can only be called on some xcontent bytes or compressed xcontent bytes];, > transferring to failure > > > Am i doing anything wrong here or do i need extra processor to convert into > right format what elasticsearch understands? > > > > On Thu, Apr 7, 2016 at 7:49 AM, Madhukar Thota <madhukar.th...@gmail.com > <mailto:madhukar.th...@gmail.com>> wrote: > Friends, > > I am exploring ExecuteSQL processor in nifi and my goal to get sql data > ingested in Elasticsearch. > > Can someone share or guide what's the flow looks like? > > > Thanks in Advance. > > >