Re: VolatileContentRepository

2017-08-10 Thread Pierre Villard
Hi Margus, I believe that your memory settings are enough. Giving more memory will likely increase duration of garbage collections and won't increase your performances. Others on this mailing list will probably be able to give better recommendations on this one. Also, keep in mind that volatile

Re: VolatileContentRepository

2017-08-10 Thread BD International
Pierre, Just a clarification on what you said below the potential data loss on nifi shutdown would only be on provenance information right? Or would the data loss affect other repos? Thanks Brian On 10 Aug 2017 8:57 am, "Pierre Villard" wrote: Hi Margus, I

Re: VolatileContentRepository

2017-08-10 Thread Pierre Villard
Hi Brian, The question is about the content volatile repository. If you are only using the provenance volatile repository (not the content one), then only the provenance data would be lost in case of NiFi shutdown, you're right. Pierre 2017-08-10 10:03 GMT+02:00 BD International

Re: Pivot: Columns to Rows

2017-08-10 Thread Andy LoPresto
You can use the ExecuteScript processor to have Python or another language perform column transposition. Here is a more complete answer with some caveats and further discussion [1]. [1] https://stackoverflow.com/a/42639228/70465 Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP

ConvertJSONToAvro not working in NIFi despite successful with avro-tools validation

2017-08-10 Thread Manish Mehndiratta
Hi - Problem statement: I have a XML file which is converted to JSON. This JSON is sent to ConvertJSONToAvro where it fails. This same JSON output when run against same .asvc file in avro-tools works fine and generates the output. Can someone please look at the attached artifacts and help.

custom bulk export teradata processor

2017-08-10 Thread Margarita Uk
Hello, Has anyone found a way to write a custom processor that is able to leverage Teradata's bulk export utility? We would like to leverage this rather than running an "executeSQL" processor as it is more efficient against the source database system. Thanks, Margarita Uk

Re: Pivot: Columns to Rows

2017-08-10 Thread mayank rathi
Thanks Andy :) On Thu, Aug 10, 2017 at 1:13 PM, Andy LoPresto wrote: > You can use the ExecuteScript processor to have Python or another language > perform column transposition. Here is a more complete answer with some > caveats and further discussion [1]. > > [1]

Re: How to add headers to CSV file

2017-08-10 Thread Carl Berndt
Hi, It is possible to use ConvertRecord with a CSVRecordSetWriter. There is a property "include header line" true/false on that processor. hope that helps. Carl On Wed, Aug 2, 2017 at 2:36 AM, mayank rathi wrote: > Hello All, > > I am working on a NiFi flow to read data

Pivot: Columns to Rows

2017-08-10 Thread mayank rathi
Hello All, How can I pivot record based data from columns to rows. Here is an example of what I want to achieve: Input: A,B,C M,N,O Output: A,B A,C M,N M,O Thanks!! -- NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and

Re: Moving nifi dataflows through environments

2017-08-10 Thread Joe Witt
Brian, There have been issues in the past with regard to template importing and it causing duplicate controller service entries. But these should be much improved. What version are you using? Also, the registry component that is now being worked on should help this quite a bit. The