RE: Generate flowfiles from flowfile content

2015-09-24 Thread David Klim
ExtractText did the job! Thank you very much! :-) > Date: Wed, 23 Sep 2015 16:05:44 -0700 > Subject: Re: Generate flowfiles from flowfile content > From: joe.w...@gmail.com > To: users@nifi.apache.org > > Bryan - you may be right that ExtractText will be the right play once > splitjson is done

.zip archive or expression language

2015-09-24 Thread Devin Pinkston
Hello, I'm using NiFi to pull down some .zip archives off the web and want to extract them and do some processing, however it looks like i can't decompress a .zip archive. Do i need to use the ExecuteProcess processor in order to handle this with an "unzip"? If so, is there a way through

Array into MongoDB

2015-09-24 Thread Adam Williams
I have an array of JSON object I am trying to put into Mongo, but I keep hitting this on the PutMongo processor: ERROR [Timer-Driven Process Thread-1] o.a.nifi.processors.mongodb.PutMongo PutMongo[id=c576f8cc-6e21-4881-a7cd-6e3881838a91] Failed to insert

Re: Array into MongoDB

2015-09-24 Thread Aldrin Piri
Bryan is correct about the backing library reading everything into memory to do the evaluation. Might I ask what the expression you are using? On Thu, Sep 24, 2015 at 6:44 PM, Adam Williams wrote: > I tried it even with 6GB and no luck. It's receiving the

Re: removing adding node to cluster

2015-09-24 Thread Mark Payne
Jeff, In this respect, it is also considering your templates part of your flow. So there's a good chance you've got a conflicting set of templates. If you blow away your flow.xml.gz and all templates you should be okay. Templates are stored in conf/templates, if you haven't changed the

Re: Array into MongoDB

2015-09-24 Thread Bryan Bende
One other thing I thought of... I think the JSON processors read the entire FlowFile content into memory to do the splitting/evaluating, so I wonder if you are running into a memory issue with a 180MB JSON file. Are you running with the default configuration of 512mb set in conf/bootstrap.conf ?

Re: Array into MongoDB

2015-09-24 Thread Jeff
I’m having a very similar problem. The process picks up the file, a custom processor does it’s thing but no data is sent out. > On Sep 24, 2015, at 5:56 PM, Adam Williams wrote: > > For JsonSplit i am using just "$" to try and get the array into individual