Re: Reading flowfile in a stream callback

2017-11-02 Thread Andy LoPresto
James, The Python API should be the same as the Java FlowFile.java interface [1]. Matt Burgess’ blog has a good post about using Jython to do flowfile content manipulation. Something like: flowFile = session.get() if (flowFile != None): flowFile = session.write(flowFile,PyStreamCallback())

Reading flowfile in a stream callback

2017-11-02 Thread James McMahon
In python, I can use the requests library to post content something like htis: import requests url="https://abc.test.org; files={'file':open('/somedir/myfile.txt','rb')} r = requests.post(url,files=files) If I am in a python stream callback, how can I read the flowfile payload in the same way

Issue with Executescript

2017-11-02 Thread N, Vyshali
Hi, I'm using the executescript process to generate some fake data using "Faker" package and replacing it in the original data.I have attached the script for your reference. import java.io from org.apache.commons.io import IOUtils from java.nio.charset import StandardCharsets from

SplitAvro fails to split data

2017-11-02 Thread Pradip
I have a NiFi flow that queries a relational Table that has a latestUpdate date timestamp column  and forwards data in that table that was  inserted/modified in the last 10 min sends it to a JMS queue. The flow is something like thisExecuteSQL --> splitAvro --> convertAvroToJson --> some more

What would be a good way to build NoSQL entity from SQL data?

2017-11-02 Thread Eric Chaves
Hi fellows, I'm planning a flow to feed data from a relational database into elasticsearch and would like to ask for some advice. The database has an entity Person with 1:N relation to Email and Phone (email and phone are objects, not just scalar values). The flow needs to feed ES with JSON

Re: NiFi Processor Lifecycle Diagram

2017-11-02 Thread Pierre Villard
Hi, There is actually a JIRA for that [1]. Feel free to add attachments to this JIRA or submit a PR with your work. I'm sure someone will review and get it merged into our documentation! [1] https://issues.apache.org/jira/browse/NIFI-1345 Thanks 2017-11-02 15:51 GMT+01:00 Ryan H

Re: NiFi Processor Lifecycle Diagram

2017-11-02 Thread Ryan H
Thanks for links. I've whiteboarded some stuff, but nothing serious. I'll be sure to share if I get something going. Ryan On Mon, Oct 30, 2017 at 7:18 PM, Andy LoPresto wrote: > Hi Ryan, > > I am not aware of a diagram, but the best resource for the component >

HandleHttpRequest

2017-11-02 Thread Juan Sequeiros
Hello all, Is it considered a best practice to leave the thread count on HanddleHttpRequest to 1? Assuming the JETTY server is handling requests? are we making things worst by increasing that? We are moving towards using HanddleHttpRequest versus ListenHttp but we are struggling with