Hello Shahzad, Unfortunately the "stream" functionality of pushshift.io doesn't fit into any current NiFi processor. Processors work by having an "OnTrigger" method that is used to create FlowFiles with each call. This works nicely for aspects of the pushshit.io api like "https://api.pushshift.io/reddit/search?q=Einstein&limit=100"; where it returns a single "unit" of information with each http request. If you are able to get the same information you need using the base "api" call for pushshift instead of "stream that would work best.
Else you may be able to create a custom processor around your java code although it may be pretty difficult. You would need to translate the stream into chunks of information that would be put into the contents of FlowFiles and routed to a relationship using Session.Transter. For more information on creating a custom processor check out the developer guide: https://nifi.apache.org/developer-guide.html. Do either of those help or is a general processor that streams over HTTP necessary? Joe - - - - - - Joseph Percivall linkedin.com/in/Percivall e: [email protected] On Friday, January 8, 2016 11:16 AM, Shahzad K <[email protected]> wrote: Hi My name is Shahzad Karamat, i am trying to read some tweets from http://stream.pushshift.io/ <http://stream.pushshift.io/> into nifi. I am using a mac and can read the stream using curl -i 'http://stream.pushshift.io/?subreddit=askreddit' I can get the stream into my terminal and i have also developed system to read this by using Java code. The question is: The strings i read from http://stream.pushshift.io/ <http://stream.pushshift.io/> by using java, how can i make flowFile of this stream to transfer it to a certain relation? Regards Shahzad K
