Matt G: Understood. It turns out that the spark receiver uses our client library which does exactly what you suggest. We should probably put together a blog/writeup/howto for clients that want to push or pull from nifi using this direct mechanism. We've always focused on supporting whatever protocols a given client and producer wants to use so we've never really emphasized this. But both models make sense.
Thanks Joe On Fri, May 15, 2015 at 9:34 AM, Matthew Gaulin <[email protected]> wrote: > I think it would be awesome to have a java driver for sending and receiving > data to/from a nifi flow directly, similar to the nifi-spark-receiver. > Then it would make it much easier to directly link with any java > application or something like storm. Currently our team is using kafka to > bridge that gap, but it would be great if that could be streamlined by > going directly to and from a nifi flow. > > Thanks! > Matt > > On Thu, May 14, 2015 at 10:58 PM Sean Busbey <[email protected]> wrote: > >> how easy / hard is it to dynamically add and remove nodes from cluster mode >> right now? Being able to scale elastically would make gaining ground in >> cloud deployments way easier. >> >> On Thu, May 14, 2015 at 9:29 PM, Joe Witt <[email protected]> wrote: >> >> > All, >> > >> > With the 0.1.0 release hopefully soon available it is time to turn >> > towards the next release or so and get a sense of what we should focus >> > on. >> > >> > This should include both 0.1.x but also 0.2.0. >> > >> > Obviously first and foremost we need to work the existing PRs and >> > patches that exist. >> > >> > Beyond that we have slated the following so far: >> > >> > 0.1.1 >> > >> > >> https://issues.apache.org/jira/browse/NIFI/fixforversion/12332286/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel >> > >> > 0.2.0 >> > >> > >> https://issues.apache.org/jira/browse/NIFI/fixforversion/12329653/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel >> > >> > But what i'd like to throw out for general discussion is what are some >> > of the bigger thematic things we should focus on? Things which will >> > help further with community growth and utility of the product for >> > those folks using it now? >> > >> > For example: >> > I'd like to see us start digging into the cluster robustness issues >> > (HA cluster manager w/ legit leader election, etc..). But there are >> > other things as well that may be more important sooner. >> > >> > Please share your thoughts as this is a great time to effect those >> > releases. >> > >> > Thanks >> > Joe >> > >> >> >> >> -- >> Sean >>
