Joe, the ticket [1] mentions tensorflow, implying, to some extent, that option A from my email wouldn't cut the mustard.
1. https://issues.apache.org/jira/browse/NIFI-3641 On Jun 6, 2017 8:15 PM, "Joe Witt" <[email protected]> wrote: > Mike, Tony, > > This certainly sounds interesting and from reading through the > motivations/design behind it there are clearly some well thought out > reasons for it. > > For site-to-site support I can see advantages for interoperability. > For other factors it would be good to identify limitations of the > current options (raw sockets/http) so that we can clearly and > measurable improve gaps for certain use cases. Would be good to hear > any of those you have in mind. > > Outside of site-to-site it seems like it could make sense for a > processor configured with a given gRPC/proto being able to talk to > another service to share data. Is that planned as part of this effort > or is that just what the referenced JIRA was about? > > In either case this seems like one heck of an initial area to > contribute to and a good discussion point! Thanks > > Joe > > http://www.grpc.io/blog/principles > > On Tue, Jun 6, 2017 at 7:21 PM, Tony Kurc <[email protected]> wrote: > > Mike, > > I think what you're saying is you are debating two options: > > > > A) gRPC as a transport mechanism and support the deployment use cases > from > > the HTTP s2s document [1] to include using nifi-specific peer selection > in > > the client if the destination is a cluster. > > B) Building a different implementation with an additional deployment > case, > > which is sending from a client (in the diagrams as NiFi site-to-site > > client) to a cluster which isn't NiFi and delegating peer selection "to > > gRPC" which lightens what the receiving cluster would have to implement? > > > > Sound pretty close to the decision you're looking for input on? > > > > 1. > > https://cwiki.apache.org/confluence/display/NIFI/ > Support+HTTP%28S%29+as+a+transport+mechanism+for+Site- > to-Site#SupportHTTP(S)asatransportmechanismforSite- > to-Site-Deploymentexamples > > > > On Tue, Jun 6, 2017 at 2:28 PM, Michael Hogue < > [email protected]> > > wrote: > > > >> All, > >> > >> As an initial dive into nifi i elected to take a stab at NIFI-3641 > >> <https://issues.apache.org/jira/browse/NIFI-3641> (in particular, the > >> site-to-site bit), but i've got a few high level questions before taking > >> off down a path. > >> > >> Would it be more desirable to have a gRPC site-to-site mechanism or > the > >> ability to generally interact with external gRPC services, such as > tensor > >> flow, in a processor? That might help guide the approach I take in > >> addressing the issue. I'm not entirely sure it's possible to do both > with > >> the same solution since the current site-to-site API assumes the other > end > >> is a nifi. On top of that, gRPC provides a number of load balancing, > remote > >> gRPC server (i.e. peer) selection, and serialization mechanisms that the > >> current site-to-site implementation does itself. > >> > >> What i'm looking for here are some thoughts and/or guidance on a > >> recommended approach and intended goal. > >> > >> Thanks, > >> Mike > >> >
