Ryan,
We definitely want to get to the point that we have the ability to
load-balance the data in a connection across a cluster. But yes, that is
a fairly large undertaking.
I can definitely appreciate that it would be more convenient to add a
port in the middle of a group, but I would shy away implementing
something like that as a stop-gap when the load-balanced connection is
the true desire. This stop-gap really would be quite a lot of work, as
well, and I think would introduce more confusion.
Thanks
-Mark
------ Original Message ------
From: "Ryan Blue" <[email protected]>
To: [email protected]
Sent: 4/6/2015 11:32:12 AM
Subject: Re: Site to Site not working within process groups
On 04/03/2015 03:36 PM, Matt Gilman wrote:
Ricky,
What your seeing is by design. While the approach can be limiting,
especially if your looking to expose an [in|out]put port remotely in a
sub group, it was done for consistency and simplicity. Groups can have
input and output ports. This facilitates data flow into and out of the
groups. When this is done at the root level, it allows us to abstract
a NiFi instance as a group to a remote NiFi.
Matt Gilman
I think the problem is that sometimes you want a process group to be
able to use the trick where you send to a local "remote" input port to
load balance. It would be great to be able to hide that detail within a
process group, but the reuse of ports for both purposes prevents it.
Could we add an option to select whether the port is for a process
group or should listen for remote connections? That seems like an easy
way to solve the problem, though I think adding an option to load
balance a connection in cluster mode would solve the problem more
cleanly. But that would be more work, right?
rb
-- Ryan Blue
Software Engineer
Cloudera, Inc.
