Paresh,
All Site-to-Site data comes across a single port connection. When
adding a RPG to the source NiFi, the URL you provide will be the URL for
the NCM of your internal NiFi cluster. You will add a a different RPG for
each cluster you are connecting to. so in your case the external facing
NiFi cluster will have 2 RPGs on the graph with 1 configured form the URL
of the NCM on internal NiFi cluster 1 and the other for the NCM of internal
NiFi cluster 2. Once a connection is established to an NCM, the NCM will
communicate to the RPG the URLs and site-to-site ports for each of the
clusters connected Nodes as well as the current load on those nodes. The
NiFi with the RPG will then do a smart load-balanced delivery of data to
those nodes. On the target cluster you will add input ports at the root
level of the graph (They cannot be nested inside of any process groups) .
You can add as many uniquely named input ports as you would like. These
input ports will be exposed to the RPG on the sending system. When you
draw a connection to the RPG, you will be provided witha pull-down
selection menu of all available ports. Selecting one will complete this
connection. So all data between NiFi systems via Site-to-Site will use the
same site-to-site port but as you can see it enters the flow on the
destination system via input ports.
[image: Inline image 1]
So on your externally visible NiFi you would have two RPGs setup like below:
[image: Inline image 3]
The receiving NiFi cluster will have one or more uniquely named input
ports:
[image: Inline image 4]
As you can see from the above two screenshots the sending system's
connection to the RPG shows that the connection as being connected to the
input port "Node-Input" on the receiving cluster.
Thanks,
Matt
On Wed, Jan 20, 2016 at 2:59 PM, Paresh Shah <[email protected]>
wrote:
>
> We are trying to see how to run our pipelines in a clustered env using
> site-to-site. Our scenario is something like the following.
>
> 1. 3 different clusters.
> 2. One of the clusters is externally visible and will be the primary
> cluster.
> 3. All the data receivers will run on the primary node.
>
> Each of the pipelines has a RemoteProcessorGroup( RPG ) that would in
> –turn forward the data to their corresponding Input port.
>
> Looking at the way the site-to-site configuration indicates that we are
> only able to specify one port that would be started on a particular node.
> So the question is how can we pin a particular RPG to it corresponding
> InputPort.
>
> Any insights would be greatly appreciated.
>
> Thanks
> Paresh
> ________________________________
> The information contained in this transmission may contain privileged and
> confidential information. It is intended only for the use of the person(s)
> named above. If you are not the intended recipient, you are hereby notified
> that any review, dissemination, distribution or duplication of this
> communication is strictly prohibited. If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies
> of the original message.
> ________________________________
>
