Another question.
To use the Remote Processor Group, on the sender side should we do the
following.
1. Create Output Port.
2. Create Remote Processor Group using the OutputPort created.
Or
1. Create the target pipeline with the input port
2. Create Remote Processor Group with the input port -> This gives the
error that it has timed out reading from the port. But we need to be sending
the data to this port rather then read from it.
Thanks
Paresh
From: Paresh Shah <[email protected]<mailto:[email protected]>>
Date: Wednesday, January 20, 2016 at 4:01 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: How to configure/start multiple Input ports.
Still don’t see the images.
Also if the Input Ports are at the root canvas, we are not able to connect it
to the ProcessorGroup. We have all our pipelines within different processor
groups.
Paresh
From: Matthew Clarke
<[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, January 20, 2016 at 2:37 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: How to configure/start multiple Input ports.
Three images attached:
On Wed, Jan 20, 2016 at 4:31 PM, Paresh Shah
<[email protected]<mailto:[email protected]>> wrote:
I cannot see the pictures. Can you please resend it with attachements.
Thanks
Paresh
From: Matthew Clarke
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
Reply-To:
"[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>"
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
Date: Wednesday, January 20, 2016 at 12:55 PM
To:
"[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>"
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
Subject: Re: How to configure/start multiple Input ports.
Paresh,
All Site-to-Site data comes across a single port connection. When
adding a RPG to the source NiFi, the URL you provide will be the URL for the
NCM of your internal NiFi cluster. You will add a a different RPG for each
cluster you are connecting to. so in your case the external facing NiFi cluster
will have 2 RPGs on the graph with 1 configured form the URL of the NCM on
internal NiFi cluster 1 and the other for the NCM of internal NiFi cluster 2.
Once a connection is established to an NCM, the NCM will communicate to the RPG
the URLs and site-to-site ports for each of the clusters connected Nodes as
well as the current load on those nodes. The NiFi with the RPG will then do a
smart load-balanced delivery of data to those nodes. On the target cluster you
will add input ports at the root level of the graph (They cannot be nested
inside of any process groups) . You can add as many uniquely named input ports
as you would like. These input ports will be exposed to the RPG on the sending
system. When you draw a connection to the RPG, you will be provided witha
pull-down selection menu of all available ports. Selecting one will complete
this connection. So all data between NiFi systems via Site-to-Site will use
the same site-to-site port but as you can see it enters the flow on the
destination system via input ports.
[Inline image 1]
So on your externally visible NiFi you would have two RPGs setup like below:
[Inline image 3]
The receiving NiFi cluster will have one or more uniquely named input ports:
[Inline image 4]
As you can see from the above two screenshots the sending system's connection
to the RPG shows that the connection as being connected to the input port
"Node-Input" on the receiving cluster.
Thanks,
Matt
On Wed, Jan 20, 2016 at 2:59 PM, Paresh Shah
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
wrote:
We are trying to see how to run our pipelines in a clustered env using
site-to-site. Our scenario is something like the following.
1. 3 different clusters.
2. One of the clusters is externally visible and will be the primary cluster.
3. All the data receivers will run on the primary node.
Each of the pipelines has a RemoteProcessorGroup( RPG ) that would in –turn
forward the data to their corresponding Input port.
Looking at the way the site-to-site configuration indicates that we are only
able to specify one port that would be started on a particular node. So the
question is how can we pin a particular RPG to it corresponding InputPort.
Any insights would be greatly appreciated.
Thanks
Paresh
________________________________
The information contained in this transmission may contain privileged and
confidential information. It is intended only for the use of the person(s)
named above. If you are not the intended recipient, you are hereby notified
that any review, dissemination, distribution or duplication of this
communication is strictly prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of the original
message.
________________________________
________________________________
The information contained in this transmission may contain privileged and
confidential information. It is intended only for the use of the person(s)
named above. If you are not the intended recipient, you are hereby notified
that any review, dissemination, distribution or duplication of this
communication is strictly prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of the original
message.
________________________________
________________________________
The information contained in this transmission may contain privileged and
confidential information. It is intended only for the use of the person(s)
named above. If you are not the intended recipient, you are hereby notified
that any review, dissemination, distribution or duplication of this
communication is strictly prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of the original
message.
________________________________
