Brian, As part of the site-to-site communication, the client node (i.e., the NiFi node establishing the connection, the one with the Remote Process Group) will connect and request a list of "peers" (i.e., a list of nodes in the cluster). If you are running a very new version of 0.3.0-SNAPSHOT, you will be able to configure what value is returned by that node by setting a value for the "nifi.remote.input.socket.host" property in the nifi.properties file. If what you are running is a bit older, or if that property is not set, what gets returned is the hostname that is returned by Java.
Generally, when you see this error, it is because the hostname returned by the remote nifi instance is not in your /etc/hosts file on the client. If you run the "hostname" command on the remote instance, can you verify that the value returned is in /etc/hosts on the client instance? Thanks -Mark ---------------------------------------- > Date: Mon, 24 Aug 2015 21:41:19 -0400 > Subject: Re: NiFi Site-to-Site > From: [email protected] > To: [email protected] > > Hey Mark, > > Thanks for the reply and the references. I've taken a look through, and it > seems like everything should be set up properly. I am seeing the following > exception in the NiFi Consumer nifi-app.log. I can ping and wget to the > NiFi Producer instance fine from the Consumer box, but Site-to-Site is > still giving some trouble. That's why I started looking at AWS Security > Groups and Firewall settings. Does this stack trace ring a bell with > anything? > > 2015-08-25 01:34:30,709 WARN [Timer-Driven Process Thread-8] > o.a.n.c.t.ContinuallyRunConnectableTask > RemoteGroupPort[name=data-output,target=http://nifi-producer-host:8080/nifi] > Administratively Pausing for 10 seconds due to processing failure: > java.lang.RuntimeException: java.nio.channels.UnresolvedAddressException > java.lang.RuntimeException: java.nio.channels.UnresolvedAddressException > at > org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:234) > ~[nifi-framework-core-api-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:80) > [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:40) > [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_45] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_45] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_45] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_45] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_45] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_45] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] > Caused by: java.nio.channels.UnresolvedAddressException: null > at sun.nio.ch.Net.checkAddress(Net.java:123) ~[na:1.8.0_45] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) > ~[na:1.8.0_45] > at java.nio.channels.SocketChannel.open(SocketChannel.java:189) > ~[na:1.8.0_45] > at > org.apache.nifi.remote.client.socket.EndpointConnectionPool.establishSiteToSiteConnection(EndpointConnectionPool.java:708) > ~[nifi-site-to-site-client-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > org.apache.nifi.remote.client.socket.EndpointConnectionPool.establishSiteToSiteConnection(EndpointConnectionPool.java:682) > ~[nifi-site-to-site-client-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > org.apache.nifi.remote.client.socket.EndpointConnectionPool.getEndpointConnection(EndpointConnectionPool.java:300) > ~[nifi-site-to-site-client-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > org.apache.nifi.remote.client.socket.SocketClient.createTransaction(SocketClient.java:129) > ~[nifi-site-to-site-client-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > org.apache.nifi.remote.StandardRemoteGroupPort.onTrigger(StandardRemoteGroupPort.java:167) > ~[nifi-site-to-site-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > at > org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:227) > ~[nifi-framework-core-api-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT] > ... 10 common frames omitted > > Thanks again, > Brian > > On Mon, Aug 24, 2015 at 8:05 PM, Mark Payne <[email protected]> wrote: > >> Hey Brian, >> >> The latest version of the User Guide has instructions on setting up >> site-to-site, but that version >> hasn't yet been posted to the website, I don't believe. If you click the >> "help" menu in the top-right >> corner of your NiFi instance, and go to the User Guide from there, you >> should have the up-to-date >> guide there. >> >> If there's anything missing, please let us know so that we can update the >> guide. >> >> For convenience, I have pasted the text of that section below: >> >> >> In order to communicate with a remote NiFi instance via Site-to-Site, >> simply drag a Remote Process Group onto the graph and enter the URL of the >> remote NiFi instance (for more information on the components of a Remote >> Process Group, see Remote Process Group Transmission section of this >> guide.) The URL is the same URL you would use to go to that instance’s User >> Interface. At that point, you can drag a connection to or from the Remote >> Process Group in the same way you would drag a connection to or from a >> Processor or a local Process Group. When you drag the connection, you will >> have a chance to choose which Port to connect to. Note that it may take up >> to one minute for the Remote Process Group to determine which ports are >> available. >> >> If the connection is dragged starting from the Remote Process Group, the >> ports shown will be the Output Ports of the remote group, as this indicates >> that you will be pulling data from the remote instance. If the connection >> instead ends on the Remote Process Group, the ports shown will be the Input >> Ports of the remote group, as this implies that you will be pushing data to >> the remote instance. >> >> Note: if the remote instance is configured to use secure data >> transmission, you will see only ports that you are authorized to >> communicate with. For information on configuring NiFi to run securely, see >> the Admin Guide. >> >> In order to allow another NiFi instance to push data to your local >> instance, you can simply drag an Input Port onto the Root Process Group of >> your graph. After entering a name for the port, it will be added to your >> flow. You can now right-click on the Input Port and choose Configure in >> order to adjust the name and the number of concurrent tasks that are used >> for the port. If Site-to-Site is configured to run securely, you will also >> be given the ability to adjust who has access to the port. If secure, only >> those who have been granted access to communicate with the port will be >> able to see that the port exists. >> >> After being given access to a particular port, in order to see that port, >> the operator of a remote NiFi instance may need to right-click on their >> Remote Process Group and choose to "Refresh" the flow. >> >> Similar to an Input Port, a DataFlow Manager may choose to add an Output >> Port to the Root Process Group. The Output Port allows an authorized NiFi >> instance to remotely connect to your instance and pull data from the Output >> Port. Configuring the Output Port will again allow the DFM to control how >> many concurrent tasks are allowed, as well as which NiFi instances are >> authorized to pull data from the instance being configured. >> >> In addition to other instances of NiFi, some other applications may use a >> Site-to-Site client in order to push data to or receive data from a NiFi >> instance. For example, NiFi provides an Apache Storm spout and an Apache >> Spark Receiver that are able to pull data from NiFi’s Root Group Output >> Ports. >> >> If your instance of NiFi is running securely, the first time that a client >> establishes a connection to your instance, the client will be forbidden and >> a request for an account for that client will automatically be generated. >> The client will need to be granted the NiFi role in order to communicate >> via Site-to-Site. For more information on managing user accounts, see >> theControlling Levels of Access section of the Admin Guide. >> >> For information on how to enable and configure Site-to-Site on a NiFi >> instance, see the Site-to-Site Properties section of the Admin Guide. >> >> Thanks >> -Mark >> >> >> ---------------------------------------- >>> Date: Mon, 24 Aug 2015 19:36:55 -0400 >>> Subject: NiFi Site-to-Site >>> From: [email protected] >>> To: [email protected] >>> >>> Is there any step-by-step guide to setting up NiFi Site-to-Site with >> Remote >>> Process Groups? Any details on what port range(s) needs to be available? >>> >>> My setup: NiFi Producer provides data on output port "data-output" to be >>> picked up by NiFi Consumer. >>> >>> While trying to get a simple setup running, I set >>> "nifi.remote.input.secure" to false and "nifi.remote.input.socket.port" >> to >>> 8082 in NiFi Producer (also added TCP ingress to the firewall). The >> Remote >>> Process Group in NiFi Consumer has the "data-output" port turned on and >> is >>> simply routing to LogAttribute for verification. >>> >>> Logs in NiFi Producer indicate that it's periodically establishing a >> socket >>> connection with NiFi Consumer over ports in the 54000 - 55999 range, but >> no >>> data actually comes across to NiFi Consumer. I added opened NiFi >>> Consumer's firewall for ingress from 54000 - 55999. >>> >>> Eventually, we'll need to get it over SSL and lock down the firewall to a >>> minimum required range, but I'm hoping to prove it out first. >>> >>> Any guidance would be greatly appreciated! >>> >>> Thanks, >>> Brian >> >> > > > > -- > Brian Ghigiarelli > 570-878-9139
