Matt - Thanks. It worked.
So I have two independent work flows. ( I am trying to be detailed so that this
archive mail may help some one achieve site to site setup)
Flow 1
Listent http — > Nifi flow (SitetoSite) (Here is asked me to
Flow 2
Inputport —> Putfile.
I tried to set secure site-to-site, I see following errors. Here both nodes are
set with https keys.
2015-10-13 16:38:41,475 ERROR [Site-to-Site Worker Thread-254]
o.a.n.r.io.socket.ssl.SSLSocketChannel
org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel@2b7eba71 Failed to
connect due to {}
javax.net.ssl.SSLHandshakeException: null cert chain
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1431) ~[na:1.8.0_45]
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
~[na:1.8.0_45]
at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
~[na:1.8.0_45]
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186) ~[na:1.8.0_45]
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) ~[na:1.8.0_45]
at
org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.performHandshake(SSLSocketChannel.java:187)
~[nifi-utils-0.3.0.jar:0.3.0]
at
org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.connect(SSLSocketChannel.java:140)
~[nifi-utils-0.3.0.jar:0.3.0]
at
org.apache.nifi.remote.SocketRemoteSiteListener$1$1.run(SocketRemoteSiteListener.java:155)
[nifi-site-to-site-0.3.0.jar:0.3.0]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
Caused by: javax.net.ssl.SSLHandshakeException: null cert chain
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[na:1.8.0_45]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[na:1.8.0_45]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304) ~[na:1.8.0_45]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:292) ~[na:1.8.0_45]
at
sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1804)
~[na:1.8.0_45]
at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:222)
~[na:1.8.0_45]
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[na:1.8.0_45]
at sun.security.ssl.Handshaker$1.run(Handshaker.java:919) ~[na:1.8.0_45]
at sun.security.ssl.Handshaker$1.run(Handshaker.java:916) ~[na:1.8.0_45]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_45]
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369)
~[na:1.8.0_45]
at
org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.performTasks(SSLSocketChannel.java:250)
~[nifi-utils-0.3.0.jar:0.3.0]
at
org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.performHandshake(SSLSocketChannel.java:236)
~[nifi-utils-0.3.0.jar:0.3.0]
... 3 common frames omitted
2015-10-13 16:38:41,475 ERROR [Site-to-Site Worker Thread-254]
o.a.nifi.remote.SocketRemoteSiteListener RemoteSiteListener Unable to accept
connection from Socket[unconnected] due to javax.net.ssl.SSLException: Inbound
closed before receiving peer's close_notify: possible truncation attack?
From: Matt Clarke <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Tuesday, October 13, 2015 at 2:51 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: site to site setup - Remore instance not configured
So you're "nifi flow" RPG should not have any connection lines coming out of
it. The input port should not have any connections coming in to it. You should
have what looks like two separate independent flows on your graph. Flow 1 would
consist of the listenHTTP connecting to the RPG directly. When draw the
connection to the RPG, the connection window will ask you which input port you
want to connect with. Flow 2 would consist of the input port you picked in flow
1 connecting directly to the putFile. The RPG will handle the load balancing
for you automatically.
Thanks,
Matt
Sent from my Verizon Wireless 4G LTE DROID
Chakrader Dewaragatla
<[email protected]<mailto:[email protected]>>
wrote:
Sweet, we need load balancer data on other end.
how do I make it work ? Here is the nifi canvas screenshot.
http://tinyurl.com/nq9fbqr
From: Matthew Clarke
<[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Tuesday, October 13, 2015 at 2:17 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: site to site setup - Remore instance not configured
Yes, the Remote Process Group (RPG, also known as the site-to-site) will talk
to the NCM which will provide connected nodes status information. It will then
load balancer data to the nodes in that cluster.
On Oct 13, 2015 5:06 PM, "Chakrader Dewaragatla"
<[email protected]<mailto:[email protected]>>
wrote:
Thanks Matthew it worked, no errors this time.
As noted below, we would like to consume http data on primary node and send it
back to cluster for processing.
I have site to site setup with http listener(on primary node) —> Input port
—> SitetoSite to NCM cluster instance —> putfile —> S3upload .
Does my setup achieve the purpose?
Data movement from http listener to input port is not working and no errors
reported.
Site-to-site admin document has limited information to understand.
Thanks,
-Chakri
From: Matthew Clarke
<[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Monday, October 12, 2015 at 4:49 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: site to site setup - Remore instance not configured
For Site to Site properties the input socket host is optional. When it is
configured it should be set the hostname or IP of the the system the NiFi
instance is running on. you must however configure an input socket port on
every instance. This includes all nodes and the NCM. If you cluster is
configured to run securely (HTTPS), you should also have input secure set to
true. Make sure you have the ports you use open in the firewalls between
systems.
On Oct 12, 2015 7:41 PM, "Chakrader Dewaragatla"
<[email protected]<mailto:[email protected]>>
wrote:
Hi – I have a use case to collect http post requests on a nifi-clustered setup.
My cluster has three nodes.
Ncm —> Slave 1 (Primary)
—> Slave 2.
I would like to setup a http listener on primary node and establish site to
site connection to the same cluster for further processing data. So http
payload receive as follows
http post —> Slave 1 (ListenHttp )— > (Site-to-site) — > NCM (put file) and
(S3 upload) (I assume this data process by two slaves nodes)
I have following error at site-to-site setup : Remote instance Is not
configured for site-to-site communications at this time.
I followed the admin doc to set the properties, as follows (on slaves).
# Site to Site properties
nifi.remote.input.socket.host=10.83.14.59 (NCM ip)
nifi.remote.input.socket.port=
nifi.remote.input.secure=false
Any thoughts?
Thanks,
-Chakri
________________________________
The information contained in this transmission may contain privileged and
confidential information. It is intended only for the use of the person(s)
named above. If you are not the intended recipient, you are hereby notified
that any review, dissemination, distribution or duplication of this
communication is strictly prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of the original
message.
________________________________
________________________________
The information contained in this transmission may contain privileged and
confidential information. It is intended only for the use of the person(s)
named above. If you are not the intended recipient, you are hereby notified
that any review, dissemination, distribution or duplication of this
communication is strictly prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of the original
message.
________________________________
________________________________
The information contained in this transmission may contain privileged and
confidential information. It is intended only for the use of the person(s)
named above. If you are not the intended recipient, you are hereby notified
that any review, dissemination, distribution or duplication of this
communication is strictly prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of the original
message.
________________________________
________________________________
The information contained in this transmission may contain privileged and
confidential information. It is intended only for the use of the person(s)
named above. If you are not the intended recipient, you are hereby notified
that any review, dissemination, distribution or duplication of this
communication is strictly prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of the original
message.
________________________________
