Hi Conrad,
I think there are a couple of things at play here...
One is that the SSL properties need to be set on the
SiteToSiteClientBuilder, rather than through system properties. There
should be methods to set the keystore and other values.
In a secured NiFi instance, the certificate you are authenticating with
(the keystore used by the s2s client) would need to have an account in
NiFi, and would need to have access to the output port.
If you attempt to make a request with that cert, and then you go into the
NiFi UI as another user, you should be able to go into the accounts section
(top right) and approve the account for that certificate.
Then if you stop your output port, right-click and Configure... and from
the Access Controls tab started typing the DN from your cert and add that
user to the Allowed Users list. Hit Apply and started the port again.
We probably need to document this better, or write up an article about it
somewhere.
Let us know if its still not working.
Thanks,
Bryan
On Thu, May 19, 2016 at 11:54 AM, Conrad Crampton <
conrad.cramp...@secdata.com> wrote:
> Hi,
> Tried following a couple of blog posts about this [1], [2], but neither of
> these refer to using NiFi in clustered environment with SSL and I suspect
> this is where I am hitting problems (but don’t know where).
>
> The blogs state that using an output port (in the root process group I.e.
> on main canvas) which I have done and tried to connect thus..
>
> System.setProperty("javax.net.ssl.keyStore", "/spark-processor.jks");
> System.setProperty("javax.net.ssl.keyStorePassword", *“******");
> System.setProperty("javax.net.ssl.trustStore", *“*/cacerts.jks");
>
> SiteToSiteClientConfig config = new SiteToSiteClient.Builder()
> .url("https://yarn-cm1.mis-cds.local:9090/nifi";)
> .portName("Spark test out")
> .buildConfig();
>
> SparkConf sparkConf = new SparkConf().setMaster("local[2]").setAppName("NiFi
> Spark Log Processor");
> JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, new
> Duration(5000));
> JavaReceiverInputDStream<NiFiDataPacket> packetStream =
> jssc.receiverStream(new NiFiReceiver(config, StorageLevel.MEMORY_ONLY()));
>
> JavaDStream text = packetStream.map(dataPacket -> new
> String(dataPacket.getContent(), StandardCharsets.UTF_8));
> text.print();
> jssc.start();
> jssc.awaitTermination();
>
> The error I am getting is
>
> 16/05/19 16:39:03 WARN ReceiverSupervisorImpl: Restarting receiver with
> delay 2000 ms: Failed to receive data from NiFi
> java.io.IOException: Server returned HTTP response code: 401 for URL:
> https://yarn-cm1.mis-cds.local:9090/nifi-api/controller
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1889)
> at
> sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1884)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1883)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1456)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)
> at
> sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
> at
> org.apache.nifi.remote.util.NiFiRestApiUtil.getController(NiFiRestApiUtil.java:69)
> at
> org.apache.nifi.remote.client.socket.EndpointConnectionPool.refreshRemoteInfo(EndpointConnectionPool.java:891)
> at
> org.apache.nifi.remote.client.socket.EndpointConnectionPool.getPortIdentifier(EndpointConnectionPool.java:878)
> at
> org.apache.nifi.remote.client.socket.EndpointConnectionPool.getOutputPortIdentifier(EndpointConnectionPool.java:862)
> at
> org.apache.nifi.remote.client.socket.SocketClient.getPortIdentifier(SocketClient.java:81)
> at
> org.apache.nifi.remote.client.socket.SocketClient.createTransaction(SocketClient.java:123)
> at
> org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable.run(NiFiReceiver.java:149)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Server returned HTTP response code: 401
> for URL: https://yarn-cm1.mis-cds.local:9090/nifi-api/controller
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1839)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at
> sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
> at
> org.apache.nifi.remote.util.NiFiRestApiUtil.getController(NiFiRestApiUtil.java:66)
> ... 7 more
>
> Any pointers would be helpful in getting this working. I don’t know if I
> have to set up a remote process group with the output port (not sure how
> this works), or what. When I go to
> https://yarn-cm1.mis-cds.local:9090/nifi-api/controller in the browser, I
> get an access denied error.
> I have created keystore and signed by the RootCA used to sign all the self
> signed certs for the cluster.
>
> Running 0.6.1, 6 node cluster.
>
> Thanks
> Conrad
>
> [1[ -
> https://community.hortonworks.com/articles/12708/nifi-feeding-data-to-spark-streaming.html
> [2] - https://blogs.apache.org/nifi/entry/stream_processing_nifi_and_spark
>
>
> SecureData, combating cyber threats
>
> ------------------------------
>
> The information contained in this message or any of its attachments may be
> privileged and confidential and intended for the exclusive use of the
> intended recipient. If you are not the intended recipient any disclosure,
> reproduction, distribution or other dissemination or use of this
> communications is strictly prohibited. The views expressed in this email
> are those of the individual and not necessarily of SecureData Europe Ltd.
> Any prices quoted are only valid if followed up by a formal written quote.
>
> SecureData Europe Limited. Registered in England & Wales 04365896.
> Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
> Maidstone, Kent, ME16 9NT
>
