Hi Conrad, Sorry this has been so challenging to setup. After trying it out myself, I believe the problem you ran into when you didn't set the System properties is actually a legit bug in the SiteToSiteClient... I wrote it up in this JIRA [1], but the short answer is that it never uses those properties to create an SSLContext and ends up trying to make a normal connection to the https end-point, and thus ends up failing.
I made some quick code changes to work around the above issue, and eventually got it working using Storm, since I don't have spark streaming setup. Here is what I did... In conf/nifi.properties I set the following: # Site to Site properties nifi.remote.input.socket.host= nifi.remote.input.socket.port=8088 nifi.remote.input.secure=true # web properties # nifi.web.war.directory=./lib nifi.web.http.host= nifi.web.http.port=8080 nifi.web.https.host= nifi.web.https.port=8443 nifi.web.jetty.working.directory=./work/jetty nifi.web.jetty.threads=200 # security properties # nifi.sensitive.props.key= nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL nifi.sensitive.props.provider=BC nifi.security.keystore=/path/to/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/localhost-ks.jks nifi.security.keystoreType=JKS nifi.security.keystorePasswd=localtest nifi.security.keyPasswd=localtest nifi.security.truststore=/path/to/nifi//nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/localhost-ts.jks nifi.security.truststoreType=JKS nifi.security.truststorePasswd=localtest I started NiFi and used the unsecure url (http://localhost:8080/nifi) to create a flow with GenerateFlowFile -> Output Port named "Data for Storm". There is an example Storm topology that is part of the code base [2], so I started with that, and modified the SiteToSiteClientConfig: final SiteToSiteClientConfig inputConfig = new SiteToSiteClient.Builder() .url("https://localhost:8443/nifi";) .portName("Data for Storm") .keystoreFilename("/path/to/nifi//nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/localhost-ks.jks") .keystoreType(KeystoreType.JKS) .keystorePass("localtest") .truststoreFilename("/path/to/nifi//nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/localhost-ts.jks") .truststoreType(KeystoreType.JKS) .truststorePass("localtest") .buildConfig(); Now of course setting those properties only worked because of local changes I made, but after that I got a 401 Unauthorized when I ran the topology, which I think was where you were originally at. I went back into the unsecure url and checked the users section and didn't see anything, so I think I was incorrect that it automatically creates a pending account. I then put that localhost cert into my browser (I already had it as p12 from something else) and I went to https://localhost:8443/nifi and it prompted for the account request and I submitted it. Went back to the unsecure UI and approved the account with role NiFi, then went to the Output Port and gave access to the localhost user. After that it was working... I think since you were already at the point of getting the 401, if you can just get the account created for that certificate and the access controls on the ports, then it should probably work using the System properties as a work around for now, but not totally sure. Again, sorry for all the confusion, definitely planning to address the JIRA soon. -Bryan [1] https://issues.apache.org/jira/browse/NIFI-1907 [2] https://github.com/apache/nifi/blob/e12a79ea929a222a93fd64bfc63382441e31060f/nifi-external/nifi-storm-spout/src/test/java/org/apache/nifi/storm/NiFiStormTopology.java On Fri, May 20, 2016 at 4:16 AM, Conrad Crampton < [email protected]> wrote: > Thanks for the pointers Bryan, however wrt your first suggestion. I tried > without setting SSL properties on System properties and get an unable to > find ssl path error – this gets resolved by doing as I have done (but of > course this may be a red herring). I initially tried setting on site > builder but got the same error as below – it appears to make no difference > as to what is logged in the nifi-users.log if I include SSL props on site > builder or not, I get the same error viz: > > 2016-05-20 08:59:47,082 INFO [NiFi Web Server-29590180] > o.a.n.w.s.NiFiAuthenticationFilter Attempting request for > (<CN=spark-processor.m.xxx, OU=Development, O=Secure Data Europe Ltd, > L=Maidstone, ST=Kent, C=GB>) GET > https://yarn-cm1.m.xxxx:9090/nifi-api/controller (source ip: xx.xx.xx.1) > 2016-05-20 08:59:47,082 INFO [NiFi Web Server-29494759] > o.a.n.w.s.NiFiAuthenticationFilter Attempting request for > (<CN=spark-processor.m.xxx, OU=Development, O=Secure Data Europe Ltd, > L=Maidstone, ST=Kent, C=GB>) GET > https://yarn-cm1.m.xxx:9090/nifi-api/controller (source ip: xx.xx.xx.1) > 2016-05-20 08:59:47,083 INFO [NiFi Web Server-29590180] > o.a.n.w.s.NiFiAuthenticationFilter Rejecting access to web api: Unable to > verify access for CN=spark-processor.m.xxx, OU=Development, O=Secure Data > Europe Ltd, L=Maidstone, ST=Kent, C=GB > > I am using self signed certs if that makes a difference (but these work > fine on across the cluster). I am not seeing my spark user appear in the > list of users to grant access. > > I have turned on debug for ssl to see if that is throwing up anything but > nothing appears obvious – here is the snipet that I would expect errors to > be shown from that log. > > ... no IV derived for this protocol > %% Server resumed [Session-4, TLS_RSA_WITH_AES_128_CBC_SHA256] > NiFi Receiver, READ: TLSv1.2 Change Cipher Spec, length = 1 > NiFi Receiver, READ: TLSv1.2 Handshake, length = 80 > *** Finished > verify_data: { 109, 126, 134, 14, 33, 110, 224, 83, 198, 116, 54, 228 } > *** > NiFi Receiver, WRITE: TLSv1.2 Change Cipher Spec, length = 1 > *** Finished > verify_data: { 83, 120, 49, 158, 181, 136, 127, 219, 30, 194, 58, 167 } > *** > NiFi Receiver, WRITE: TLSv1.2 Handshake, length = 80 > NiFi Receiver, WRITE: TLSv1.2 Application Data, length = 240 > > I don’t really know enough about certificates and how client java apps > would use them wrt to the host name/ ip address etc. of details is included > in them. The nifi-user.log is showing access from a specific IP address > which clearly doesn’t match the CN details in the cert. Just clutching at > straws here! > > Any other suggestions? > > Thanks > Conrad > > From: Bryan Bende <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Thursday, 19 May 2016 at 17:08 > To: "[email protected]" <[email protected]> > Subject: Re: Spark & NiFi question > > Hi Conrad, > > I think there are a couple of things at play here... > > One is that the SSL properties need to be set on the > SiteToSiteClientBuilder, rather than through system properties. There > should be methods to set the keystore and other values. > > In a secured NiFi instance, the certificate you are authenticating with > (the keystore used by the s2s client) would need to have an account in > NiFi, and would need to have access to the output port. > If you attempt to make a request with that cert, and then you go into the > NiFi UI as another user, you should be able to go into the accounts section > (top right) and approve the account for that certificate. > > Then if you stop your output port, right-click and Configure... and from > the Access Controls tab started typing the DN from your cert and add that > user to the Allowed Users list. Hit Apply and started the port again. > > We probably need to document this better, or write up an article about it > somewhere. > > Let us know if its still not working. > > Thanks, > > Bryan > > > On Thu, May 19, 2016 at 11:54 AM, Conrad Crampton < > [email protected]> wrote: > >> Hi, >> Tried following a couple of blog posts about this [1], [2], but neither >> of these refer to using NiFi in clustered environment with SSL and I >> suspect this is where I am hitting problems (but don’t know where). >> >> The blogs state that using an output port (in the root process group I.e. >> on main canvas) which I have done and tried to connect thus.. >> >> System.setProperty("javax.net.ssl.keyStore", "/spark-processor.jks"); >> System.setProperty("javax.net.ssl.keyStorePassword", *“******"); >> System.setProperty("javax.net.ssl.trustStore", *“*/cacerts.jks"); >> >> SiteToSiteClientConfig config = new SiteToSiteClient.Builder() >> .url("https://yarn-cm1.mis-cds.local:9090/nifi";) >> .portName("Spark test out") >> .buildConfig(); >> >> SparkConf sparkConf = new SparkConf().setMaster("local[2]").setAppName("NiFi >> Spark Log Processor"); >> JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, new >> Duration(5000)); >> JavaReceiverInputDStream<NiFiDataPacket> packetStream = >> jssc.receiverStream(new NiFiReceiver(config, StorageLevel.MEMORY_ONLY())); >> >> JavaDStream text = packetStream.map(dataPacket -> new >> String(dataPacket.getContent(), StandardCharsets.UTF_8)); >> text.print(); >> jssc.start(); >> jssc.awaitTermination(); >> >> The error I am getting is >> >> 16/05/19 16:39:03 WARN ReceiverSupervisorImpl: Restarting receiver with >> delay 2000 ms: Failed to receive data from NiFi >> java.io.IOException: Server returned HTTP response code: 401 for URL: >> https://yarn-cm1.mis-cds.local:9090/nifi-api/controller >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) >> at >> sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1889) >> at >> sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1884) >> at java.security.AccessController.doPrivileged(Native Method) >> at >> sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1883) >> at >> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1456) >> at >> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440) >> at >> sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) >> at >> org.apache.nifi.remote.util.NiFiRestApiUtil.getController(NiFiRestApiUtil.java:69) >> at >> org.apache.nifi.remote.client.socket.EndpointConnectionPool.refreshRemoteInfo(EndpointConnectionPool.java:891) >> at >> org.apache.nifi.remote.client.socket.EndpointConnectionPool.getPortIdentifier(EndpointConnectionPool.java:878) >> at >> org.apache.nifi.remote.client.socket.EndpointConnectionPool.getOutputPortIdentifier(EndpointConnectionPool.java:862) >> at >> org.apache.nifi.remote.client.socket.SocketClient.getPortIdentifier(SocketClient.java:81) >> at >> org.apache.nifi.remote.client.socket.SocketClient.createTransaction(SocketClient.java:123) >> at >> org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable.run(NiFiReceiver.java:149) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.io.IOException: Server returned HTTP response code: 401 >> for URL: https://yarn-cm1.mis-cds.local:9090/nifi-api/controller >> at >> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1839) >> at >> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440) >> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) >> at >> sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338) >> at >> org.apache.nifi.remote.util.NiFiRestApiUtil.getController(NiFiRestApiUtil.java:66) >> ... 7 more >> >> Any pointers would be helpful in getting this working. I don’t know if I >> have to set up a remote process group with the output port (not sure how >> this works), or what. When I go to >> https://yarn-cm1.mis-cds.local:9090/nifi-api/controller in the browser, >> I get an access denied error. >> I have created keystore and signed by the RootCA used to sign all the >> self signed certs for the cluster. >> >> Running 0.6.1, 6 node cluster. >> >> Thanks >> Conrad >> >> [1[ - >> https://community.hortonworks.com/articles/12708/nifi-feeding-data-to-spark-streaming.html >> [2] - >> https://blogs.apache.org/nifi/entry/stream_processing_nifi_and_spark >> >> >> SecureData, combating cyber threats >> >> ------------------------------ >> >> The information contained in this message or any of its attachments may >> be privileged and confidential and intended for the exclusive use of the >> intended recipient. If you are not the intended recipient any disclosure, >> reproduction, distribution or other dissemination or use of this >> communications is strictly prohibited. The views expressed in this email >> are those of the individual and not necessarily of SecureData Europe Ltd. >> Any prices quoted are only valid if followed up by a formal written quote. >> >> SecureData Europe Limited. Registered in England & Wales 04365896. >> Registered Address: SecureData House, Hermitage Court, Hermitage Lane, >> Maidstone, Kent, ME16 9NT >> > > > > ***This email originated outside SecureData*** > > Click here > <https://www.mailcontrol.com/sr/JOj4ovws70LGX2PQPOmvUqa7UuQeNDoM5CPuVUMi!aLghcUmWuJbL8QAhL3vPgRnasXOF8Vdo14NCU1!U1Tbvw==> > to report this email as spam. >
