I have an issue to validate, reported at before: http://apache-nifi.1125220.n5.nabble.com/Error-instantiating-template-on-cluster-The-specified-observer-identifier-already-exists-td12973.html
I reproduced it at apache nifi 1.5.0. I will try to check at nifi 1.9.2. The template to load is near to 50MB. Juan On Fri, 18 Oct 2019 at 14:13, Joe Witt <joe.w...@gmail.com> wrote: > is a daily effort at this point. i am close to pushing first rc. have > been watching for stability on bug fixes. > > On Fri, Oct 18, 2019 at 1:10 PM Juan Pablo Gardella < > gardellajuanpa...@gmail.com> wrote: > >> Any ETA for Nifi 1.10 release? >> >> On Fri, 18 Oct 2019 at 13:39, Mike Thomsen <mikerthom...@gmail.com> >> wrote: >> >>> Peter, >>> >>> Are you configuring the service as a trust-only configuration? If so, >>> that's been addressed in the 1.10 which is due for release in the near(ish) >>> future. >>> >>> https://issues.apache.org/jira/browse/NIFI-6228 >>> >>> Thanks, >>> >>> Mike >>> >>> On Fri, Oct 18, 2019 at 11:06 AM Peter Moberg <peter.mob...@gmail.com> >>> wrote: >>> >>>> As a follow-up. >>>> >>>> On the Nifi node I am able to do a GET to Elastic Search using curl. I >>>> specify the —cacert option giving it the self-signed root certificate. >>>> >>>> Of course, this isn’t using the TrustStore but I am able to use the >>>> TrustStore if I use other ES processors… just not the >>>> ElasticSearchClientServicesImpl. >>>> >>>> On Oct 18, 2019, 12:48 AM -0500, Peter Moberg <peter.mob...@gmail.com>, >>>> wrote: >>>> >>>> Hi Andy, >>>> >>>> thanks for your suggestions. Here is what I have tried so far (still no >>>> luck). >>>> >>>> Connecting with openssl and viewing the certs it presents >>>> >>>> *openssl s_client -connect quickstart-es-http.es >>>> <http://quickstart-es-http.es>-cluster -showcerts* >>>> >>>> If I then look inside the server cert I can find this >>>> >>>> >>>> >>>> >>>> >>>> *Server Cert: Issuer: OU = quickstart, CN = quickstart-http X509v3 >>>> Subject Alternative Name: DNS:quickstart-es-http.es >>>> <http://quickstart-es-http.es>-cluster.es.local, DNS:quickstart-es-http, >>>> DNS:quickstart-es-http.es <http://quickstart-es-http.es>-cluster.svc, >>>> DNS:quickstart-es-http.es <http://quickstart-es-http.es>-cluster* >>>> >>>> >>>> If I look in to the self-signed root cert I find this: >>>> >>>> >>>> *Root Cert: Subject: OU = quickstart, CN = quickstart-http* >>>> >>>> >>>> I now double check my trust store to make sure the Root Cert is there. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *Trust store content Your keystore contains 1 entry Alias name: >>>> ca_elastic Creation date: Oct 16, 2019 Entry type: trustedCertEntry Owner: >>>> CN=quickstart-http, OU=quickstart Issuer: CN=quickstart-http, OU=quickstart >>>> Serial number: 5aa50b6806d2394fff6f98d2b7d4c287 Valid from: Fri Oct 11 >>>> 14:35:01 UTC 2019 until: Sat Oct 10 14:36:01 UTC 2020 Certificate >>>> fingerprints: MD5: 1E:E3:33:13:EA:AC:B5:61:23:DE:2E:1A:D7:9C:AA:F0 SHA1: >>>> 62:EC:5B:EB:32:6A:38:3D:6A:6B:F7:10:5A:DE:E6:F1:F0:5B:07:99 SHA256: >>>> B4:B5:06:9C:50:5F:E8:A1:58:7C:C7:2C:37:52:2F:E0:CF:32:18:18:68:E4:C7:37:F8:82:B3:BC:61:EB:5B:CF >>>> Signature algorithm name: SHA256withRSA Subject Public Key Algorithm: >>>> 2048-bit RSA key Version: 3 Extensions: #1: ObjectId: 2.5.29.19 >>>> Criticality=true BasicConstraints:[ CA:true PathLen:2147483647 ] #2: >>>> ObjectId: 2.5.29.37 Criticality=false ExtendedKeyUsages [ serverAuth >>>> clientAuth ] #3: ObjectId: 2.5.29.15 Criticality=true KeyUsage [ >>>> DigitalSignature Key_CertSign ]* >>>> >>>> So everything looks Ok. But when I run the >>>> ElasticSearchClientServicesImpl with a SSLContext pointing to my trust >>>> store I still get the following output in the log. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine >>>> problem at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at >>>> sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728) at >>>> sun.security.ssl.Handshaker.fatalSE(Handshaker.java:330) at >>>> sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322) at >>>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1633) >>>> at >>>> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) >>>> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052) at >>>> sun.security.ssl.Handshaker$1.run(Handshaker.java:992) at >>>> sun.security.ssl.Handshaker$1.run(Handshaker.java:989) at >>>> java.security.AccessController.doPrivileged(Native Method) at >>>> sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1467) at >>>> org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:283) >>>> at >>>> org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:353) >>>> ... 9 common frames omitted Caused by: >>>> sun.security.validator.ValidatorException: PKIX path building failed: >>>> sun.security.provider.certpath.SunCertPathBuilderException: unable to find >>>> valid certification path to requested target at >>>> sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) at >>>> sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) >>>> at sun.security.validator.Validator.validate(Validator.java:262) at >>>> sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) >>>> at >>>> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281) >>>> at >>>> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136) >>>> at >>>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1620) >>>> ... 17 common frames omitted* >>>> >>>> Both the Nifi install and Elastic Search install is running in >>>> Kubernetes. The address I am using is a service address that is backed by 3 >>>> ES instances. However, I double checked all three of the ES nodes to make >>>> sure that they returned back the same SSL cert and they did. >>>> >>>> The only thing I haven't been able to figure out is how to check if >>>> Kubernetes/ES reacts differently when you do a GET vs POST. Feels strange >>>> that it would return different SSL certs but stranger things have happened… >>>> >>>> >>>> >>>> On Oct 17, 2019, 3:25 PM -0500, Andy LoPresto <alopre...@apache.org>, >>>> wrote: >>>> >>>> Hi Peter, >>>> >>>> If you can use openssl’s s_client command (example below) to connect to >>>> the endpoint and verify that the hostname matches the certificate and that >>>> the certificate contains a SubjectAlternativeName entry with that hostname >>>> (see RFC 6125 [1] for more details), this should help you debug the issue. >>>> The cause of the PKIX error is that the truststore doesn’t contain a >>>> certificate (or certificate chain) which matches the hostname presented by >>>> the remote endpoint. I think you understand that based on your message. The >>>> underlying reason for this is could be one of the following: >>>> >>>> * the server is behind an interface which responds differently to GET >>>> and POST/PUT requests >>>> * there is a load-balancer which is directing the requests >>>> coincidentally to different backend servers (one has the right cert; the >>>> other doesn’t) >>>> * I recall something around the addition of (some) Elastic Search >>>> components which handled TLS in an ES client-specific manner; I remember >>>> advocating for standard NiFi TLS interaction here but I am not sure what >>>> was ultimately contributed. If it’s not one of the above issues, I can >>>> investigate further. >>>> >>>> Hopefully this helps. >>>> >>>> [1] https://tools.ietf.org/html/rfc6125#section-6.4.4 >>>> >>>> s_client example: >>>> >>>> $ openssl s_client -connect <host:port> -debug -state -cert >>>> <path_to_your_cert.pem> -key <path_to_your_key.pem> -CAfile >>>> <path_to_your_CA_cert.pem> >>>> >>>> Andy LoPresto >>>> alopre...@apache.org >>>> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>> >>>> On Oct 16, 2019, at 8:37 PM, Peter Moberg <peter.mob...@gmail.com> >>>> wrote: >>>> >>>> I have an Elastic Search cluster that is setup with SSL. It uses a >>>> self-signed cert for this. I am working with Apache Nifi 1.9.2. I have a >>>> flow that has the PutElasticSearchHttp component. I have setup a >>>> SSLContextService for that component where I have specified a trust store >>>> that has the self-signed cert from ES. I specify an https endpoint to >>>> access Elastic Search and Im having no issues populating my Elastic Search >>>> instance using this flow. >>>> >>>> I have another flow where I want to do some lookups. So I have been >>>> using the LookupRecord processor. That one I have associated with an >>>> ElasticSearchClientServiceImpl which I have setup to point to the same >>>> SSLContextService as used above. I specified the same HTTPS Url (triple >>>> checked this). However, when I run this second Flow I am not able to verify >>>> the ES server's self-signed certificate. >>>> >>>> I check the nifi-app.log and it says: >>>> Caused by: sun.security.provider.certpath.SunCertPathBuilderException: >>>> unable to find valid certification path to requested target >>>> >>>> I am a bit surprised that I am not able to verify the same server >>>> certificate in the two different flows. >>>> >>>> Completely stuck on this so if anyone have any pointers please let me >>>> know. >>>> >>>> Thanks, >>>> >>>> Peter >>>> >>>> >>>>
