Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-22 Thread Mike Thomsen
Peter, Looks like this won't make it into 1.10, but it's a simple patch to apply if you download the source code for 1.10 and apply it manually. Mike On Tue, Oct 22, 2019 at 2:46 PM Matt Burgess wrote: > For saving space (if you want to run from the assembly/target directory) > you can add a

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-22 Thread Matt Burgess
For saving space (if you want to run from the assembly/target directory) you can add a -Ddir-only and it won’t build the tar.gz. Sent from my iPhone > On Oct 22, 2019, at 2:25 PM, Mike Thomsen wrote: > >  > If you run `mvn clean install -DskipTests=true` from the root of the source >

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-22 Thread Mike Thomsen
If you run `mvn clean install -DskipTests=true` from the root of the source folder, you'll get a tar.gz build in $ROOT/nifi-assembly/target. I'd recommend testing against that as it'll be faster. On Tue, Oct 22, 2019 at 1:56 PM Peter Moberg wrote: > Mike, > > thanks for putting together that

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-22 Thread Peter Moberg
Mike, thanks for putting together that PR. I have built everything successfully but I haven't been able to test this yet since I haven't built the new Docker image. I assume you guys just use the ‘dockermaven’  folder to build the Dockerfile with src artifacts? My current dev machine is

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-20 Thread Mike Thomsen
As a compromise, I upgraded to the latest 5.X client and manually incremented Apache HttpClient to 4.5.10. PR is here: https://github.com/apache/nifi/pull/3828 There are integration tests for that package that automatically startup and provision an ES node, and they all passed with this

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-20 Thread Mike Thomsen
There's no hard and fast reason to stay with 5.X there, so you can build your own copy of 1.9.2 with that dependency upgraded if you want to try it out. I'll try to find time to test that change on 1.10.0-SNAPSHOT. On Sun, Oct 20, 2019 at 1:52 AM Peter Moberg wrote: > The certs in the

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-19 Thread Peter Moberg
The certs in the TrustStore are marked as trusted. The host name specified in the ClientServiceImpl is: https://quickstart-es-http.es-cluster:9200 The CN field of the server certificate is: https://quickstart-es-http.es-cluster.es.local So at first glance it looks like the issue would be that

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-19 Thread Mike Thomsen
I'm far from a SSL/TLS expert, but let's get these out of the way: 1. Did you mark the server's cert as "trusted" when you created the trust store with keytool? 2. Are you sure that you're specifying the same hostname in the client service that is in the CN field in the server's cert? FWIW, if

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-19 Thread Peter Moberg
Nope. No good. I even dump the network traffic and analyzed it in Wireshark. The ES server sends back two certificates (server + self-signed one) and both of them are present in my TrustStore. I am specifying both a TrustStore and a Keystore now but it still gives the error that it can’t find

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Peter Moberg
Think I might have found the issue. Will report tonight. Mike, please don’t spend any time debugging this because I think it might be an issue on my side. Appreciate all the help so far. Thanks, Peter On Oct 18, 2019, 2:21 PM -0500, Peter Moberg , wrote: > Here it is: > > > 2019-10-18

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Peter Moberg
Here it is: 2019-10-18 18:47:02,548 ERROR [Timer-Driven Process Thread-7] o.a.n.processors.standard.LookupRecord LookupRecord[id=df596687-016d-1000--65536eb2] Failed to process StandardFlowFileRecord[uuid=64d0d1f4-1960-4a91-9394-39edc9d6c9c7,claim=StandardContentClaim

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Mike Thomsen
Can you share the stacktrace from the logs? On Fri, Oct 18, 2019 at 2:38 PM Peter Moberg wrote: > Mike, > > The SSLContextService only had the Trust store configured. I think I seen > that ticket before but didn’t pay attention to the fact it wasn’t merged in > to the code I am running. > >

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Peter Moberg
Mike, The SSLContextService only had the Trust store configured. I think I seen that ticket before but didn’t pay attention to the fact it wasn’t merged in to the code I am running. However, I configured the service to have a KeyStore now but I am getting the same errors… Thanks, Peter On

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Juan Pablo Gardella
I have an issue to validate, reported at before: http://apache-nifi.1125220.n5.nabble.com/Error-instantiating-template-on-cluster-The-specified-observer-identifier-already-exists-td12973.html I reproduced it at apache nifi 1.5.0. I will try to check at nifi 1.9.2. The template to load is near to

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Joe Witt
is a daily effort at this point. i am close to pushing first rc. have been watching for stability on bug fixes. On Fri, Oct 18, 2019 at 1:10 PM Juan Pablo Gardella < gardellajuanpa...@gmail.com> wrote: > Any ETA for Nifi 1.10 release? > > On Fri, 18 Oct 2019 at 13:39, Mike Thomsen wrote: > >>

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Juan Pablo Gardella
Any ETA for Nifi 1.10 release? On Fri, 18 Oct 2019 at 13:39, Mike Thomsen wrote: > Peter, > > Are you configuring the service as a trust-only configuration? If so, > that's been addressed in the 1.10 which is due for release in the near(ish) > future. > >

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Mike Thomsen
Peter, Are you configuring the service as a trust-only configuration? If so, that's been addressed in the 1.10 which is due for release in the near(ish) future. https://issues.apache.org/jira/browse/NIFI-6228 Thanks, Mike On Fri, Oct 18, 2019 at 11:06 AM Peter Moberg wrote: > As a

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-18 Thread Peter Moberg
As a follow-up. On the Nifi node I am able to do a GET to Elastic Search using curl. I specify the —cacert option giving it the self-signed root certificate. Of course, this isn’t using the TrustStore but I am able to use the TrustStore if I use other ES processors… just not the

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-17 Thread Peter Moberg
Hi Andy, thanks for your suggestions. Here is what I have tried so far (still no luck). Connecting with openssl and viewing the certs it presents openssl s_client -connect quickstart-es-http.es-cluster -showcerts If I then look inside the server cert I can find this Server Cert: Issuer: OU =

Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-17 Thread Andy LoPresto
Hi Peter, If you can use openssl’s s_client command (example below) to connect to the endpoint and verify that the hostname matches the certificate and that the certificate contains a SubjectAlternativeName entry with that hostname (see RFC 6125 [1] for more details), this should help you

ElasticSearchClientServiceImpl not working for secured ElasticSearch

2019-10-16 Thread Peter Moberg
I have an Elastic Search cluster that is setup with SSL. It uses a self-signed cert for this. I am working with Apache Nifi 1.9.2. I have a flow that has the PutElasticSearchHttp component. I have setup a SSLContextService for that component where I have specified a trust store that has the