Pat, The 403 status means that you are not providing your client certificate for authentication, not that there is something wrong with the server certificate. I’m operating off very limited information here, but I’ll describe what I believe is happening and how to fix it:
1. Your curl command has two interesting pieces. The first is that you must provide a client certificate (—cert) and private key (—key) in order to access the Mitre resource. This indicates that the TLS negotiation requires mutual authentication (i.e. the server cares who is connecting to it, not just accepting incoming connections from any client and proving the server identity and encrypting the communication channel). To replicate this in Apache NiFi, you’ll need to configure a StandardSSLControllerService, which allows you to provide a *keystore*, which is a Java file containing your client certificate and private key, and reference it in your processor in order to authenticate yourself to the remote resource. You can google the commands to convert your PEM-encoded files into the JKS format. 2. The second interesting thing about your curl command is that it only works when using —insecure. This indicates that the server certificate presented by xxx.mitre.org is not signed by a Certificate Authority (CA) which is publicly known and preloaded as part of the standard OS or browser configuration. You can view the certificate in full by using OpenSSL tools to download the certificate. I’ve provided that command below. You’ll need to load this server certificate containing the public key into a *truststore*, which is just a keystore file that does not contain any private keys and is used by the client (NiFi in this case) for verifying the other end of a connection. The truststore is also referenced by your SSL controller service to allow the processor to verify the Mitre server. 3. All of this should work with GetHTTP, but it is an older processor and I would recommend you switch to InvokeHTTP. It will perform the same functionality but is more robust and better tested, and the difference will continue to grow in the future. I’ve also included some other resources that may help you. This can be complicated so if you get stuck on a specific piece, please write back and I can provide more specific help. My battery is dying or I would include some of the command line commands you need. But these are all available by searching online, especially on Stack Overflow. Good luck. [1] https://community.hortonworks.com/questions/19476/connecting-to-facebook-graph-api-using-nifi-postht.html <https://community.hortonworks.com/questions/19476/connecting-to-facebook-graph-api-using-nifi-postht.html> Command to download a server certificate: openssl s_client -connect nifi.apache.org:443 -showcerts Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 12, 2017, at 5:29 AM, James Srinivasan <[email protected]> > wrote: > > I might've been wrong about the certificate issue, since the server is > returning a 403. What happens if you omit --insecure from the curl > command? Might the server be doing something funky e.g. looking at the > user agent string? > > I suspect the equivalent of --insecure requires recompiling the > processor and is almost certainly not the right thing to do. > > On 12 October 2017 at 13:13, Jones, Patrick L. <[email protected]> wrote: >> Thanks for the reply. >> >> I did not generate the certs myself, they were generated by a third >> party. As I said the only way I can access the site is with curl and >> --insecure. Does anyone know of a way in NIFI to do the equivalence of >> "--insecure" ? The curl that works is: >> curl --noproxy "*" --insecure --cacert ./ca.pem --cert ./c_cert.pem --key >> ./c_key.pem https://xxx.mitre.org/xx/xx >> >> thank you >> >> >> Pat >> >> >> -----Original Message----- >> From: James Srinivasan [mailto:[email protected]] >> Sent: Wednesday, October 11, 2017 4:15 PM >> To: [email protected] >> Subject: Re: GetHTTP 403:Forbidden >> >> Hi, >> >> Doesn't sound like you have the certs set up correctly since --insecure for >> curl skips certificate validation. I'm not aware of a similar option for >> NiFi, but assuming you generated the certificate yourself, searching for >> something like "java https self signed web" >> should help. If the certificate was generated by a third party, then make >> sure the appropriate intermediate and root certificates are also in your >> store. >> >> James >> >> On 11 October 2017 at 20:37, pat <[email protected]> wrote: >>> Greetings, >>> >>> I am using GetHTTP to call >>> https://xxx.mitre.org/xx/xx >>> >>> I have set up the StandardSSLContextService with my certs >>> >>> When GetHTTP runs I get the error: >>> [Timer-Driven Process Thread-6] o.a.nifi.processors.standard.GetHTTP >>> GetHTTP[id=015e1160-a849-126f-0306-deadef9b45f3] received status code >>> 403:Forbidden from https://xxx.mitre.org/xx/xx >>> >>> I can reach the site via firefox if I add a security exception. >>> I can also reach the site with curl like: >>> curl --noproxy "*" --insecure --cacert .ca.pem --cert .c_cert.pem --key >>> ./c_key.pem https://xxx.mitre.org/xx/xx >>> >>> I assume the problem is that I don't know how to have NIFI do >>> something like the "--insecure" options. >>> Is there a way to do this in NIFI (or a work around) >>> >>> thank you >>> >>> >>> >>> -- >>> Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
signature.asc
Description: Message signed with OpenPGP using GPGMail
