Matt, Sorry, the confusion about where it was supported is entirely mine.
Thanks again for the help! Joe On Wed, Dec 21, 2016 at 4:58 PM, Matt Gilman <[email protected]> wrote: > Joe, > > Just to be completely clear. It was only ever offered for the REST API. I > don't believe that is broken. I verified that we can introduce it in other > places using the built-in Java capabilities. Because of that, I think we > can remove the legacy verification. > > Yes, all SSL traffic should support OCSP once NIFI-1364 is implemented [1]. > > Matt > > [1] https://issues.apache.org/jira/browse/NIFI-1364 > > On Wed, Dec 21, 2016 at 11:32 AM, Joe Skora <[email protected]> wrote: > > > Matt, > > > > Thanks for digging into this. Since it's verified to be broken in the > > current releases, I'll call off the folks trying to test it on our end. > > > > After these changes will all SSL traffic for the Web UI, REST API, > > Site-to-Site, and Clustering support OCSP? > > > > Thanks, > > Joe > > > > On Wed, Dec 21, 2016 at 3:25 PM, Matt Gilman <[email protected]> > > wrote: > > > > > Joe, > > > > > > I was able to successfully verify revoked certificates for clients and > > > nodes joining the cluster. This did require some code changes. > > > Specifically, the changes you were suggesting > > > (PKIXBuilderParameters.setRevocationEnabled and the ocsp.enabled > > Security > > > property). > > > > > > I think the best path forward would be to introduce these changes, > > > consolidating where possible, and remove the existing legacy OCSP > > > verification in a PR for NIFI-1364. I'll comment regarding my finding > on > > > the JIRA. > > > > > > Matt > > > > > > On Tue, Dec 20, 2016 at 10:36 AM, Matt Gilman <[email protected] > > > > > wrote: > > > > > > > Joe, > > > > > > > > I believe the JIRA is questioning whether we need to be manually > > > > verifying. It probably makes sense to answer that first. Once we know > > > that > > > > answer we can establish the best path to appropriately resolve > > NIFI-1364 > > > > and ensure that we are verifying the certificates in all places. I'm > > > going > > > > to do some testing locally to see if I can answer that question. > Please > > > do > > > > the same and then let's sync up again here. > > > > > > > > Also, a lot of good details here [1]. > > > > > > > > Matt > > > > > > > > [1] http://blogs.nologin.es/rickyepoderi/index.php?/ > > > > archives/77-BUG-in-Java-OCSP-Implementation-PKIX.html > > > > > > > > On Tue, Dec 20, 2016 at 8:33 AM, Joe Skora <[email protected]> wrote: > > > > > > > >> Matt, > > > >> > > > >> We found the "ocsp.enable" Java Security setting in > > > >> $JRE/lib/security/java.security, but setting that did NOT change > the > > > >> behavior and the node with the revoked certificate could still join > > the > > > >> cluster. > > > >> > > > >> Looking at the posts referenced by NIFI-1364 [1], they all seem to > > > discuss > > > >> needed code changes. For example, the first link [2], includes this > > > >> example. > > > >> > > > >> PKIXParameters params = new PKIXParameters(anchors); > > > >> > > > >> // Activate certificate revocation checking > > > >> params.setRevocationEnabled(true); > > > >> > > > >> // Activate OCSP > > > >> Security.setProperty("ocsp.enable", "true"); > > > >> > > > >> > > > >> It looks like "ocsp.enable" may only matter if revocation is enabled > > in > > > >> the > > > >> first place and that appears to be what the > > > >> PKIXParameters.setRevocationEnabled(true) call accomplishes. I > don't > > > >> know > > > >> the clustering code well enough to dive in, but if you can point out > > > where > > > >> to look I can try digging into it myself. For now, it still appears > > > that > > > >> cluster nodes are not rejected due to OCSP revocation. > > > >> > > > >> Thanks, > > > >> Joe > > > >> > > > >> [1] https://issues.apache.org/jira/browse/NIFI-1364 > > > >> [2] https://blogs.oracle.com/xuelei/entry/enable_ocsp_checking > > > >> > > > >> On Mon, Dec 19, 2016 at 5:57 PM, Joe Skora <[email protected]> > wrote: > > > >> > > > >> > Thanks, I'll check on the configuration used for the tests and > reply > > > >> back > > > >> > here once that's clear. > > > >> > > > > >> > On Mon, Dec 19, 2016 at 12:48 PM, Matt Gilman < > > > [email protected]> > > > >> > wrote: > > > >> > > > > >> >> The existing OCSP logic is part of the REST API filter chain. The > > > >> >> communications you're referring to are happening outside of that. > > > Have > > > >> you > > > >> >> tried enabling OCSP as part of the SSL/TLS handshake as was > > mentioned > > > >> in > > > >> >> the JIRA [1]. Using the built-in features should allow us to use > it > > > >> >> throughout the application regardless of the communications in > > > question > > > >> >> (cluster protocol, site to site, REST API, etc). > > > >> >> > > > >> >> Matt > > > >> >> > > > >> >> [1] https://issues.apache.org/jira/browse/NIFI-1364 > > > >> >> > > > >> >> On Mon, Dec 19, 2016 at 12:18 PM, Joe Skora <[email protected]> > > > wrote: > > > >> >> > > > >> >> > Matt, > > > >> >> > > > > >> >> > It's not clients we are concerned with, but cluster servers. > > > >> >> > > > > >> >> > The test process used Java 1.8.0_65 and NiFi 0.7.1 to do the > > > >> following. > > > >> >> > > > > >> >> > 1. Configure a cluster with valid certificates for each > node, > > > >> >> > 2. revoke one node's certificate, > > > >> >> > 3. restart the cluster, > > > >> >> > 4. confirm with keytool that the node is invalid, and > > > >> >> > 5. test whether the node can still join the cluster. > > > >> >> > > > > >> >> > The expectation was that in #5 the node would not be able to > join > > > to > > > >> the > > > >> >> > cluster, but it could. > > > >> >> > > > > >> >> > Whether the OCSP check should be handled by NiFi or Java, it > > > doesn't > > > >> >> appear > > > >> >> > to be happening. > > > >> >> > > > > >> >> > Thanks, > > > >> >> > Joe > > > >> >> > > > > >> >> > On Mon, Dec 19, 2016 at 11:22 AM, Matt Gilman < > > > >> [email protected]> > > > >> >> > wrote: > > > >> >> > > > > >> >> > > Joe, > > > >> >> > > > > > >> >> > > If a server connects through the REST API it should be > subject > > to > > > >> the > > > >> >> > same > > > >> >> > > checks as a regular user. Can you provide more details > > regarding > > > >> the > > > >> >> > > requests that aren't being checked correctly? > > > >> >> > > > > > >> >> > > Additionally, there was some discussion whether we need the > > > >> additional > > > >> >> > > checks in the first place as we may be able to leverage > checks > > > >> built > > > >> >> into > > > >> >> > > Java [1]. > > > >> >> > > > > > >> >> > > Matt > > > >> >> > > > > > >> >> > > [1] https://issues.apache.org/jira/browse/NIFI-1364 > > > >> >> > > > > > >> >> > > On Mon, Dec 19, 2016 at 10:57 AM, Joe Skora < > [email protected]> > > > >> wrote: > > > >> >> > > > > > >> >> > > > This could very soon be a show stopper for us. > > > >> >> > > > > > > >> >> > > > Does anyone have any thoughts that might help us get this > > > >> straight? > > > >> >> > > > > > > >> >> > > > On Wed, Dec 14, 2016 at 2:23 PM, Joe Skora < > [email protected] > > > > > > >> >> wrote: > > > >> >> > > > > > > >> >> > > > > Running Apache NiFi 0.7.1, we see clients rejected due to > > > OCSP > > > >> >> > > revocation > > > >> >> > > > > of their certificates but we think we are seeing > instances > > > >> where > > > >> >> > > servers > > > >> >> > > > > using OCSP revoked certificates are still able to connect > > to > > > a > > > >> >> > cluster. > > > >> >> > > > > > > > >> >> > > > > Should OCSP revocation cause these servers to be rejected > > by > > > >> the > > > >> >> > > cluster? > > > >> >> > > > > > > > >> >> > > > > Could this be a configuration problem even though the > > revoked > > > >> >> clients > > > >> >> > > > > certificates are rejected? > > > >> >> > > > > > > > >> >> > > > > Thanks, > > > >> >> > > > > Joe > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > >
