I have reviewed a couple different PRs so I'll add some context where I can. Obviously Simon would be the most qualified to answer but I'll add my thoughts.
For question 1, while they may not all be necessary I think it does make sense to include them in this feature branch if our primary goal is integrating Knox SSO. We could push off removing JDBC authentication for reasons I'll get to in my response to question 2. If we want to do one at a time (switch to spring boot, add Zuul as a dependency, then add Knox SSO) then that's ok but I do think there are dependencies and should be done in order. For example, adding Knox SSO requires some work around request filtering. If we were to do this before moving to Spring Boot we would need to implement the filters in Nodejs which would be throwaway once we get around to migrating away from that. For Zuul, I believe it's purpose is to facilitate the filtering (although it does a lot more) so it doesn't make sense to add that separate from the Knox SSO work. For question 2, I think you bring up a good point. We probably don't want to just rip our current authentication method out. We might want to consider deprecating it instead and making Knox SSO and LDAP authentication optional. For question 3, this is a bigger shift than just a component upgrade. It's more like shifting platforms, from Elasticsearch to Solr for example. Like I alluded to in my response to question 1, I don't think we should require throwaway work just because we want to review these parts separately. For question 4, I will defer to Simon. I don't believe we necessarily require Zuul so I will let him elaborate on why he choose that library and what the potential impact is of adding it to our project. For question 5 and 6, I will also defer to Simon on this. The focus of this feature as I understand it is a consistent authentication mechanism and support for SSO. I will let him lay out his vision for micro services. Knox SSO would be a great improvement and is what I think we should focus on in this feature branch. Micro services is something we should certainly discuss but it might be a bit of a distraction and I wouldn't want to hold up the other useful parts. On Fri, Sep 14, 2018 at 8:38 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Hey all, > > I started looking through the Knox SSO feature branch (see here > https://issues.apache.org/jira/browse/METRON-1663). This is some great new > security functionality work and it looks like it will bring some important > new features to the Metron platform. I'm coming at this pretty green, so I > do have some questions regarding the proposed changes from a high level > architectural perspective. There are a few changes within the current FB > PR's that I think could use some further explanation. At first glance, it > seems we could potentially simplify this branch a great deal and get it > completed much sooner if we narrowed the focus a bit. But I could certainly > be wrong here and happy for other opinions. I searched through the mailing > list history to see if there is any additional background and the main > DISCUSS thread I could find was regarding initially setting up the feature > branch, which talked about adding Knox and LDAP. > > https://lists.apache.org/thread.html/cac2e6314284015b487121e77abf730abbb7ebec4ace014b19093b4c@%3Cdev.metron.apache.org%3E > . > If I've missed any follow-up, please let me know. > > Looking at the broader set of Jiras associated with 1663 and the first PR > 1665, it looks like there are 4 main thrusts of this branch right now: > > 1. Knox/SSO > 2. Node migrated to Spring Boot > 3. JDBC removed completely in favor of LDAP > 4. Introduction of Zuul, also microservices? > > I strongly urge for the purpose of reviewing this feature branch that we > base much of the discussion off of > https://issues.apache.org/jira/browse/METRON-1755, the architecture > diagram. Minimally, an explanation of the current architecture along with > discussion around the additional proposed changes and rationale would be > useful for evaluation. I don't have a solid enough understanding yet of the > full scope of changes and how they differ from the existing architecture > just from looking at the PR's alone. > > 1. The first question is a general one regarding the necessity of the 3 > additional features alongside Knox - migrating Node to Spring Boot, > removing JDBC altogether, adding dependencies on Netflix's Zuul > framework. > Are these necessary for adding Knox/SSO? They seem like potentially > separate features, imo. > 2. It looks like LDAP will be a required component for interacting with > Metron via the UI's. I see this PR > https://github.com/apache/metron/pull/1186 which removes JDBC > authentication. Are we ready to remove it completely or would it be > better > to leave it as a minimal installation option? What is the proposed > migration path for existing users? Do we feel comfortable requiring that > all installations, including full dev, install and configure LDAP? For > comparison, in the PCAP feature branch we discussed removing the > existing > PCAP REST application in the initial discussion, got agreement, and > later > removed it in the course of working on the feature branch. The PR is > fairly > clear, however I think we're just missing some basic discussion around > the > implications, as I've outlined above. Some additional relevant > discussion > occurred on this PR https://github.com/apache/metron/pull/1112 which > would be good to summarize for purposes of this overarching architecture > discussion. > 3. Migration from Node to Spring Boot. I believe this is already used by > the REST application and if anything brings some cohesion to our server > strategy. Strictly speaking, is there a reason this is required for > Knox? > It seems comparable to a component upgrade, such as moving from ES 2.x > to > 5.6.x and upgrading Angular 6. > 4. Introduction of Netflix's Zuul. > https://issues.apache.org/jira/browse/METRON-1665. > - > "The UIs currently proxy to the REST API to avoid CORS issues, > this will be achieved with Zuul." > - Can we elaborate more on where or how CORS is a problem with our > existing architecture, how Zuul will help solve that, and how it > fits with > Knox? Wouldn't this be handled by Knox? Since Larry McCay chimed in > with > interest on the original SSO thread about the FB, I'm hoping he is > also > willing to chime in on this as well. > - This looks like it has the potential to be a rather large piece of > fundamental infrastructure (as it's also pertinent to microservices) > to > pull into the platform, and I'd like to be sure the community is > aware of > and is OK with the implications. > 5. > "The proposal is to use a spring boot application, allowing us to > harmonize the security implementation across the UI static servers and > the > REST layer, and to provide a routing platform for later microservices." > - > https://issues.apache.org/jira/browse/METRON-1665. > - Microservices is a pretty loaded term. I know there had been some > discussion a while back during the PCAP feature branch start, but I > don't > recall ever reaching a consensus on it. More detail in this thread - > > https://lists.apache.org/thread.html/1db7c6fa1b0f364f8c03520db9989b4f7a446de82eb4d9786055048c@%3Cdev.metron.apache.org%3E > . > Can we get some clarification on what is meant by microservices > in the case > of this FB and relevant PR's, what that architecture looks like, and > how > it's achieved with the proposed changes in this PR/FB? It seems Zuul > is > also pertinent to this discussion, but there are many ways to > skin this cat > so I don't want to presume - > > https://blog.heroku.com/using_netflix_zuul_to_proxy_your_microservices > 6. Zuul, Spring Boot, and microservices - Closely related to point 5 > above. It seems that we weren't quite ready for this when it was > brought up > in May, or at the very least we had some concern of what direction to > go. > What is the operational impact, mpack impact, and how we propose to > manage > it with Kerberos, etc.? > > https://lists.apache.org/thread.html/c19904681e6a6d9ea3131be3d1a65b24447dca31b4aff588b263fd87@%3Cdev.metron.apache.org%3E > > There is a lot to like in this feature branch, imo. Great feature addition > with Knox and SSO. Introduction of LDAP support for authentication for > Metron UI's. Simplification/unification of our server hosting > infrastructure. I'm hoping we can flesh out some of the details pointed out > above a bit more and get this feature through. Great work so far! > > Best, > Mike Miklavcic >
