I just pushed KNOX-1394 for changing the default whitelist to align with the DEMO LDAP config in gateway-site.xml. This will realign with the dev and demo environment assumptions that Knox has always had OOTB.
I will cut an RC shortly. On Fri, Jul 20, 2018 at 12:40 AM, larry mccay <[email protected]> wrote: > Yes, along with ERROR level logging when falling back to specific host/ip > or localhost variants. > > On Thu, Jul 19, 2018, 11:48 PM Philip Zampino <[email protected]> wrote: > >> Sure, but I'm also going to implement the following for the default >> behavior (when the DEFAULT value is specified for the whitelist property): >> >> >> 1. Attempt to determine the domain from the X-Forwarded-Host header >> value >> 2. If domain could not be determined, attempt to determine the domain >> from the InetAddress.getLocalHost().getCanonicalHostName() value >> 3. If domain could not be determined, attempt to determine the domain >> from the requested host name >> 4. If the domain could be determined from any of these sources, then >> the >> default whitelist will be based on that domain >> 5. If the domain cannot be determined >> a. If the requested host name is NOT a variant of localhost, then the >> whitelist will be restricted to that specific host name >> b. Otherwise, the localhost whitelist will be the default >> >> Does that sound like a good strategy? >> >> >> On Thu, Jul 19, 2018 at 8:15 PM larry mccay <[email protected]> wrote: >> >> > Yes, does that sound appropriate to you? >> > If the LDAP config in gateway-site.xml gets updated to product the >> > whitelist would be in the same place. >> > >> > On Thu, Jul 19, 2018 at 6:26 PM, Philip Zampino <[email protected]> >> > wrote: >> > >> > > I am working on a solution for the ip address being treated as a >> hostname >> > > issue. >> > > >> > > On Thu, Jul 19, 2018 at 6:24 PM larry mccay <[email protected]> >> wrote: >> > > >> > > > Playing around a bit more, I noticed that there is nondeterministic >> > > > behavior of the default whitelist feature. >> > > > Especially on macs - since the hostname ends up being any number of >> > > things. >> > > > I have noticed the following things when there is no explicit >> whitelist >> > > > configured: >> > > > >> > > > * ip address based whitelist being derived which is treated like a >> > domain >> > > > * localhost is not supported out of the box unless the logic is >> unable >> > to >> > > > determine a domain >> > > > * sometimes my host is HW14155.home and sometimes it is >> new-host-5.home >> > > for >> > > > some reason >> > > > >> > > > Given that all of our samples and docs assume localhost and OOTB we >> are >> > > > setup for DEMO LDAP server, I propose that we at least add localhost >> > back >> > > > for OOTB. >> > > > Ip address handling may be worth tackling as well but only if we >> can do >> > > it >> > > > in a day. >> > > > >> > > > Thoughts? >> > > > >> > > > >> > > > On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <[email protected]> >> > wrote: >> > > > >> > > > > Awesome - just checked it out and I will kick off a new build >> > shortly! >> > > > > >> > > > > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré < >> [email protected] >> > > >> > > > > wrote: >> > > > > >> > > > >> Hello Larry, >> > > > >> >> > > > >> I committed the fix to master and v1.1.0, it is under the JIRA >> > > KNOX-1391 >> > > > >> <https://issues.apache.org/jira/browse/KNOX-1391>. >> > > > >> we should be good to to cut the RC, provided there are no more >> > issues >> > > ! >> > > > >> >> > > > >> Thanks ! >> > > > >> Sandeep >> > > > >> >> > > > >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay < >> [email protected]> >> > > > >> wrote: >> > > > >> >> > > > >> > Awesome, @sandeep! >> > > > >> > I'll keep an eye out. >> > > > >> > >> > > > >> > Once that lands, you can bump this thread and I'll cut the RC. >> > > > >> > Obviously, we will need it in both master and v1.1.0 branches. >> > > > >> > >> > > > >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré < >> > > [email protected]> >> > > > >> > wrote: >> > > > >> > >> > > > >> > > Hello Larry, >> > > > >> > > >> > > > >> > > Yes, I have seen those exceptions, they seem to be happening >> > > fairly >> > > > >> > > consistently and only for KnoxSSO redirects when trying to >> > access >> > > > >> admin >> > > > >> > UI, >> > > > >> > > I am taking a look at them as we speak, will open up a JIRA >> for >> > it >> > > > as >> > > > >> > well. >> > > > >> > > It would be good if we can get it in, I will try to get the >> fix >> > > out >> > > > as >> > > > >> > soon >> > > > >> > > as I can. >> > > > >> > > >> > > > >> > > Best, >> > > > >> > > Sandeep >> > > > >> > > >> > > > >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay < >> [email protected]> >> > > > >> wrote: >> > > > >> > > >> > > > >> > > > @Phil, I see a couple commits land that seem to address the >> > NPE. >> > > > >> > > > Is that correct? >> > > > >> > > > >> > > > >> > > > I have also seen an IllegalStateException during redirect >> from >> > > > >> Admin UI >> > > > >> > > to >> > > > >> > > > KnoxSSO. >> > > > >> > > > Has anyone seen this and/or is working on it - is it >> related >> > to >> > > > the >> > > > >> > NPE? >> > > > >> > > > I don't think it is since I see it more frequently and not >> > > always >> > > > >> with >> > > > >> > > the >> > > > >> > > > NPEs. >> > > > >> > > > >> > > > >> > > > I'd like to get a new RC cut by end of the week, if >> possible. >> > > > >> > > > >> > > > >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay < >> > [email protected] >> > > > >> > > > >> > wrote: >> > > > >> > > > >> > > > >> > > > > Agreed, Phil. >> > > > >> > > > > I have cut an RC but we need to address this first. I'll >> > hold >> > > > >> off on >> > > > >> > > > > announcing it. >> > > > >> > > > > >> > > > >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino < >> > > > [email protected]> >> > > > >> > > wrote: >> > > > >> > > > > >> > > > >> > > > >> During some testing of the proposed 1.1.0 code, I've >> > > discovered >> > > > >> some >> > > > >> > > > NPEs >> > > > >> > > > >> in filters (e.g., AclsAuthorizationFilter, >> > > > >> > HadoopGroupProviderFilter), >> > > > >> > > > >> which are concerning. >> > > > >> > > > >> >> > > > >> > > > >> I've committed a change to address the >> > > AclsAuthorizationFilter, >> > > > >> but >> > > > >> > > > seeing >> > > > >> > > > >> similar behavior for the HadoopGroupProviderFilter has >> > > > increased >> > > > >> my >> > > > >> > > > >> concern >> > > > >> > > > >> that there may be a more fundamental problem. >> > > > >> > > > >> In both cases, it seems that the filters are being >> invoked >> > > > prior >> > > > >> to >> > > > >> > > (or >> > > > >> > > > >> during) their respective init() methods have been >> invoked. >> > > > Thus, >> > > > >> > > members >> > > > >> > > > >> which should be initialized in the init() method are not >> > yet >> > > > >> > > > initialized. >> > > > >> > > > >> >> > > > >> > > > >> This can be consistently reproduced, though it is a bit >> of >> > a >> > > > >> pain: >> > > > >> > > > >> >> > > > >> > > > >> - Install Knox (‘ant install-test-home’, or just >> unzip >> > > > >> > > > knox-1.1.0.zip) >> > > > >> > > > >> - Start the gateway >> > > > >> > > > >> - Access the Admin UI >> > > > >> > > > >> >> > > > >> > > > >> >> > > > >> > > > >> Note that the latest 1.1.0 source has a *fix* for the >> > > > >> > > > >> AclsAuthorizationFilter NPE, but master does not yet >> have >> > > this >> > > > >> > change. >> > > > >> > > > >> This >> > > > >> > > > >> is important because that change effectively hides the >> > issue. >> > > > >> > > > >> >> > > > >> > > > >> I think we should determine what's happening with this >> > before >> > > > >> > > > >> producing/testing a release candidate. >> > > > >> > > > >> >> > > > >> > > > >> >> > > > >> > > > >> >> > > > >> > > > >> >> > > > >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay < >> > > > [email protected]> >> > > > >> > > wrote: >> > > > >> > > > >> >> > > > >> > > > >> > All - >> > > > >> > > > >> > >> > > > >> > > > >> > Sorry for the delay on this topic. >> > > > >> > > > >> > >> > > > >> > > > >> > We are going to start of this planning thread with ~85 >> > > > >> Unresolved >> > > > >> > > > JIRAs >> > > > >> > > > >> in >> > > > >> > > > >> > either 1.1.0 or 0.15.0 fixVersion. >> > > > >> > > > >> > >> > > > >> > > > >> > project = KNOX AND resolution = Unresolved AND >> fixVersion >> > > in >> > > > >> > (1.1.0, >> > > > >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC >> > > > >> > > > >> > >> > > > >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 >> to >> > > begin >> > > > >> with >> > > > >> > > and >> > > > >> > > > >> then >> > > > >> > > > >> > we will need to go through and see what is already >> taken >> > > care >> > > > >> of >> > > > >> > or >> > > > >> > > > can >> > > > >> > > > >> > wait for a 1.2.0 or later. >> > > > >> > > > >> > >> > > > >> > > > >> > I also have a couple KIPs in mind to target larger >> > > > >> features/themes >> > > > >> > > for >> > > > >> > > > >> this >> > > > >> > > > >> > release. >> > > > >> > > > >> > >> > > > >> > > > >> > Off the top of my head: >> > > > >> > > > >> > >> > > > >> > > > >> > * I think we need to address some cloud specific >> usecases >> > > and >> > > > >> plan >> > > > >> > > to >> > > > >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox >> > > > instances, >> > > > >> > Azure >> > > > >> > > > AD >> > > > >> > > > >> > integration, ID mapping from Hadoop user to IAM >> > > users/roles, >> > > > >> etc. >> > > > >> > > > >> Perhaps >> > > > >> > > > >> > some CASB-like features if they make sense. >> > > > >> > > > >> > >> > > > >> > > > >> > * I also think we need one for articulating a >> reasonable >> > > flow >> > > > >> for >> > > > >> > > > >> Logout in >> > > > >> > > > >> > KnoxSSO. There are a lot of little nuances to logout >> > across >> > > > >> > multiple >> > > > >> > > > >> apps >> > > > >> > > > >> > and between different IDPs. This will require some >> > > > discussion. >> > > > >> > > > >> > >> > > > >> > > > >> > * Another thing that has been tugging at my interest >> has >> > > been >> > > > >> the >> > > > >> > > fact >> > > > >> > > > >> that >> > > > >> > > > >> > we may be able provide some common libraries to help >> > > > ecosystem >> > > > >> > > > >> applications >> > > > >> > > > >> > uptake the trusted proxy pattern and KnoxSSO. >> > > > >> > > > >> > >> > > > >> > > > >> > Anyway, these are my initial thoughts, please feel >> free >> > to >> > > > >> raise >> > > > >> > > > >> additional >> > > > >> > > > >> > ideas/themes for KIPs, etc. >> > > > >> > > > >> > >> > > > >> > > > >> > I was thinking that we could try and target an end of >> > March >> > > > or >> > > > >> Mid >> > > > >> > > > April >> > > > >> > > > >> > 1.1.0 release. >> > > > >> > > > >> > >> > > > >> > > > >> > Thoughts? >> > > > >> > > > >> > >> > > > >> > > > >> > --larry >> > > > >> > > > >> > >> > > > >> > > > >> >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > > >> > >> > > > >> >> > > > > >> > > > > >> > > > >> > > >> > >> >
