Re: [RESULT] [VOTE] Release Apache Knox 2.0.0 - RC 2

2023-02-24 Thread larry mccay
Congrats on the 2.0 milestone everyone - looking forward to the release! Thanks, @Phil Zampino for managing the release! On Fri, Feb 24, 2023 at 3:49 PM Phil Zampino wrote: > The VOTE for 2.0.0 RC 2 passes with: 4 binding +1's > 0 -1's I will be working on promoting 2.0.0 RC 2 to an official

Re: [VOTE] Release Apache Knox 2.0.0 - RC2

2023-02-18 Thread larry mccay
+1 for RC2 * Verified signatures * Checked NOTICE, LICENSE, CHANGES * Built from source and ran unit tests * Tested KnoxSSO, Homepage * Tested Token Management Page * Tested JWTProvider with JWT as HTTP Basic and Bearer Token * Tested APIs via KnoxShell: webhdfs, hive * Tested proxying Livy job

Re: Apache Commons Text issue - CVE 2022-42889

2022-10-24 Thread larry mccay
I do not believe so but there will be a more indepth investigation and likely an upgrade in the next release. :~/Projects/knox$ grep -r StringSubstitutor . Binary file ./install/knox-2.0.0-SNAPSHOT/dep/commons-text-1.9.jar matches It appears to only exist as the API in the lib itself. On Mon,

Re: Missing groups and/or roles claims with KnoxSSO in hadoop-jwt cookie

2022-09-26 Thread larry mccay
Hi - Thank you for your question. You are correct in that neither roles nor groups are included in the JWT token for KNOXSSO or KNOXTOKEN services. I believe we are adding (or may have already added an option) to request groups as well but there is nothing for roles. The separation of

CVE-2021-42357: DOM based XSS Vulnerability in Apache Knox

2022-01-17 Thread larry mccay
Severity: moderate Description: When using Knox SSO in affected releases, a request could be crafted to redirect a user to a malicious page due to improper URL parsing. A request that included a specially crafted request parameter could be used to redirect the user to a page controlled by an

Re: [DISCUSS] - Upgrading Log4j to 2.17.1 on Knox 1.6 line

2022-01-13 Thread larry mccay
s/dependent/vulnerable/ On Thu, Jan 13, 2022 at 10:34 AM larry mccay wrote: > We are not vulnerable to those issues as they are in log4j-core and we > don't use that in the 1.x line. > Why would we need to upgrade libs that are not dependent? > > On Thu, Jan 13, 2022 at 6:47

Re: [DISCUSS] - Upgrading Log4j to 2.17.1 on Knox 1.6 line

2022-01-13 Thread larry mccay
We are not vulnerable to those issues as they are in log4j-core and we don't use that in the 1.x line. Why would we need to upgrade libs that are not dependent? On Thu, Jan 13, 2022 at 6:47 AM Sandeep Moré wrote: > Awesome! that sounds great Sandor, thanks! > > On Thu, Jan 13, 2022 at 5:46 AM

Re: [VOTE] Release Apache Knox 1.6.1 - RC 1

2022-01-07 Thread larry mccay
Thanks for spinning this release, Sandor! * verified signatures * built from source and ran unit tests * tested KnoxSSO * Checked homepage, admin ui Barring any other issues, the copyright can wait for the next release. +1 On Wed, Jan 5, 2022 at 7:57 AM Attila Magyar wrote: > +1 from me > >

Re: Topology redeployment changes

2021-11-23 Thread larry mccay
This is an incompatible behavior change. I wonder whether we could make it configurable and an explicit opt-in behavior. I guess that the issue is that in certain deployments that a management application like Ambari may be writing out topologies based on a restart or config push even though

Re: [VOTE] Release Apache Knox 1.6.0 - RC 4

2021-11-03 Thread larry mccay
* Verified signatures * Checked NOTICE, LICENSE, CHANGES files * Downloaded and built from source * Ran unit tests * Verified basic functionality for HDFS, Hive for both the gateway server and KnoxShell client * Tested logout from home page * Tested the enable/disable of token management feature *

Re: [VOTE] Release Apache Knox 1.6.0 - RC 3

2021-11-02 Thread larry mccay
to v1.6.0, in my opinion. Here is my -1 on RC3. On Mon, Nov 1, 2021 at 3:53 PM larry mccay wrote: > Looking into an issue that may result in me withdrawing my +1. > I'll let you know when I've decided. > > > On Mon, Nov 1, 2021 at 10:08 AM Sandeep Moré > wrote:

Re: [VOTE] Release Apache Knox 1.6.0 - RC 3

2021-11-01 Thread larry mccay
Looking into an issue that may result in me withdrawing my +1. I'll let you know when I've decided. On Mon, Nov 1, 2021 at 10:08 AM Sandeep Moré wrote: > Here is my +1 > > * Downloaded and built from source > * Checked LICENSE and NOTICE files > * Verified GPG/SHA signatures for Knox source,

Re: [VOTE] Release Apache Knox 1.6.0 - RC 3

2021-10-31 Thread larry mccay
* Verified signatures * Checked NOTICE, LICENSE, CHANGES files * Downloaded and built from source * Ran unit tests * Verified basic functionality for HDFS, Hive for both the gateway server and KnoxShell client * Tested logout from home page * Tested the enable/disable of token management feature

Re: [VOTE] Release Apache Knox 1.6.0 - RC 1

2021-10-22 Thread larry mccay
Agreed - that's a blocker. On Fri, Oct 22, 2021 at 8:38 AM Sandeep Moré wrote: > Sounds great, I’ll file a JIRA with the details. > > On Fri, Oct 22, 2021 at 8:26 AM Sandor Molnar > wrote: > >> Thanks, Sandeep! >> >> I'd call this issue a release blocker and we need to file a JIRA to fix >>

Re: Proxying WebHDFS on kerberized cluster with CAS server as authenticator

2021-09-11 Thread larry mccay
Hi Tien Dat PHAN - It is indeed a valid usecase and should work. If the documentation available in the user guide [1] is not working then we may have a bug in 1.5.0. There was a regression in OIDC support due to an upgraded dependency that was out of step with one of the others. Please do let us

Re: Apache Spark History server API

2021-09-11 Thread larry mccay
Hi Jeff - Your questions are very Cloudera specific. It is inappropriate for us to discuss these things in the Apache community. I can tell you that there are multiple reasons that admins and distros provide multiple topologies in Apache Knox. One of the most common is to provide access to the

Re: Jdbc access through Knox Hive endpoint.

2021-09-04 Thread larry mccay
Hi Jeff - Can you share the errors that you are encountering? Thanks, --larry On Fri, Sep 3, 2021, 5:57 PM Jeffrey Rodriguez wrote: > Hi Apache Knox users, > Working on Hive access through Knox. In CDP cloud it > advertises endpoint > >

Re: Apache Knox RESOURCEMANAGER API not returning information

2021-08-22 Thread larry mccay
Hi Jeff - I am not sure what your deployment is set up to do here but it looks as though you are trying to send HTTP Basic credentials to a KnoxSSO protected cdp-proxy topology that is configured for SAML via pac4j. I assume that there is a cdp-proxy-api topology in your deployment as well which

Re: Troubleshooting Knox issue

2021-06-15 Thread larry mccay
Hi Ying - You didn't mention whether you added a service definition for your REST service or not. I would really expect a 404 without one but it sort of depends on what your topology looks like. If you would like to send the gateway.log, gateway-audit.log and topology files, I'll take a look.

Re: java.lang.NoClassDefFoundError in KNOX

2021-01-02 Thread larry mccay
Hi Ebrahim - This is obviously a classpath related issue that would not be caused simply by restarting the gateway. Either, you have changed jars or the command line in gateway.sh script. Given that this is a Ranger Plugin related class, I assume that you have changed that jar or introduced an

[ANNOUNCE] - Apache Knox 1.5.0 Release

2020-12-10 Thread larry mccay
The Apache Knox team is proud to announce the release of Apache Knox 1.5.0! Apache Knox is a REST API Gateway for providing secure access to the data and processing resources of Apache Hadoop clusters. More details on Apache Knox can be found at: https://knox.apache.org/ Downloads, KEYS and

Re: Method Level Authorization for Knox

2020-12-08 Thread larry mccay
Hi Ebrahim - I tried replying to the Ranger thread but my subscription seems messed up. I believe that Bosco was referring to the interface within the Ranger Knox Plugin code that would need to change ALONG with the Ranger side changes you already made. Based on what I see in [1], there is no

[RESULT] [VOTE] Release Apache Knox 1.5.0 - RC 1

2020-12-05 Thread larry mccay
The VOTE for 1.5.0 RC 1 PASSES with: 3 binding +1's 0 nonbinding +1 0 -1's I will be working on promoting 1.5.0 RC1 to an official release shortly. Thank you for taking the time to test this release and contributing to the Apache Knox community! On Sat, Dec 5, 2020 at 11:34 AM larry mccay

Re: [VOTE] Release Apache Knox 1.5.0 - RC 1

2020-12-05 Thread larry mccay
> > LGTM, > Phil > > On Sun, Nov 22, 2020 at 2:59 PM larry mccay wrote: > > > All - > > > > Release candidate #1 for the Apache Knox 1.5.0 release is available at: > > > > https://dist.apache.org/repos/dist/dev/knox/knox-1.5.0/ > > > >

Re: [VOTE] Release Apache Knox 1.5.0 - RC 1

2020-11-27 Thread larry mccay
Yes, it should. Not sure whether this is a blocker or not. Thoughts? On Wed, Nov 25, 2020, 9:59 PM Phil Zampino wrote: > Checking the NOTICE file in the RC, it has the copyright date as > 2012-20*19*. > Should it rather be 2012-20*20*? > > > > On Sun, Nov 22, 2020 at 2:59

[VOTE] Release Apache Knox 1.5.0 - RC 1

2020-11-22 Thread larry mccay
All - Release candidate #1 for the Apache Knox 1.5.0 release is available at: https://dist.apache.org/repos/dist/dev/knox/knox-1.5.0/ The release candidate is a zip archive of the sources in: https://https://gitbox.apache.org/repos/asf/knox.git Branch v1.5.0 (git checkout -b v1.5.0) The KEYS

Re: Kerberos Delegation

2020-10-06 Thread larry mccay
Hi Jeff - No, Knox is a trusted proxy in the Hadoop ecosystem and uses that pattern with Livy as well as the other ecosystem services to strongly authenticate as Knox and assert the identity of the authenticated user via doAs query param. HTH, --larry On Tue, Oct 6, 2020 at 6:27 PM jeff saremi

[ANNOUNCE] - Apache Knox 1.4.0 Release

2020-04-27 Thread larry mccay
The Apache Knox team is proud to announce the release of Apache Knox 1.4.0! Apache Knox is a REST API Gateway for providing secure access to the data and processing resources of Hadoop clusters. More details on Apache Knox can be found at: https://knox.apache.org/ Downloads, KEYS and

Re: Switching user going from KNOX to WebHDFS

2019-11-18 Thread larry mccay
remi > *Sent:* Monday, November 18, 2019 1:12 PM > *To:* larry mccay ; user@knox.apache.org < > user@knox.apache.org> > *Subject:* Re: Switching user going from KNOX to WebHDFS > > @kevin, yes we're not using Kerberos or any AD > > So you're saying that whatever user I

Re: Switching user going from KNOX to WebHDFS

2019-11-18 Thread larry mccay
Hi Jeff - Thanks for reaching out! Rather than try and unpack all of that, I'd like to get to step back to a description of what you are trying to accomplish with your deployment and the addition of Knox within it. As you have described it, it seems like a very unsecured environment. Whether

Re: [DISCUSS] Planning for Apache Knox 1.4

2019-11-14 Thread larry mccay
ovember release, but need to make sure not trying to > rush in new things just because a release will happen. There will be more > releases. > > > Kevin Risden > > > On Fri, Nov 1, 2019 at 11:51 AM Sandeep Moré > wrote: > > > Thanks for starting the planning thread Larr

Re: [DISCUSS] - Integrating Knox with Swagger

2019-11-13 Thread larry mccay
+ dev@... Thank you for the idea! Yes, I am familiar with Swagger and that would be huge for our current APIs and others that may come along. I think the effort to add a swagger filter or the like will be only one part of the larger effort of how it integrates into Knox, the site, the Admin UI,

[DISCUSS] Planning for Apache Knox 1.4

2019-10-31 Thread larry mccay
Folks - Out last release with end of July, I apologize for the delay in starting the planning thread for 1.4. We currently have a backlog of ~65 JIRAs slated for a Fix Version of 1.4. There has been some work going on within KnoxShell to provide a general purpose representation for tabular

Re: Adding a web.xml to gateway.jar

2019-09-10 Thread larry mccay
RemoteException":{"exception":"IllegalArgumentException","javaClassName":"java.lang.IllegalArgumentException","message":"Invalid > value for webhdfs parameter \"op\": STATUS is not a valid GET operation."}} >

Re: Adding a web.xml to gateway.jar

2019-09-10 Thread larry mccay
gt; > > > > Error 503 Service Unavailable > > HTTP ERROR 503 > Problem accessing /gateway/default/webhdfs/v1/. Reason: > Service Unavailablehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.12.v20180830 > > > > > --

Re: Adding a web.xml to gateway.jar

2019-09-09 Thread larry mccay
> > -- > *From:* jeff saremi > *Sent:* Saturday, September 7, 2019 4:28 PM > *To:* user@knox.apache.org > *Subject:* Re: Adding a web.xml to gateway.jar > > Great suggestions! Thanks Larry > I will work on getting the web.xml and the servlet

Re: Adding a web.xml to gateway.jar

2019-09-06 Thread larry mccay
Hi Jeff - This is an interesting idea and we should consider discussing this as a feature of Knox rather than just something that you are trying to hack into an existing release/deployment. In order to get this to work, I would first change the web.xml in the deployments directory for a given

Re: [ANNOUNCE] Apache Knox 1.3.0 Release

2019-07-25 Thread larry mccay
others interested in helping out to engage the community on the dev and users lists! --Apache Knox PMC and community On Thu, Jul 25, 2019 at 8:51 PM larry mccay wrote: > The Apache Knox team is proud to announce the release of Apache Knox 1.2.0! > > Apache Knox is a REST API Gateway for

[ANNOUNCE] Apache Knox 1.3.0 Release

2019-07-25 Thread larry mccay
The Apache Knox team is proud to announce the release of Apache Knox 1.2.0! Apache Knox is a REST API Gateway for providing secure access to the data and processing resources of Hadoop clusters. More details on Apache Knox can be found at: https://knox.apache.org/ Downloads, KEYS and

[VOTE] Release Apache Knox 1.3.0

2019-07-10 Thread larry mccay
All - A candidate for the Apache Knox 1.3.0 release is available at: https://dist.apache.org/repos/dist/dev/knox/knox-1.3.0/ The release candidate is a zip archive of the sources in: https://https://gitbox.apache.org/repos/asf/knox.git Branch v1.3.0 (git checkout -b v1.3.0) The KEYS file for

Re: KnoxSSO Logout

2019-02-21 Thread larry mccay
> > > What am I doing wrong here? Also what will be my logout url ? I tried > hitting http://:8443/gateway/knoxssout/api/v1/webssout from > browser but I see error ERR_EMPTY_RESPONSE > > > > Regards, > > Rajat > > > > *From: *larry mccay > *Reply-To

Re: KnoxSSO Logout

2019-02-21 Thread larry mccay
Hi Rajat - KNOXSSOUT will work in limited usecases and it isn't really documented or anything due to those limitations. Depending on what your actual SSO IdP is it may not work for you. Let me describe the issue in the context of a SAML provider... * SSOCookieProvider determines that there is

Re: LDAP configuration using knox.

2018-12-20 Thread larry mccay
oProvider. > Nothing happened. Access is open as it was. > > Tom > > On Thu, 20 Dec 2018 at 14:39, larry mccay wrote: > >> If you followed the proxying article and your service definition is >> indicating the anonymous authentication provider then that is the issue. >>

Re: LDAP configuration using knox.

2018-12-20 Thread larry mccay
If you followed the proxying article and your service definition is indicating the anonymous authentication provider then that is the issue. That overrides any provider configured in the topology. On Thu, Dec 20, 2018, 8:27 AM Kevin Risden If your service.xml has > >

Re: Question about shared provides & simplified descriptors

2018-12-05 Thread larry mccay
Hi Lars - It is indeed true that shared provider configs are only consumed by simple descriptors. They do so by having them pulled into the compiled topologies as you suspect. It would be good to make that clear. The answer to whether you should use one or the other is - it depends... 1. Do you

Re: [CANCEL][VOTE] Release Apache Knox 1.2.0 RC 2

2018-11-30 Thread larry mccay
doop group provider > > Kevin Risden > > > On Wed, Nov 28, 2018 at 2:41 PM larry mccay wrote: > >> All - >> >> Thanks to Kevin for so much work in cleaning up the backlog and taking on >> release manager work for 1.2.0! >> >> The 1.2.0 release happ

Re: Knox LDAP group filer is not working

2018-11-27 Thread larry mccay
Hi Raja - I need to better understand why you have a need to do the filtering within the authentication provider. This is more easily done within the authorization provider and leaves you with more options for doing group lookup. At some point, Instead of doing it from LDAP you may want to use

Re: load balancing of Hiveserver 2 throught Knox

2018-10-08 Thread larry mccay
Hi Rabii - HIve HA through Knox does not do LBing across Hive instances. It leverages the Hive ZK based HA for failover to another instance in the event of connection failures. thanks, --larry On Mon, Oct 8, 2018 at 7:25 AM rabii lamriq wrote: > Hi > > Can we configure HA and load balancing

Re: WebHDFS performance issue in Knox

2018-09-23 Thread larry mccay
> The SSL handshake can be slow if the client doesn't keep the connection >>>> open. >>>> >>>> Kevin Risden >>>> >>>> On Tue, Sep 11, 2018, 14:51 Guang Yang wrote: >>>> >>>>> Thanks Larry. But the only difference

Re: WebHDFS performance issue in Knox

2018-09-11 Thread larry mccay
I really don't think that kind of difference should be expected from merely SSL overhead. I don't however have any metrics to contradict it either since I do not run Knox without SSL. Given the above, I am struggling coming up with a meaningful response to this. :( I don't think you should see a

Re: turn off debug logging from org.apache.http.wire

2018-09-05 Thread larry mccay
log in > *gateway.out*. Seems it only affects* gateway.log*, not *gateway.out.* > > On Wed, Sep 5, 2018 at 10:48 AM, larry mccay wrote: > >> Hi Guang - >> >> This certainly sounds frustrating. >> I have never had trouble turning it off. >> Can you share you

Re: turn off debug logging from org.apache.http.wire

2018-09-05 Thread larry mccay
le all the DEBUG log >> thoroughly, so the service won't print logs to anywhere. >> >> We almost tried everything in *gateway-log4j.properties*, but it seems >> it only affects app.log.file=${launcher.name}.*log* instead of >> *gateway.out*. So, any idea guys? &g

Re: Impersonate/ProxyUser through Knox?

2018-09-01 Thread larry mccay
Hi Sean - The mechanism for doing such impersonation is through identity assertion providers. We have a number of them out of the box. In order to do this with the same sort of validation and trust configuration, a new one would likely be needed that took such configuration. You would then

Re: zeppelin via knox in HDP3 does not work for SQL

2018-08-31 Thread larry mccay
Replacing the service definition files alone isn't quite enough. You have to do the following to make sure that the server picks up the new service defs and redeploys the topology hosting the affected service: 1. change rewrite rules 2. restart gateway so that the gateway is aware of the new

Re: Reason why we don't use higher version of Jetty

2018-08-30 Thread larry mccay
t; >> Hey Larry, >> >> We're using 0.13.0 and running on Linux version 4.4.92 (Debian 4.9.2-10), >> the JDK version is 8. >> >> On Tue, Aug 28, 2018 at 1:01 PM, larry mccay wrote: >> >>> Hi Guang - >>> >>> I do recall this FD issue from

Re: Reason why we don't use higher version of Jetty

2018-08-28 Thread larry mccay
Hi Guang - I do recall this FD issue from looong ago. Not sure what was done to address it but I haven't seen it in a few years. What version of Knox are you using? What OS and JDK versions are you using? We generally upgrade jetty based on identification of CVEs on current version but also try

Re: HDP 3.0: zeppelin via knox shows empty page

2018-08-17 Thread larry mccay
Hi - Did you happen to upgrade the 2.6 cluster to 3.0? If so, this may be due to the versioned data directory not getting the updated service definition for zeppelin. Locate the previous version and the new version data directories and track down the zeppelin service definitions within

Re: Knox SSO - throwing null pointer exception on first time login

2018-08-17 Thread larry mccay
Hi Praveen - Is there no stacktrace anywhere? You are only getting the NPE line in the browser? thanks, --larry On Thu, Aug 16, 2018 at 11:52 PM, Ravikumar, Praveen Krishnamoorthy < rpkr...@amazon.com> wrote: > Attached the SAML Tracer logs for reference. Could anyone please help me > in

Re: HDP 3.0 installation problem

2018-07-31 Thread larry mccay
No, this is a new one for me. I will try and look into it but it may be a better question for Ambari than for Knox. I would also direct you to your HDP support team. On Tue, Jul 31, 2018 at 1:10 PM, Lian Jiang wrote: > I workaround this issue by pre-creating the required symlink >

Re: Knox SSO logout.

2018-07-30 Thread larry mccay
he > Set-Cookie header isn’t taking effect, do you have some insight there that > you can share? > > If that approach will not work and I must use the KNOXSSOUT service can > you share what a sample configuration might look like? > > Thanks in advance. > > Regards, > Ch

[ANNOUNCE] Apache Knox 1.1.0 Release

2018-07-30 Thread larry mccay
The Apache Knox team is proud to announce the release of Apache Knox 1.1.0! Apache Knox is a REST API Gateway for providing secure access to the data and processing resources of Hadoop clusters. More details on Apache Knox can be found at: http://knox.apache.org/ The release bits are available

Re: [VOTE] Release Apache Knox 1.1.0 RC 3

2018-07-29 Thread larry mccay
cket functionality > * Checked Topology Port Mapping feature > * Checked KnoxShell samples > * Tested HDFSUI (recent changes) > > Best, > Sandeep > > > On Wed, Jul 25, 2018 at 7:28 PM larry mccay wrote: > >> All - >> >> An issue with the OOTB

Re: IPv6 support

2018-07-27 Thread larry mccay
Well, it seems that you can certainly specify IPv6 using curl for a call to webhdfs as the following output shows: new-host-6:knox-1.1.0 lmccay$ curl -ivk6u guest:guest-password https://localhost:8443/gateway/sandbox/webhdfs/v1/tmp?op=LISTSTATUS * *Trying ::1*... * TCP_NODELAY set ** Connected

Re: Knox connection to Active Directory Federation Services?

2018-07-26 Thread larry mccay
I don't recall seeing anyone using ADFS yet. This would certainly be of interest and if you get it to work - it would be great to get a wiki tutorial for doing so! I have seen deployments with CAC cards where the challenge is done via proxy like WebGate or something like that and then Header

Re: [VOTE] Release Apache Knox 1.1.0 RC 3

2018-07-26 Thread larry mccay
min UI issues discovered in RC1 (URL field validation, > topology list refreshing) have been addressed > * Tested service discovery and topology generation > * Verified the resolution of the gateway-site.xml duplicate property issue > discovered in RC2 > > -- Phil > &

[VOTE] Release Apache Knox 1.1.0 RC 3

2018-07-25 Thread larry mccay
All - An issue with the OOTB configuration was found and subsequently fixed based on testing of RC 2. This is a minimal incremental change over the previous RC. Release candidate #3 for the Apache Knox 1.1.0 is available at: https://dist.apache.org/repos/dist/dev/knox/knox-1.1.0/ The release

Re: Integrate ranger-knox plugin.

2018-07-25 Thread larry mccay
This generally means that your configured truststore cannot be opened or is empty and is plain vanilla SSL related issue not related to Knox or Ranger specifically. On Wed, Jul 25, 2018 at 8:17 AM, Dhruv Goyal <777.dh...@gmail.com> wrote: > Hello, > > We are trying to enable ranger-knox plugin

[VOTE] Release Apache Knox 1.1.0 RC 2

2018-07-24 Thread larry mccay
All - A number of issues were identified and subsequently fixed based on testing of RC 1. Release candidate #2 for the Apache Knox 1.1.0 is available at: https://dist.apache.org/repos/dist/dev/knox/knox-1.1.0/ The release candidate is a zip archive of the sources in:

[CANCEL] [VOTE] Release Apache Knox 1.1.0 RC 1

2018-07-24 Thread larry mccay
validation (e.g., WEBHDFS, WEBHBASE) > - The Admin UI URL input field validation requires a port, which it should > not. > > While there are work-arounds for both, they adversely affect the user > experience. > > > On Sat, Jul 21, 2018 at 3:58 PM larry mccay wrote

[VOTE] Release Apache Knox 1.1.0 RC 1

2018-07-21 Thread larry mccay
All - Release candidate #1 for the Apache Knox 1.1.0 is available at: https://dist.apache.org/repos/dist/dev/knox/knox-1.1.0/ The release candidate is a zip archive of the sources in: https://git-wip-us.apache.org/repos/asf/knox.git Branch v1.1.0 (git checkout -b v1.1.0) Tag is v1.1.0-rc1 The

Re: Need help in enabling SAML auth in Apache Knox

2018-07-18 Thread larry mccay
dUrl = > null > > > > > > << *gateway-audit.log* >> > > > > 18/07/18 01:41:06 ||be6bf57b-7b96-4292-93ce-00ed574ecd6e|audit|10.89.78. > 49|YARNUIaccess|uri|/gateway/gate1/yarn/|unavailable|Request method:

Re: Need help in enabling SAML auth in Apache Knox

2018-07-17 Thread larry mccay
Whitelist - this has nothing to do with determining where to redirect - it may not allow you to redirect somewhere if it doesn't match the expression but it is not used to determine where to redirect to. Not sure why the URL would have to be rewritten when proxying. * try to access YARNUI

Re: [DISCUSS] Planning for Apache Knox 1.1.0 Release

2018-07-13 Thread larry mccay
ctively hides the issue. > > I think we should determine what's happening with this before > producing/testing a release candidate. > > > > > On Sat, Feb 24, 2018 at 12:57 PM larry mccay wrote: > > > All - > > > > Sorry for the delay on this topic. >

Re: KnoxSSO JWT authentication by third party.

2018-07-13 Thread larry mccay
on and move on as it seems to behave correctly on > subsequent HDP releases. Thanks for you help in this matter. > > Regards, > Christopher Jackson > > On Jul 12, 2018, at 11:43 PM, larry mccay wrote: > > Hi - > > I just verified that it works as expected with the

Re: Rewrite rules necessary for knox sso?

2018-07-12 Thread larry mccay
that I'm not seeing the issues that you are describing but it is a bit concerning to me that you are. Can you reproduce this in other clusters as well? On Thu, Jul 12, 2018 at 10:57 PM, larry mccay wrote: > Hi Christopher - > > Proxying and SSO together may require a rewrite rule. &g

Re: KnoxSSO JWT authentication by third party.

2018-07-12 Thread larry mccay
that there is an issue with the cluster rather that with the knox.py script though I'm not really sure what that condition is even checking. This doesn't really solve anything for you but hope it is helpful in some way. thanks, --larry On Thu, Jul 12, 2018 at 8:04 PM, larry mccay wrote: > Hi Christop

Re: Rewrite rules necessary for knox sso?

2018-07-12 Thread larry mccay
Hi Christopher - Proxying and SSO together may require a rewrite rule. I'll look at what SSO Cookie Provider is doing though. I would expect originalUrl to be the gateway url from there. Thanks, --larry On Thu, Jul 12, 2018, 4:17 PM Christopher Jackson < jackson.christopher@gmail.com>

Re: KnoxSSO JWT authentication by third party.

2018-07-12 Thread larry mccay
p/current/knox-server/data/security/master'", 'user': 'knox'} > > > This is on HDP 2.6.2 using Knox 0.12.0. I’ve created issue > https://issues.apache.org/jira/browse/AMBARI-24285 to track. > > Regards, > Christopher Jackson > > > > On Jun 27, 2018, at 7:13 P

Re: knox OS auth does not work if /tmp has noexec

2018-07-02 Thread larry mccay
Hi Lian - I haven't encountered this before. You will likely need to dig into the shiro PAM support itself if not even lower into the Pam module code. I will try and find some time to dig a bit myself. Thanks, -larry On Mon, Jul 2, 2018, 2:58 PM Lian Jiang wrote: > Hi, > > When /tmp has

Re: Knox OS authentication fail due to "password check failed for user"

2018-06-30 Thread larry mccay
Are you on the Knox host when testing with Pam tester? The accounts will need to be on the Knox host. On Sat, Jun 30, 2018, 2:22 AM Lian Jiang wrote: > I am using OS auth for knox and have verified the username and password > work: > > sudo pamtester -v knox guest authenticate > pamtester:

Re: KnoxSSO JWT authentication by third party.

2018-06-27 Thread larry mccay
made to “Advanced topology” are correctly written to disk after an > update to the config and a subsequent restart of the knox service. It seems > to just be the “Advanced knoxsso-topology” that has the issue. > > Regards, > > Christopher Jackson > > > > On Jun 27, 201

Re: KnoxSSO JWT authentication by third party.

2018-06-27 Thread larry mccay
Hi Christopher - 1) Is it possible to include additional claims that contain group information for the user from LDAP? Not currently - there are a couple issues with this appproach but I wouldn't be against a patch that optionally enables it. * There can be 100's of groups sometimes for a given

Re: livy LDAP authentication

2018-06-26 Thread larry mccay
Hi Lian - I believe Livy does kerberos authentication and also leverages a doas user in the request to determine the effective user. HTH, --larry On Tue, Jun 26, 2018 at 11:57 AM, Lian Jiang wrote: > I have a HDP 2.6 cluster which uses knox as gateway and LDAP for > authentication. I enabled

Re: Knox and MapReduce Job History Server

2018-05-09 Thread larry mccay
sden > > On Thu, May 3, 2018 at 10:10 AM, larry mccay <lmc...@apache.org> wrote: > >> This can only be addressed in Hadoop, AFAICT. >> There are so many UIs and even APIs not supporting trusted proxies and it >> is really becoming a problem. >> >> We ne

Re: Knox and MapReduce Job History Server

2018-05-03 Thread larry mccay
This can only be addressed in Hadoop, AFAICT. There are so many UIs and even APIs not supporting trusted proxies and it is really becoming a problem. We need to file JIRAs where this support is missing and potentially provide patches as it seems folks are reluctant to add proper support for it

Re: turn off debug logging from org.apache.http.wire

2018-04-15 Thread larry mccay
ei...@uber.com> wrote: > >> Interesting. Thanks Larry. I'll dig more on my side. >> >> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay <lmc...@apache.org> wrote: >> >>> No, I cannot reproduce it. >>> If you are modifying the correct gateway-log

Re: turn off debug logging from org.apache.http.wire

2018-04-15 Thread larry mccay
way.out. In fact it > outputs the actual content on the wire(security hole?) > > 06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire - > http-outgoing-2 << "[0x0][0x0 > > Let me know if you're able to repro this. > > Thanks. > > On Sat, Apr 14, 2018 a

Re: turn off debug logging from org.apache.http.wire

2018-04-14 Thread larry mccay
Hi Wei - If you look at your gateway-log4j.properties file, you should see something like the following near the top: app.log.dir=${launcher.dir}/../logs app.log.file=${launcher.name}.log app.audit.file=${launcher.name}-audit.log log4j.rootLogger=ERROR, drfa

Re: CRL (Certificate Revocation List) & mutual authentication with SSL

2018-03-19 Thread larry mccay
Hi Dominique - It does not seem that we have implemented CRL as yet. It would need to be added to the creation of the SSLContextFactory in the JettySSLService. Could I bother you to file a JIRA for this for 1.1.0 release? Feel free to contribute a patch for it as well, if you like. thanks!

Re: KnoxSSO OpenID Error: Unable to renew the session. The session store may not support this feature

2018-03-04 Thread larry mccay
There was an issue found with the google oidc integration recently. This may be the same issue, I will need to dig that up in the dev@ or user@ list and verify. On Sun, Mar 4, 2018 at 11:47 AM, Ryan H wrote: > Hi Knox Users, > > I am rethreading this error I

Re: Knox 1.0.0 Unknown Host Exception for KnoxSSO Config

2018-03-03 Thread larry mccay
'm seeing a correlation here... > > -Ryan > > On Sat, Mar 3, 2018 at 10:32 PM, larry mccay <lmc...@apache.org> wrote: > >> Hi Ryan - >> >> Welcome to Knox-ville! >> >> Going to start with a very obvious question - can you ping that host from >&g

Re: Knox 1.0.0 Unknown Host Exception for KnoxSSO Config

2018-03-03 Thread larry mccay
Hi Ryan - Welcome to Knox-ville! Going to start with a very obvious question - can you ping that host from the machine where the gateway is running? thanks, --larry On Sat, Mar 3, 2018 at 10:07 PM, Ryan H wrote: > Hi All, > > Disclaimer: I am very new to

Re: Knox SSO JWT consumption in WebSphere Liberty.

2018-02-26 Thread larry mccay
Hi Christopher - That sounds very strange is the AUTH_HEADER a standard header that I am unaware of? I will try and reproduce this. thanks, --larry On Mon, Feb 26, 2018 at 5:48 PM, Christopher Jackson < jackson.christopher@gmail.com> wrote: > Hi All, > > I have some questions around the

[DISCUSS] Planning for Apache Knox 1.1.0 Release

2018-02-24 Thread larry mccay
All - Sorry for the delay on this topic. We are going to start of this planning thread with ~85 Unresolved JIRAs in either 1.1.0 or 0.15.0 fixVersion. project = KNOX AND resolution = Unresolved AND fixVersion in (1.1.0, 0.15.0) ORDER BY priority DESC, updated DESC I will spend some time

Re: KNOX Pac4j Azure AD Open ID

2018-02-20 Thread larry mccay
and it worked! Perhaps we should change from storing the profile in a > cookie to an attribute on the session instead? > > Colm. > > On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lmc...@apache.org> wrote: > >> KnoxSSO service (WebSSOResource) uses it to redirect to the or

Re: KNOX Pac4j Azure AD Open ID

2018-02-19 Thread larry mccay
ace to extract "originalUrl" from the "pac4jRequestedUrl" > parameter and redirect to this instead? > > Colm. > > On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lmc...@apache.org> wrote: > >> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvi

Re: KNOX Pac4j Azure AD Open ID

2018-02-19 Thread larry mccay
ould > have failed. > My issue is only when I use pac4j with Oidc client and Azure AD. > > On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lmc...@apache.org> wrote: > >> It looks like you may be using ip addresses for your Knox URLs - to >> webhdfs. >> In order

Board Report

2018-02-17 Thread larry mccay
All - I have submitted the following as our quarterly board report. If any corrections or additions are needed it is still possible to edit it - so please feel free to let me know. thanks, --larry ## Description: - The Apache Knox Gateway is an Application Gateway for interacting with the

Re: KNOX Pac4j Azure AD Open ID

2018-02-16 Thread larry mccay
It looks like you may be using ip addresses for your Knox URLs - to webhdfs. In order to rule out cookie related issue can you do a couple things: 1. check whether a cookie called hadoop-jwt is actually set in your browser 2. if not, you may want to set an actual domain in your /etc/hosts or

Re: conflicting outbound rewrite rules

2017-12-27 Thread larry mccay
to the service? >> >> Thanks. >> >> On Wed, Dec 13, 2017 at 12:12 PM, larry mccay <lmc...@apache.org> wrote: >> >>> Hi Wei - >>> >>> Thank you for tracking that down! >>> >>> Yes, I believe it is a bug bu

  1   2   >