[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249746#comment-16249746 ] ASF subversion and git services commented on KNOX-970: -- Commit 6b250b1930235e0eff526c1c787a2680207ae150 in knox's branch refs/heads/KNOX-998-Package_Restructuring from [~lmccay] [ https://git-wip-us.apache.org/repos/asf?p=knox.git;h=6b250b1 ] KNOX-970 - add NiFi HA dispatch to service def (Jeff Storck via lmccay) > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249745#comment-16249745 ] ASF subversion and git services commented on KNOX-970: -- Commit 89dd77886e7f9990e2b5ac2a78012c0d8dfc7cbd in knox's branch refs/heads/KNOX-998-Package_Restructuring from [~lmccay] [ https://git-wip-us.apache.org/repos/asf?p=knox.git;h=89dd778 ] KNOX-970 - Add support for proxying NiFi (Jeff Storck via lmccay) > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241151#comment-16241151 ] ASF subversion and git services commented on KNOX-970: -- Commit 6b250b1930235e0eff526c1c787a2680207ae150 in knox's branch refs/heads/master from [~lmccay] [ https://git-wip-us.apache.org/repos/asf?p=knox.git;h=6b250b1 ] KNOX-970 - add NiFi HA dispatch to service def (Jeff Storck via lmccay) > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241150#comment-16241150 ] ASF subversion and git services commented on KNOX-970: -- Commit 89dd77886e7f9990e2b5ac2a78012c0d8dfc7cbd in knox's branch refs/heads/master from [~lmccay] [ https://git-wip-us.apache.org/repos/asf?p=knox.git;h=89dd778 ] KNOX-970 - Add support for proxying NiFi (Jeff Storck via lmccay) > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241115#comment-16241115 ] Larry McCay commented on KNOX-970: -- [~jtstorck] - I am going to make the above discussed adjustments and commit. We will also need to file a JIRA for following up on a couple small details that will work fine for now but are probably a little more brittle than they can be. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240494#comment-16240494 ] Jeff Storck commented on KNOX-970: -- [~lmccay] Regarding the removal of the use-two-way-ssl attribute from the dispatch tag, it would bring NiFi's service.xml in line with the rest of the services, in terms of config. It's certainly not a critical change, but it would bring it back to the convention used in the other service.xml definitions; keeping it simple, and not explicitly setting default values. For docs, I should have something to contribute today. I'll contribute the unit tests as soon as they're ready, in a patch on a separate JIRA. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240473#comment-16240473 ] Larry McCay commented on KNOX-970: -- [~jtstorck] - I agree with the first two points for sure and can make those simple changes in the patch. Whether the setting of use-two-way-ssl should be removed from the service definition, I'm not sure whether there is any downside to it being there even though it seems redundant. We can file a followup JIRA for the unit tests for the next release. We will also need one for docs for this release. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[
https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236715#comment-16236715
]
Jeff Storck commented on KNOX-970:
--
[~lmccay] In my patch, there are a few things I need to address:
* ServiceDefinitionDeploymentContributor.addDispatchFilterForClass(), for the
method that takes the useTwoWaySsl param, sets the "useTwoWaySsl" param with
the value read from service.xml after the for loop that adds all params. I
need to move the line that sets the "default" value of "useTwoWaySsl" to before
the for loop to prevent overwriting of the "useTwoWaySsl" param if one was
defined for a service in the topology.
* In NiFi's service.xml, I'd like to add
{{ha-classname=org.apache.hadoop.gateway.dispatch.NiFiHaDispatch}} to the
dispatch element.
* In NiFi's service.xml, since useTwoWaySsl defaults to false (in
CustomDispatch) and is explicitly being set to "false", the "use-two-way-ssl"
attribute can (and should?) probably be removed from NiFi's service.xml...
Thoughts on that?
* Unit tests for the NiFi dispatch are still in the works. I've been swamped
with some other tasks, but should be able to contribute those in the next
couple days.
> Add support for proxying NiFi
> -
>
> Key: KNOX-970
> URL: https://issues.apache.org/jira/browse/KNOX-970
> Project: Apache Knox
> Issue Type: New Feature
> Components: Server
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Major
> Fix For: 0.14.0
>
> Attachments: KNOX-970-PR-9-full.patch
>
>
> Apache NiFi hosts several known UIs/APIs at various context paths (/nifi,
> /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs
> depending on individual installations/configurations of NiFi through multiple
> component versions and custom NARs.
> Knox needs to be able to proxy to all of the available context paths in NiFi
> without being configured for each one individually.
> The X-Forwarded-Context header set by Knox when proxying needs to include the
> context path at which Knox is hosted (for example, /gateway/sandbox) and the
> path at which the NiFi services are proxied (for example, nifi-web). Using
> this header with the extra context path information (from the given examples,
> /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming
> requests to the root context of the web server hosted by NiFi.
> When proxying to a secured NiFi instance/cluster set up with multi-tenancy,
> Knox also needs to set an additional header required by NiFi,
> X-ProxiedEntitiesChain, which will contain the identity of the user making
> the request to Knox. If the header is present in an incoming request to
> Knox, it must be able to take the DN from the SSL cert of the requesting
> client (two-way SSL) and add it to the value received in the header. The
> requests made from Knox to NiFi must also be made with two-way SSL so that
> NiFi can obtain the Knox server DN from its certificate. The values present
> in the X-ProxiedEntitiesChain will be used to authorize each identity
> specified in the header of the proxied request before the operation will be
> performed by NiFi.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225422#comment-16225422 ] Jeff Storck commented on KNOX-970: -- [~lmccay] I will try to update my patch with tests tomorrow, 10/31. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224183#comment-16224183 ] Larry McCay commented on KNOX-970: -- [~jtstorck] - If we want to get this into 0.14.0/1.0.0 then we will need to get some tests added in the next day or so. We are closing down in anticipation of an RC on the 31st or so. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192934#comment-16192934 ] Jeff Storck commented on KNOX-970: -- [^KNOX-970-PR-9-full.patch] New patch based on comments from [~lmccay]. Some cleanup, defaulting to unsecure (http) for NiFi to match the example service definition in sandbox.xml, and updated NiFi dispatch filter param "use-two-way-ssl" to "useTwoWaySsl". > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192440#comment-16192440 ] Larry McCay commented on KNOX-970: -- [~jtstorck] - this patch looks pretty good. One thing that bothers me a bit is the service param name being use-two-way-ssl with dashes. I would have rather have seen it be with dots but there is already a precedent set in the file ServiceDefinitionDeploymentContributor for camelCase. I think the attribute name in the service definition itself is fine with the dashes. Beyond that, I am having trouble actually building and running tests on master and need to get to the bottom of that but if you are so inclined a revision to address the above would be appreciated. Thanks for this contribution, the 2-way ssl support in dispatch is a great improvement that I can already see other uses for! > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190663#comment-16190663 ] Jeff Storck commented on KNOX-970: -- [~lmccay] I will update the NiFi dispatch to remove all "Cookie" headers. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190540#comment-16190540 ] Jeff Storck commented on KNOX-970: -- [~moresandeep] I agree with your first point, I need to add unit tests before this can be merged to master. I removed the commented configuration sections in sandbox.xml as you recommended in your second point, and updated the NIFI service by default to proxy to an unsecured NiFi instance on port 9090, to bring it in line with other service definitions in the topology. I updated the method-scoped variable "twoWaySslAlias" as you recommended in your third point, good catch! In response to your fourth point, the coercion of "anonymous" to "<>" in the X-ProxiedEntitiesChain shouldn't effect logging of Knox. It's just how the anonymous user must be represented in the X-ProxiedEntitiesChain so that NiFi knows the user being proxied was not authenticated by the proxy. In the edge case that there is a user named "anonymous", NiFi recognizes "<>" in the entities chain as an unauthenticated user. Regarding your fifth point, the dispatch does not currently have access to the configuration to know what the SSO cookie name should be, and [~lmccay] said for now I could hardcode it for now. I will update the patch regarding points 2-5 tonight. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck > Fix For: 0.14.0 > > Attachments: KNOX-970-PR-9-full.patch > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[
https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185894#comment-16185894
]
Sandeep More commented on KNOX-970:
---
Great, thanks for the patch Jeff, this is some great work !
Given the scope of the patch I would also like [~lmccay] to review it.
When I try to apply the patch it fails with the following error
{code}
knox git:(master) git apply
/Users/smore/dev/review-patches/KNOX-970-PR-9-full.patch
error:
gateway-service-nifi/src/main/java/org/apache/hadoop/gateway/dispatch/NiFIRequestModifier.java:
No such file or directory
/Users/smore/dev/review-patches/KNOX-970-PR-9-full.patch:1471: new blank line
at EOF.
+
error:
gateway-service-nifi/src/main/java/org/apache/hadoop/gateway/dispatch/NiFiRequestUtil.java:
No such file or directory
{code}
Looks like the patch did not pickup addition of new files or initial commits.
Following are my comments based on the the patch.
1. We should add UnitTests for this feature.
2. I am not sure whether we need to keep the commented out configuration
section in sandbox.xml, it definately needs to go in Knox Docs but I think we
should move it from here to keep sandbox.xml simple.
3. Just a suggestion, in class ServiceDefinitionDeploymentContributor.java ,
the variable 'twoWaySslAlias' can be changed to 'useTwoWaySsl' given it is a
value and not an alias.
4. In NiFiRequestUtil class at this line
'effectivePrincipalName.equalsIgnoreCase("anonymous")' you assign it as blank,
why ? this could affect some parts for e.g. logging anonymous users in audit.log
5. In NiFiRequestUtil class, ssoCookieName is hard coded, I think users have
the ability to change this, this could be an issue, may be [~lmccay] can keep
me honest here.
Overall looks terrific !
> Add support for proxying NiFi
> -
>
> Key: KNOX-970
> URL: https://issues.apache.org/jira/browse/KNOX-970
> Project: Apache Knox
> Issue Type: New Feature
> Components: Server
>Reporter: Jeff Storck
> Fix For: 0.14.0
>
> Attachments: KNOX-970-PR-9-full.patch
>
>
> Apache NiFi hosts several known UIs/APIs at various context paths (/nifi,
> /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs
> depending on individual installations/configurations of NiFi through multiple
> component versions and custom NARs.
> Knox needs to be able to proxy to all of the available context paths in NiFi
> without being configured for each one individually.
> The X-Forwarded-Context header set by Knox when proxying needs to include the
> context path at which Knox is hosted (for example, /gateway/sandbox) and the
> path at which the NiFi services are proxied (for example, nifi-web). Using
> this header with the extra context path information (from the given examples,
> /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming
> requests to the root context of the web server hosted by NiFi.
> When proxying to a secured NiFi instance/cluster set up with multi-tenancy,
> Knox also needs to set an additional header required by NiFi,
> X-ProxiedEntitiesChain, which will contain the identity of the user making
> the request to Knox. If the header is present in an incoming request to
> Knox, it must be able to take the DN from the SSL cert of the requesting
> client (two-way SSL) and add it to the value received in the header. The
> requests made from Knox to NiFi must also be made with two-way SSL so that
> NiFi can obtain the Knox server DN from its certificate. The values present
> in the X-ProxiedEntitiesChain will be used to authorize each identity
> specified in the header of the proxied request before the operation will be
> performed by NiFi.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148282#comment-16148282 ] Jeff Storck commented on KNOX-970: -- Submitted PR: https://github.com/apache/knox/pull/9 > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck > Fix For: 0.14.0 > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059693#comment-16059693 ] Jeff Storck commented on KNOX-970: -- NiFi will have the support added for X-Forwarded-* headers in the 1.4.0 release. I'm expecting Knox 0.13.0 to be released before NiFi 1.4.0; most likely, this contribution will be in 0.14.0. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck > Fix For: 0.13.0 > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051355#comment-16051355 ] Larry McCay commented on KNOX-970: -- I've added this to 0.13.0 release via Fix Version. If we have to push it out to 0.14.0 then it will make it in at that time. Thanks for contributing this, [~jtstorck]! > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck > Fix For: 0.13.0 > > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-970) Add support for proxying NiFi
[ https://issues.apache.org/jira/browse/KNOX-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051350#comment-16051350 ] Jeff Storck commented on KNOX-970: -- I have begun work on this JIRA, and have created a new Maven module with a custom NiFi dispatch. I will be implementing the two-way SSL connections and the creation and setting of the X-ProxiedEntitiesChain. I will also contribute example service.xml and rewrite.xml configurations to enable Knox to proxy to the root context of the web server hosted by NiFi. > Add support for proxying NiFi > - > > Key: KNOX-970 > URL: https://issues.apache.org/jira/browse/KNOX-970 > Project: Apache Knox > Issue Type: New Feature > Components: Server >Reporter: Jeff Storck > > Apache NiFi hosts several known UIs/APIs at various context paths (/nifi, > /nifi-api, /nifi-docs, etc) and several dynamically discovered UIs/APIs > depending on individual installations/configurations of NiFi through multiple > component versions and custom NARs. > Knox needs to be able to proxy to all of the available context paths in NiFi > without being configured for each one individually. > The X-Forwarded-Context header set by Knox when proxying needs to include the > context path at which Knox is hosted (for example, /gateway/sandbox) and the > path at which the NiFi services are proxied (for example, nifi-web). Using > this header with the extra context path information (from the given examples, > /gateway/sandbox/nifi-web), Knox needs to be able to rewrite URLs of incoming > requests to the root context of the web server hosted by NiFi. > When proxying to a secured NiFi instance/cluster set up with multi-tenancy, > Knox also needs to set an additional header required by NiFi, > X-ProxiedEntitiesChain, which will contain the identity of the user making > the request to Knox. If the header is present in an incoming request to > Knox, it must be able to take the DN from the SSL cert of the requesting > client (two-way SSL) and add it to the value received in the header. The > requests made from Knox to NiFi must also be made with two-way SSL so that > NiFi can obtain the Knox server DN from its certificate. The values present > in the X-ProxiedEntitiesChain will be used to authorize each identity > specified in the header of the proxied request before the operation will be > performed by NiFi. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
