[jira] [Commented] (KNOX-791) "
[ https://issues.apache.org/jira/browse/KNOX-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946481#comment-15946481 ] Jeffrey E Rodriguez commented on KNOX-791: --- If we decide an appropriate way to solve this issue I would like to work on it. Thanks. jeff > "
[jira] [Commented] (KNOX-791) "
[
https://issues.apache.org/jira/browse/KNOX-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946480#comment-15946480
]
Jeffrey E Rodriguez commented on KNOX-791:
---
Would like bring up am issue that this Jira may have cause. Maybe the community
at large can comment.
Before KNOX-791 "amp;" used to be decoded to "&" during html parsing.
The issue is that in OUT rewrite rule we will not able to match the query
string.
Example:
given html
http://foo:99/test/q=bar”>test
rule:
{code:xml}
http://newtest/test/?{*}?{scheme}?{host}?{port}"/>
{code}
This rule above will not work since the “ amp;“ will not be matched.
The following test illustrates the issue of what we would like
to solve.
{code:title=testAmpRewrite.java|borderStyle=solid}
@Test
public void testAmpRewrite() throws Exception {
URI goodUri, badUri, outputUri;
Matcher matcher;
Matcher.Match match;
Template input, pattern, template;
badUri = new URI(
"http://foo:99/test/q=bar;); // we should be able to match
goodUri = new URI(
"http://foo:99/test/?q=bar;);
input = Parser.parseLiteral(goodUri.toString());
pattern = Parser.parseTemplate("{scheme}://{host}:{port}/test/?{*}");
template =
Parser.parseTemplate("http://newtest/test/?{*}?{scheme}?{host}?{port};);
// Match query string
String expectedUri =
"http://newtest/test/?scheme=http=foo=99=bar;;
matcher = new Matcher();
matcher.add(pattern, null);
match = matcher.match(input);
assertNotNull( match );
outputUri = Expander.expand(template, match.getParams(), null);
assertEquals("Not matched", expectedUri, outputUri.toString());//
// no match of query string ?
input = Parser.parseLiteral(badUri.toString());
match = matcher.match(input);
assertNull( match); // Shows that we are not able to match ?
}
{code}
Maybe we should made an exception for "&" on a query string. Another
alternative is to add a decode option or alternative way to decode the encoded
"&".
This is causing some issues on the Spark UI through Knox we are planning
contributing.
Thanks.
Jeff Rodriguez
> "
[jira] [Comment Edited] (KNOX-791) "
[
https://issues.apache.org/jira/browse/KNOX-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946480#comment-15946480
]
Jeffrey E Rodriguez edited comment on KNOX-791 at 3/29/17 3:43 AM:
Would like bring up am issue that this Jira may have caused. Maybe the
community at large can comment.
Before KNOX-791 "amp;" used to be decoded to "&" during html parsing.
The issue is that in OUT rewrite rule we will not able to match the query
string.
Example:
given html
http://foo:99/test/q=bar”>test
rule:
{code:xml}
http://newtest/test/?{*}?{scheme}?{host}?{port}"/>
{code}
This rule above will not work since the “ amp;“ will not be matched.
The following test illustrates the issue of what we would like
to solve.
{code:title=testAmpRewrite.java|borderStyle=solid}
@Test
public void testAmpRewrite() throws Exception {
URI goodUri, badUri, outputUri;
Matcher matcher;
Matcher.Match match;
Template input, pattern, template;
badUri = new URI(
"http://foo:99/test/q=bar;); // we should be able to match
goodUri = new URI(
"http://foo:99/test/?q=bar;);
input = Parser.parseLiteral(goodUri.toString());
pattern = Parser.parseTemplate("{scheme}://{host}:{port}/test/?{*}");
template =
Parser.parseTemplate("http://newtest/test/?{*}?{scheme}?{host}?{port};);
// Match query string
String expectedUri =
"http://newtest/test/?scheme=http=foo=99=bar;;
matcher = new Matcher();
matcher.add(pattern, null);
match = matcher.match(input);
assertNotNull( match );
outputUri = Expander.expand(template, match.getParams(), null);
assertEquals("Not matched", expectedUri, outputUri.toString());//
// no match of query string ?
input = Parser.parseLiteral(badUri.toString());
match = matcher.match(input);
assertNull( match); // Shows that we are not able to match ?
}
{code}
Maybe we should made an exception for "&" on a query string. Another
alternative is to add a decode option or alternative way to decode the encoded
"&".
This is causing some issues on the Spark UI through Knox we are planning
contributing.
Thanks.
Jeff Rodriguez
was (Author: jeffreyr97):
Would like bring up am issue that this Jira may have cause. Maybe the community
at large can comment.
Before KNOX-791 "amp;" used to be decoded to "&" during html parsing.
The issue is that in OUT rewrite rule we will not able to match the query
string.
Example:
given html
http://foo:99/test/q=bar”>test
rule:
{code:xml}
http://newtest/test/?{*}?{scheme}?{host}?{port}"/>
{code}
This rule above will not work since the “ amp;“ will not be matched.
The following test illustrates the issue of what we would like
to solve.
{code:title=testAmpRewrite.java|borderStyle=solid}
@Test
public void testAmpRewrite() throws Exception {
URI goodUri, badUri, outputUri;
Matcher matcher;
Matcher.Match match;
Template input, pattern, template;
badUri = new URI(
"http://foo:99/test/q=bar;); // we should be able to match
goodUri = new URI(
"http://foo:99/test/?q=bar;);
input = Parser.parseLiteral(goodUri.toString());
pattern = Parser.parseTemplate("{scheme}://{host}:{port}/test/?{*}");
template =
Parser.parseTemplate("http://newtest/test/?{*}?{scheme}?{host}?{port};);
// Match query string
String expectedUri =
"http://newtest/test/?scheme=http=foo=99=bar;;
matcher = new Matcher();
matcher.add(pattern, null);
match = matcher.match(input);
assertNotNull( match );
outputUri = Expander.expand(template, match.getParams(), null);
assertEquals("Not matched", expectedUri, outputUri.toString());//
// no match of query string ?
input = Parser.parseLiteral(badUri.toString());
match = matcher.match(input);
assertNull( match); // Shows that we are not able to match ?
}
{code}
Maybe we should made an exception for "&" on a query string. Another
alternative is to add a decode option or alternative way to decode the encoded
"&".
This is causing some issues on the Spark UI through Knox we are planning
contributing.
Thanks.
Jeff Rodriguez
> "
[jira] [Commented] (KNOX-751) Need rewrite function to capture href information from single page host to rewrite output of node hosts
[
https://issues.apache.org/jira/browse/KNOX-751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946455#comment-15946455
]
Larry McCay commented on KNOX-751:
--
Thanks, [~jeffreyr97] - we will certainly review.
> Need rewrite function to capture href information from single page host to
> rewrite output of node hosts
> ---
>
> Key: KNOX-751
> URL: https://issues.apache.org/jira/browse/KNOX-751
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Affects Versions: 0.7.0, 0.8.0, 0.9.0, 0.9.1
> Environment: All environment support Knox
>Reporter: Jeffrey E Rodriguez
>Assignee: Jeffrey E Rodriguez
> Fix For: 0.13.0
>
> Attachments: KNOX-751.001.patch
>
>
> User Interface page rewriting through Knox presents many challenges. One of
> the challenges is that a Web page may have more than one link to other hosts
> of a given type (e.g. Yarn may have a link to more than one host) which may
> be different from the originating one. The main issue here is that Knox maps
> a single host to a rewrite and service. If the page has multiple hosts of a
> single type the rewrite rules may not be enough without some extra
> information. For example, Yarn UI may have links to Node URLs (more than one
> node) and even through the Yarn page may be rewritten to include the schema,
> and hostname as a query parameters so the dispatcher is able to fetch the
> correct host like:
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox1?port=8042
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox2?port=8042
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox3?port=8042
> the response pages from each node need to have the right scheme, host, and
> port attached to their query parameters. We need a function that given a
> request to one of the links above can save the information needed to rewrite
> their request output with the query parameters needed for every response
> links.
> e.g.
> if we click link
> http://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox2?port=8042
> this can be rewritten as:
> “/node/node” to
> https://localhost:8443/gateway/nodeui/node/node?scheme=http?host=sandbox2?port=8042
> These functions are needed:
> $hrefUrl.ur
> $hrefUrl.host
> $hrefUrl.scheme
> $hrefUrl.port
> $hrefUrl.path
> $hrefUrl.query
> The rewrite would then be something like this:
> {code:xml}
>
>
> template="{$gateway[url]}/node/node?scheme={hrefUrl.scheme}?host={hrefUrl.host}?port={hrefUrl.port}?{**}”
>
> {code}
> This way then we are not only able to request the page as we do today but the
> output response would be rewritten to have the information needed to be
> accessed.
> There is a KNOX-618 ( patch) but after evaluation we think that relying on
> the “Host” header still doesn’t provide all the information needed such as
> the scheme. We have access to the request object so it may be easier to get
> the information directly from the href request than expecting a header. The
> “Host” header see https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
> section 14.23, only defines host and port. There is no reliable way to get
> the scheme from the response headers. Thus we can't really make use of the
> patch since we would still need the function to provide us the scheme
> function. Also there is still more information we may want such the path and
> query but those as not as critical to our effort to map all links from Yarn
> UI at this point.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
KNOX-751 - rewrite function for OUTPUT rewrite to get query parameter.
Hi folks, The usage for this solution is to be able to add an obtain an inbound request query parameter so in the OUTPUT rewrite we can access it. This allow us to attach the query parameter which may have information we may want to use on the content of the OUTPUT rewrite. In Knox, we would create one single service definition with a one url. This map one-to-one a service definition to one service. Let’s say we want to use Knox with multiple instances of a service, for example we may have multiple DataNodes UIs (on port 50075). We could support this case by: 1.Creating a topology service definition to one of the DataNodes. 2.From the UI supporting the links to multiple DataNode hosts we could do: a.Link to DataNode1 can be IN rewritten using Knox to the hostname of DataNode1, and we add the hostname to query parameter. b.Link to DataNode2 can be IN rewritten using Knox to the hostname of DataNode2. And we add the hostname to query parameter. c. ……. 3.Then when response is OUT rewritten. a.We can attach using the function of this Jira to the query parameter passed to the URI content (in this case HTML href). b.….. 4.Since we have the parameter on the query (and the links are rewritten to go through Knox) we can do the same as step 2, and forward to the host defined in query parameter thus supporting multiple instances. One advantage of this solution is to we can add a new datanode and we would be able to proxy to it without modifying the topology. One improvement for this Jira KNOX-751 would be to encrypt the value of the query parameter we are passing. I may explore opening this additional Jira. The places where I found the need of this two function are: In the HDFS UI links to DataNodes, and the links of the Yarn RM UI to their local logs. ( I will upload these into additional Jiras to update the service definitions for URI that we previously contributed). Thank you very much for your feedback. Jeffrey E Rodriguez Viana
[jira] [Updated] (KNOX-751) Need rewrite function to capture href information from single page host to rewrite output of node hosts
[
https://issues.apache.org/jira/browse/KNOX-751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey E Rodriguez updated KNOX-751:
--
Status: In Progress (was: Patch Available)
Please requesting a review by community.
This patch is needed for HDFS UI DataNode UI and Yarn UI. This UI updates will
be contributed in separate Jiras.
Thanks,
> Need rewrite function to capture href information from single page host to
> rewrite output of node hosts
> ---
>
> Key: KNOX-751
> URL: https://issues.apache.org/jira/browse/KNOX-751
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Affects Versions: 0.9.1, 0.9.0, 0.8.0, 0.7.0
> Environment: All environment support Knox
>Reporter: Jeffrey E Rodriguez
>Assignee: Jeffrey E Rodriguez
> Fix For: 0.13.0
>
> Attachments: KNOX-751.001.patch
>
>
> User Interface page rewriting through Knox presents many challenges. One of
> the challenges is that a Web page may have more than one link to other hosts
> of a given type (e.g. Yarn may have a link to more than one host) which may
> be different from the originating one. The main issue here is that Knox maps
> a single host to a rewrite and service. If the page has multiple hosts of a
> single type the rewrite rules may not be enough without some extra
> information. For example, Yarn UI may have links to Node URLs (more than one
> node) and even through the Yarn page may be rewritten to include the schema,
> and hostname as a query parameters so the dispatcher is able to fetch the
> correct host like:
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox1?port=8042
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox2?port=8042
> https://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox3?port=8042
> the response pages from each node need to have the right scheme, host, and
> port attached to their query parameters. We need a function that given a
> request to one of the links above can save the information needed to rewrite
> their request output with the query parameters needed for every response
> links.
> e.g.
> if we click link
> http://localhost:8443/gateway/nodeui/node?scheme=http?host=sandbox2?port=8042
> this can be rewritten as:
> “/node/node” to
> https://localhost:8443/gateway/nodeui/node/node?scheme=http?host=sandbox2?port=8042
> These functions are needed:
> $hrefUrl.ur
> $hrefUrl.host
> $hrefUrl.scheme
> $hrefUrl.port
> $hrefUrl.path
> $hrefUrl.query
> The rewrite would then be something like this:
> {code:xml}
>
>
> template="{$gateway[url]}/node/node?scheme={hrefUrl.scheme}?host={hrefUrl.host}?port={hrefUrl.port}?{**}”
>
> {code}
> This way then we are not only able to request the page as we do today but the
> output response would be rewritten to have the information needed to be
> accessed.
> There is a KNOX-618 ( patch) but after evaluation we think that relying on
> the “Host” header still doesn’t provide all the information needed such as
> the scheme. We have access to the request object so it may be easier to get
> the information directly from the href request than expecting a header. The
> “Host” header see https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
> section 14.23, only defines host and port. There is no reliable way to get
> the scheme from the response headers. Thus we can't really make use of the
> patch since we would still need the function to provide us the scheme
> function. Also there is still more information we may want such the path and
> query but those as not as critical to our effort to map all links from Yarn
> UI at this point.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KNOX-816) Make private inner classes static
[ https://issues.apache.org/jira/browse/KNOX-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15945472#comment-15945472 ] Sandeep More commented on KNOX-816: --- Hello [~coheigea] thanks for the patch ! I have a question, there are some inner classes that are not made static is there a reason why ? e.g. in XmlRewriteRulesDigester.java only RuleFactory.java is made static and not the others. > Make private inner classes static > - > > Key: KNOX-816 > URL: https://issues.apache.org/jira/browse/KNOX-816 > Project: Apache Knox > Issue Type: Improvement >Reporter: Colm O hEigeartaigh >Assignee: Colm O hEigeartaigh >Priority: Trivial > Fix For: 0.13.0 > > Attachments: 0001-KNOX-816-Make-private-inner-classes-static.patch > > > As discussed on KNOX-792, this JIRA is a placeholder for a patch to make all > private inner classes static for performance reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KNOX-915) Adding example PAM config for Ubuntu in Knox User Guide under PAM Based Authentication section
[ https://issues.apache.org/jira/browse/KNOX-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15945344#comment-15945344 ] Sandeep More commented on KNOX-915: --- Thanks for the patch [~kpandey] ! For some reason I wasn't able to apply the patch from the root of 'knox-site' using 'patch -p0 ' Can you create a patch from the root i.e. knox-site. > Adding example PAM config for Ubuntu in Knox User Guide under PAM Based > Authentication section > -- > > Key: KNOX-915 > URL: https://issues.apache.org/jira/browse/KNOX-915 > Project: Apache Knox > Issue Type: Improvement > Components: KnoxCLI >Affects Versions: 0.11.0 > Environment: Ubuntu-16.04-64bit >Reporter: Krishna Pandey >Priority: Minor > Labels: documentation > Fix For: 0.13.0 > > Attachments: KNOX-915_v0.12.0_v1.patch > > > Current documentation covers an example of PAM configuration on OSX. It will > be useful if we can add at least one linux variant example to help deploy > Knox in Linux environment configured for PAM based authentication. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-915) Adding example PAM config for Ubuntu in Knox User Guide under PAM Based Authentication section
[ https://issues.apache.org/jira/browse/KNOX-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep More updated KNOX-915: -- Fix Version/s: 0.13.0 > Adding example PAM config for Ubuntu in Knox User Guide under PAM Based > Authentication section > -- > > Key: KNOX-915 > URL: https://issues.apache.org/jira/browse/KNOX-915 > Project: Apache Knox > Issue Type: Improvement > Components: KnoxCLI >Affects Versions: 0.11.0 > Environment: Ubuntu-16.04-64bit >Reporter: Krishna Pandey >Priority: Minor > Labels: documentation > Fix For: 0.13.0 > > Attachments: KNOX-915_v0.12.0_v1.patch > > > Current documentation covers an example of PAM configuration on OSX. It will > be useful if we can add at least one linux variant example to help deploy > Knox in Linux environment configured for PAM based authentication. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-915) Adding example PAM config for Ubuntu in Knox User Guide under PAM Based Authentication section
[ https://issues.apache.org/jira/browse/KNOX-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Pandey updated KNOX-915: Attachment: KNOX-915_v0.12.0_v1.patch > Adding example PAM config for Ubuntu in Knox User Guide under PAM Based > Authentication section > -- > > Key: KNOX-915 > URL: https://issues.apache.org/jira/browse/KNOX-915 > Project: Apache Knox > Issue Type: Improvement > Components: KnoxCLI >Affects Versions: 0.11.0 > Environment: Ubuntu-16.04-64bit >Reporter: Krishna Pandey >Priority: Minor > Labels: documentation > Attachments: KNOX-915_v0.12.0_v1.patch > > > Current documentation covers an example of PAM configuration on OSX. It will > be useful if we can add at least one linux variant example to help deploy > Knox in Linux environment configured for PAM based authentication. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KNOX-915) Adding example PAM config for Ubuntu in Knox User Guide under PAM Based Authentication section
[
https://issues.apache.org/jira/browse/KNOX-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krishna Pandey updated KNOX-915:
Status: Patch Available (was: Open)
We need to have below entries in /etc/pam.d/login file in Ubuntu 16.04 system.
Default entries in fresh-installed system was not working for me. This should
also work for other versions of Ubuntu as well.
{noformat}
#%PAM-1.0
auth required pam_sepermit.so
# pam_selinux.so close should be the first session rule
sessionrequired pam_selinux.so close
sessionrequired pam_loginuid.so
# pam_selinux.so open should only be followed by sessions to be executed in the
user context
sessionrequired pam_selinux.so open env_params
sessionoptional pam_keyinit.so force revoke
sessionrequired pam_env.so user_readenv=1 envfile=/etc/default/locale
@include password-auth
{noformat}
> Adding example PAM config for Ubuntu in Knox User Guide under PAM Based
> Authentication section
> --
>
> Key: KNOX-915
> URL: https://issues.apache.org/jira/browse/KNOX-915
> Project: Apache Knox
> Issue Type: Improvement
> Components: KnoxCLI
>Affects Versions: 0.11.0
> Environment: Ubuntu-16.04-64bit
>Reporter: Krishna Pandey
>Priority: Minor
> Labels: documentation
>
> Current documentation covers an example of PAM configuration on OSX. It will
> be useful if we can add at least one linux variant example to help deploy
> Knox in Linux environment configured for PAM based authentication.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (KNOX-915) Adding example PAM config for Ubuntu in Knox User Guide under PAM Based Authentication section
Krishna Pandey created KNOX-915: --- Summary: Adding example PAM config for Ubuntu in Knox User Guide under PAM Based Authentication section Key: KNOX-915 URL: https://issues.apache.org/jira/browse/KNOX-915 Project: Apache Knox Issue Type: Improvement Components: KnoxCLI Affects Versions: 0.11.0 Environment: Ubuntu-16.04-64bit Reporter: Krishna Pandey Priority: Minor Current documentation covers an example of PAM configuration on OSX. It will be useful if we can add at least one linux variant example to help deploy Knox in Linux environment configured for PAM based authentication. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
