[
https://issues.apache.org/jira/browse/KNOX-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776315#comment-16776315
]
Nicolas Colomer edited comment on KNOX-1743 at 2/24/19 4:41 PM:
----------------------------------------------------------------
Ok, I managed to extract audit logs for failing request (header described
[here|https://github.com/apache/knox/blob/master/gateway-util-common/src/main/java/org/apache/knox/gateway/audit/log4j/layout/AuditLayout.java]):
{noformat}
date time
root_request_id|parent_request_id|request_id|channel|target_service|username|proxy_username|system_username|action|resource_type|resource_name|outcome|message
19/02/24 16:07:01
||bc43a7dc-8111-4a78-ae96-7dc2cc348b76|audit|***.***.***.***|YARNUI||||access|uri|/gateway/emr/yarn/proxy/application_1549128656351_0631|unavailable|Request
method: GET
19/02/24 16:07:01
||bc43a7dc-8111-4a78-ae96-7dc2cc348b76|audit|***.***.***.***|YARNUI|anonymous|||authentication|uri|/gateway/emr/yarn/proxy/application_1549128656351_0631|success|
19/02/24 16:07:01
||bc43a7dc-8111-4a78-ae96-7dc2cc348b76|audit|***.***.***.***|YARNUI|anonymous|||dispatch|uri|http://emr-master-hostname:8088/proxy/application_1549128656351_0631|unavailable|Request
method: GET
19/02/24 16:07:01
||bc43a7dc-8111-4a78-ae96-7dc2cc348b76|audit|***.***.***.***|YARNUI|anonymous|||dispatch|uri|http://emr-master-hostname:8088/proxy/application_1549128656351_0631|success|Response
status: 404
19/02/24 16:07:01
|||audit|***.***.***.***|YARNUI|anonymous|||access|uri|/gateway/emr/yarn/proxy/application_1549128656351_0631|success|Response
status: 404{noformat}
We can see Knox tries to hit:
{noformat}
http://emr-master-hostname:8088/proxy/application_1549128656351_0631{noformat}
Whereas it should have tried to hit: (port 20888 instead of 8088)
{noformat}
http://emr-master-hostname:20888/proxy/application_1549128656351_0631/{noformat}
It seems port 20888 (return by yarn) is ignored and replaced by 8088 (the
yarnui port) while applying rewrite rules. And this looks to be confirmed
looking at
[yarnui/2.7.0/rewrite.xml|https://github.com/apache/knox/blob/master/gateway-service-definitions/src/main/resources/services/yarnui/2.7.0/rewrite.xml]
file (but I'm not an expert understanding this XML syntax. EDIT: ok I found
the useful [Understanding Rewrite Rules for Apache
Knox|https://cwiki.apache.org/confluence/display/KNOX/2017/08/14/Understanding+Rewrite+Rules+for+Apache+Knox#UnderstandingRewriteRulesforApacheKnox-Simpleservicerule]
documentation), with initial *host:port* being replaced by *$frontend[url]*:
{noformat}
<rule dir="OUT" name="YARNUI/yarn/outbound/proxy" pattern="*://*:*/proxy/{**}">
<rewrite template="{$frontend[url]}/yarn/proxy/{**}"/>
</rule>{noformat}
[~krisden] wdyt?
was (Author: ncolomer):
Ok, I managed to extract audit logs for failing request (header described
[here|https://github.com/apache/knox/blob/master/gateway-util-common/src/main/java/org/apache/knox/gateway/audit/log4j/layout/AuditLayout.java]):
{noformat}
date time
root_request_id|parent_request_id|request_id|channel|target_service|username|proxy_username|system_username|action|resource_type|resource_name|outcome|message
19/02/24 16:07:01
||bc43a7dc-8111-4a78-ae96-7dc2cc348b76|audit|***.***.***.***|YARNUI||||access|uri|/gateway/emr/yarn/proxy/application_1549128656351_0631|unavailable|Request
method: GET
19/02/24 16:07:01
||bc43a7dc-8111-4a78-ae96-7dc2cc348b76|audit|***.***.***.***|YARNUI|anonymous|||authentication|uri|/gateway/emr/yarn/proxy/application_1549128656351_0631|success|
19/02/24 16:07:01
||bc43a7dc-8111-4a78-ae96-7dc2cc348b76|audit|***.***.***.***|YARNUI|anonymous|||dispatch|uri|http://emr-master-hostname:8088/proxy/application_1549128656351_0631|unavailable|Request
method: GET
19/02/24 16:07:01
||bc43a7dc-8111-4a78-ae96-7dc2cc348b76|audit|***.***.***.***|YARNUI|anonymous|||dispatch|uri|http://emr-master-hostname:8088/proxy/application_1549128656351_0631|success|Response
status: 404
19/02/24 16:07:01
|||audit|***.***.***.***|YARNUI|anonymous|||access|uri|/gateway/emr/yarn/proxy/application_1549128656351_0631|success|Response
status: 404{noformat}
We can see Knox tries to hit:
{noformat}
http://emr-master-hostname:8088/proxy/application_1549128656351_0631{noformat}
Whereas it should have tried to hit: (port 20888 instead of 8088)
{noformat}
http://emr-master-hostname:20888/proxy/application_1549128656351_0631/{noformat}
It seems port 20888 (return by yarn) is ignored and replaced by 8088 (the
yarnui port) while applying rewrite rules. And this looks to be confirmed
looking at
[yarnui/2.7.0/rewrite.xml|https://github.com/apache/knox/blob/master/gateway-service-definitions/src/main/resources/services/yarnui/2.7.0/rewrite.xml]
file (but I'm not an expert understanding this XML syntax. EDIT: ok I found
the useful [Understanding Rewrite Rules for Apache
Knox|https://cwiki.apache.org/confluence/display/KNOX/2017/08/14/Understanding+Rewrite+Rules+for+Apache+Knox#UnderstandingRewriteRulesforApacheKnox-Simpleservicerule]
documentation), eg:
{noformat}
<rule dir="OUT" name="YARNUI/yarn/outbound/proxy" pattern="*://*:*/proxy/{**}">
<rewrite template="{$frontend[url]}/yarn/proxy/{**}"/>
</rule>{noformat}
[~krisden] wdyt?
> [Yarn UI Service] "Tracking UI" History link is broken
> ------------------------------------------------------
>
> Key: KNOX-1743
> URL: https://issues.apache.org/jira/browse/KNOX-1743
> Project: Apache Knox
> Issue Type: Bug
> Affects Versions: 1.2.0
> Reporter: Nicolas Colomer
> Priority: Minor
> Attachments: image-2019-01-22-12-58-43-987.png,
> image-2019-01-22-13-11-07-837.png, image-2019-02-22-15-03-57-484.png,
> image-2019-02-22-15-04-38-542.png
>
>
> The "Tracking UI" History link from the Yarn UI Service points to
> {noformat}
> https://knox/gateway/cluster/yarn/proxy/{application id}{noformat}
> But clicking the link leads to an HTTP 404 error.
> !image-2019-01-22-13-11-07-837.png!
> I traced the redirections (spaghetti) while simulating a click on the same
> link from a non-proxified Yarn UI. Here are the curl debug logs:
> {code:java}
> $ curl -vL --proxy socks5h://localhost:8157
> http://emr-master-hostname:20888/proxy/application_1547901173230_0014/
> * Trying ::1...
> * TCP_NODELAY set
> * SOCKS5 communication to emr-master-hostname:20888
> * SOCKS5 request granted.
> * Connected to localhost (::1) port 8157 (#0)
> > GET /proxy/application_1547901173230_0014/ HTTP/1.1
> > Host: emr-master-hostname:20888
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> < HTTP/1.1 302 Found
> < Cache-Control: no-cache
> < Expires: Sat, 19 Jan 2019 17:31:18 GMT
> < Date: Sat, 19 Jan 2019 17:31:18 GMT
> < Pragma: no-cache
> < Expires: Sat, 19 Jan 2019 17:31:18 GMT
> < Date: Sat, 19 Jan 2019 17:31:18 GMT
> < Pragma: no-cache
> < Content-Type: text/html; charset=utf-8
> < Location:
> http://emr-master-hostname:18080/history/application_1547901173230_0014/1
> < Content-Length: 288
> <
> * Ignoring the response-body
> * Connection #0 to host localhost left intact
> * Issue another request to this URL:
> 'http://emr-master-hostname:18080/history/application_1547901173230_0014/1'
> * Found bundle for host emr-master-hostname: 0x7f8f7b510780 [can pipeline]
> * Hostname localhost was found in DNS cache
> * Trying ::1...
> * TCP_NODELAY set
> * SOCKS5 communication to emr-master-hostname:18080
> * SOCKS5 request granted.
> * Connected to localhost (::1) port 8157 (#1)
> > GET /history/application_1547901173230_0014/1 HTTP/1.1
> > Host: emr-master-hostname:18080
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> < HTTP/1.1 302 Found
> < Date: Sat, 19 Jan 2019 17:31:18 GMT
> < Location:
> http://emr-master-hostname:18080/history/application_1547901173230_0014/1/
> < Content-Length: 0
> < Server: Jetty(9.3.z-SNAPSHOT)
> <
> * Connection #1 to host localhost left intact
> * Issue another request to this URL:
> 'http://emr-master-hostname:18080/history/application_1547901173230_0014/1/'
> * Found bundle for host emr-master-hostname: 0x7f8f7b510780 [can pipeline]
> * Re-using existing connection! (#1) with proxy localhost
> * Connected to localhost (::1) port 8157 (#1)
> > GET /history/application_1547901173230_0014/1/ HTTP/1.1
> > Host: emr-master-hostname:18080
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> < HTTP/1.1 302 Found
> < Date: Sat, 19 Jan 2019 17:31:18 GMT
> < Location:
> http://emr-master-hostname:18080/history/application_1547901173230_0014/1/jobs/
> < Content-Length: 0
> < Server: Jetty(9.3.z-SNAPSHOT)
> <
> * Connection #1 to host localhost left intact
> * Issue another request to this URL:
> 'http://emr-master-hostname:18080/history/application_1547901173230_0014/1/jobs/'
> * Found bundle for host emr-master-hostname: 0x7f8f7b510780 [can pipeline]
> * Re-using existing connection! (#1) with proxy localhost
> * Connected to localhost (::1) port 8157 (#1)
> > GET /history/application_1547901173230_0014/1/jobs/ HTTP/1.1
> > Host: emr-master-hostname:18080
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Date: Sat, 19 Jan 2019 17:31:18 GMT
> < Content-Type: text/html;charset=utf-8
> < Cache-Control: no-cache, no-store, must-revalidate
> < X-Frame-Options: SAMEORIGIN
> < X-XSS-Protection: 1; mode=block
> < X-Content-Type-Options: nosniff
> < Content-Length: 28376
> < Server: Jetty(9.3.z-SNAPSHOT)
> <
> {code}
> I guess adding some voodoo in the Knox yarnui's
> [rewrite.xml|https://github.com/apache/knox/blob/master/gateway-service-definitions/src/main/resources/services/yarnui/2.7.0/rewrite.xml]
> file might be enough to fix this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
