Re: Help wanted 404 errors on static files from YarnUI and Sparkhistory UI

thomas.mau...@etu.umontpellier.fr Mon, 15 Jul 2024 13:57:13 -0700

Thank you for your answer,

I will try to play with the rewrite rules then and try to make all the links and static files work !

Le 15 juil. 2024 à 13:11, Sandeep Moré <moresand...@gmail.com> a écrit :

It should have worked :( rewrite rules are contributed by the component teams (spark and yarn in this case) since we cannot keep up with all the UI changes for these components. Perhaps you can try asking on Spark or Yarn mailing list someone might have had some luck there.

On Mon, Jul 15, 2024 at 2:09 AM thomas.mau...@etu.umontpellier.fr <thomas.mau...@etu.umontpellier.fr> wrote:

Hello Sandeep,

Thank you very much for your answer I am gonna try your method to debug that. But I am wondering, shouldn’t yarnui and sparky I work out of the box ? I am using yarn 3.3.6 and sparkui 3.5.1 ? Because having to add some new rules seems odd.

Thank you for your answer,

Thomas

Le 15 juil. 2024 à 00:23, Sandeep Moré <moresand...@gmail.com> a écrit :

Hello Thomas,

Rewrite rules are tricky to troubleshoot. The way I narrow down the problem is by turning on debug log in knox.

These are the instructions on turning on DEBUG logging for Knox https://knox.apache.org/books/knox-2-0-0/user-guide.html#Logging

The way I go about debugging is by isolating a resource file (CSS or JS) and then just using that file to tweak rewrite rules instead of focusing on the entire page.

Hopefully it works good luck.

On Thu, Jul 11, 2024 at 3:14 PM Thomas Mauran <thomas.mau...@etu.umontpellier.fr> wrote:

Hello, I am writing this email to get your help on an issue with my Apache knox configuration.

I am facing a problem on both YarnUI and Sparkhistory UI where I have to write myself rewrite rules for static files like css or js ones. For example when trying to access https://<knox>:8443/gateway/default/yarn/ I get 404 errors on the following files:

gateway-audit.log

8:17 ||aa26e33e-e97b-4ff9-a977-79f5e9643ae3|audit|<ip-address>|KNOX||||access|uri|/gateway/default/default/yarn/static/yarn.css|unavailable|Request method: GET
24/07/11 15:18:17 ||aa26e33e-e97b-4ff9-a977-79f5e9643ae3|audit|<ip-address>|KNOX|user|||authentication|uri|/gateway/default/default/yarn/static/yarn.css|success|
24/07/11 15:18:17 ||aa26e33e-e97b-4ff9-a977-79f5e9643ae3|audit|<ip-address>|KNOX|user|||access|uri|/gateway/default/default/yarn/static/yarn.css|success|Response status: 404
24/07/11 15:18:17 ||475f6c06-c90d-4d14-ae39-9d81d5a51fee|audit|<ip-address>|KNOX||||access|uri|/gateway/default/default/yarn/static/jquery/themes-1.9.1/base/jquery-ui.css|unavailable|Request method: GET
24/07/11 15:18:17 ||475f6c06-c90d-4d14-ae39-9d81d5a51fee|audit|<ip-address>|KNOX|user|||authentication|uri|/gateway/default/default/yarn/static/jquery/themes-1.9.1/base/jquery-ui.css|success|
24/07/11 15:18:17 ||475f6c06-c90d-4d14-ae39-9d81d5a51fee|audit|<ip-address>|KNOX|user|||access|uri|/gateway/default/default/yarn/static/jquery/themes-1.9.1/base/jquery-ui.css|success|Response status: 404
24/07/11 15:18:17 ||0ea69ed6-8f4c-4cd1-86ca-ba8d1d30f505|audit|<ip-address>|KNOX||||access|uri|/gateway/default/default/yarn/static/jquery/jquery-3.6.0.min.js|unavailable|Request method: GET
24/07/11 15:18:17 ||0ea69ed6-8f4c-4cd1-86ca-ba8d1d30f505|audit|<ip-address>|KNOX|user|||authentication|uri|/gateway/default/default/yarn/static/jquery/jquery-3.6.0.min.js|success|
24/07/11 15:18:17 ||0ea69ed6-8f4c-4cd1-86ca-ba8d1d30f505|audit|<ip-address>|KNOX|user|||access|uri|/gateway/default/default/yarn/static/jquery/jquery-3.6.0.min.js|success|Response status: 404
24/07/11 15:18:17 ||9e5f2aeb-cff2-42cc-ae98-858e994e4214|audit|<ip-address>|KNOX||||access|uri|/gateway/default/default/yarn/static/yarn.dt.plugins.js|unavailable|Request method: GET
24/07/11 15:18:17 ||9e5f2aeb-cff2-42cc-ae98-858e994e4214|audit|<ip-address>|KNOX|user|||authentication|uri|/gateway/default/default/yarn/static/yarn.dt.plugins.js|success|
24/07/11 15:18:17 ||9e5f2aeb-cff2-42cc-ae98-858e994e4214|audit|<ip-address>|KNOX|user|||access|uri|/gateway/default/default/yarn/static/yarn.dt.plugins.js|success|Response status: 404
24/07/11 15:18:17 ||91eb4324-6b52-41ab-8473-6aebd0fec591|audit|<ip-address>|KNOX||||access|uri|/gateway/default/default/yarn/static/dt-1.10.18/css/jquery.dataTables.css|unavailable|Request method: GET
24/07/11 15:18:17 ||91eb4324-6b52-41ab-8473-6aebd0fec591|audit|<ip-address>|KNOX|user|||authentication|uri|/gateway/default/default/yarn/static/dt-1.10.18/css/jquery.dataTables.css|success|
24/07/11 15:18:17 ||5c26cfad-9397-406a-bc66-dfa668933d9a|audit|<ip-address>|KNOX||||access|uri|/gateway/default/default/yarn/static/dt-sorting/natural.js|unavailable|Request method: GET
24/07/11 15:18:17 ||91eb4324-6b52-41ab-8473-6aebd0fec591|audit|<ip-address>|KNOX|user|||access|uri|/gateway/default/default/yarn/static/dt-1.10.18/css/jquery.dataTables.css|success|Response status: 404
24/07/11 15:18:17 ||5c26cfad-9397-406a-bc66-dfa668933d9a|audit|<ip-address>|KNOX|user|||authentication|uri|/gateway/default/default/yarn/static/dt-sorting/natural.js|success|
24/07/11 15:18:17 ||5c26cfad-9397-406a-bc66-dfa668933d9a|audit|<ip-address>|KNOX|user|||access|uri|/gateway/default/default/yarn/static/dt-sorting/natural.js|success|Response status: 404
24/07/11 15:18:17 ||3aaa7592-bf99-4c84-814b-7a985a27454c|audit|<ip-address>|KNOX||||access|uri|/gateway/default/default/yarn/static/jquery/jquery-ui-1.13.2.custom.min.js|unavailable|Request method: GET
24/07/11 15:18:17 ||3aaa7592-bf99-4c84-814b-7a985a27454c|audit|<ip-address>|KNOX|user|||authentication|uri|/gateway/default/default/yarn/static/jquery/jquery-ui-1.13.2.custom.min.js|success|
24/07/11 15:18:17 ||3aaa7592-bf99-4c84-814b-7a985a27454c|audit|<ip-address>|KNOX|user|||access|uri|/gateway/default/default/yarn/static/jquery/jquery-ui-1.13.2.custom.min.js|success|Response status: 404

...

This the page I get without JS or CSS with the following message This page will not function without _javascript_ enabled. Please enable _javascript_ on your browser.

I found a post with the same issue here but after replacing my rewrite.xml with the IBM one I still had the same problems

To fix this issue I had to change the /static rule the following way.

FROM

<rule dir="OUT" name="YARNUI/yarn/outbound/static" pattern="/static/{**}">
    <rewrite template="{$frontendUrl[url]}/static/{**}"/>
</rule>

<rule dir="OUT" name="YARNUI/yarn/outbound/static" pattern="/static/{**}">
    <rewrite template="{$serviceUrl[YARNUI]}/static/{**}"/>
</rule>

Also doing that doesn’t feel right since all the links on the Yarn UI are still broken for example clicking on the application link in the left nav bar redirects me to
https://<knox-host>:8443/default/yarn/cluster/apps

(without the gateway in front for some reason) where I get a 404 error.

The same problem seems to happen to Spark History UI. When trying to access to https://<knox>:8443/gateway/default/spark3history the only thing with a response 200 is the html document of the sparkui page but every other ressources gets a 404 error.

I added the following rule in data/services/spark3historyui/3.0.0/rewrite.xml

  <rule dir="OUT" name="SPARKHISTORYUI/outbound/static" pattern="/static/{**}">
    <rewrite template="{$serviceUrl[SPARKHISTORYUI]}/static/{**}"/>
  </rule>

which fixes the css but I am still having issues with the jquery called made to get the json of all the jobs.

Here is the output of gateway.log

2024-07-11 15:44:12,879  DEBUG knox.gateway (PortMappingHelperHandler.java:handleDefaultTopologyMapping(150)) - Default topology forward from /api/v1/applications to /gateway/default/api/v1/applications
2024-07-11 15:44:12,880 568624ae-69ea-4574-b162-3faa22c9d85e DEBUG knox.gateway (GatewayFilter.java:doFilter(126)) - Received request: GET /api/v1/applications
2024-07-11 15:44:13,356  TRACE gateway.access (AccessHandler.java:log(49)) - |||194.12.154.214|GET|https://<knox-host>:8443/api/v1/applications?limit=2147483647&status=completed|-1|404|0|477

I’m pretty sure that the problems on those 2 services are linked and that I’m missing something in my configuration but I can’ t tell what at all. To give additional informations Here is my default topology and my gateway-site.xml

I replaced my host <knox-host> for privacy issues here but I am using a real host in those files.

<topology>
    <gateway>
        <provider>
            <role>webappsec</role>
            <name>WebAppSec</name>
            <enabled>true</enabled>
    </provider>
        <provider>
            <role>hostmap</role>
            <name>static</name>
            <enabled>false</enabled>
            <param><name>localhost</name><value>sandbox,sandbox.hortonworks.com</value></param>
        </provider>

        <provider>
            <role>identity-assertion</role>
            <name>Default</name>
            <enabled>false</enabled>
    </provider>

        <provider>
            <role>federation</role>
            <name>SSOCookieProvider</name>
            <enabled>true</enabled>
            <param>
            <name>sso.authentication.provider.url</name>
            <value>https://<knox-host>:8443/gateway/knoxsso/api/v1/websso</value>
            </param>
        </provider>
</gateway>
    <service>
      <role>KNOX</role>
   </service>
     <service>
        <role>HDFSUI</role>
        <version>2.7.0</version>
                <url>https://<host>:50070</url>
     </service>
        <service>
        <role>NAMENODE</role>
        <url>https://localhost:8020</url>
        <param>
                <name>webhdfs-redirect</name>
                <value>https://<host>:8443/gateway/default/webhdfs/v1</value>
        </param>

</service>
<service>
        <role>WEBHDFS</role>
        <url>https://<host>:50070/webhdfs</url>
</service>

    <service>
        <role>YARNUI</role>
        <version>2.7.0</version>
        <url>https://<host>:8088</url>
        <param>
                <name>webyarn-redirect</name>
                <value>https://<host>:8443/gateway/default/webhdfs/v1</value>
        </param>
</service>
<service>
    <role>YARN</role>
    <url>https://<host>:8088/</url>
</service>
    <service>
            <role>HBASEUI</role>
            <url>https://<host>:60010</url>
    </service>

    <service>
        <role>SPARK3HISTORYUI</role>
        <version>3.0.0</version>
        <url>https://<host>:18080</url>
        </service>



<application>
      <name>admin-ui</name>
   </application>
</topology>

gateway-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>

    <property>
        <name>gateway.service.alias.impl</name>
        <value>org.apache.knox.gateway.services.security.impl.RemoteAliasService</value>
    </property>

    <property>
            <name>default.app.topology.name</name>
            <value>default</value>

            </property>

   <property>
        <name>gateway.port</name>
        <value>8443</value>
        <description>The HTTP port for the Gateway.</description>
    </property>

    <property>
        <name>gateway.path</name>
        <value>gateway</value>
        <description>The default context path for the gateway.</description>
    </property>

    <property>
        <name>gateway.gateway.conf.dir</name>
        <value>deployments</value>
        <description>The directory within GATEWAY_HOME that contains gateway topology files and deployments.</description>
    </property>
<property>
    <name>gateway.keystore.cert.algorithm</name>
    <value>SHA256withRSA</value>
</property>

    <property>
        <name>gateway.hadoop.kerberos.secured</name>
        <value>false</value>
        <description>Boolean flag indicating whether the Hadoop cluster protected by Gateway is secured with Kerberos</description>
    </property>

    <property>
        <name>java.security.krb5.conf</name>
        <value>/etc/knox/conf/krb5.conf</value>
        <description>Absolute path to krb5.conf file</description>
    </property>

    <property>
        <name>java.security.auth.login.config</name>
        <value>/etc/knox/conf/krb5JAASLogin.conf</value>
        <description>Absolute path to JAAS login config file</description>
    </property>

    <property>
        <name>sun.security.krb5.debug</name>
        <value>false</value>
        <description>Boolean flag indicating whether to enable debug messages for krb5 authentication</description>
    </property>

    <!-- @since 0.10 Websocket configs -->
    <property>
        <name>gateway.websocket.feature.enabled</name>
        <value>true</value>
        <description>Enable/Disable websocket feature.</description>
    </property>

    <property>
        <name>gateway.scope.cookies.feature.enabled</name>
        <value>true</value>
        <description>Enable/Disable cookie scoping feature.</description>
    </property>

    <property>
        <name>gateway.cluster.config.monitor.ambari.enabled</name>
        <value>false</value>
        <description>Enable/disable Ambari cluster configuration monitoring.</description>
    </property>

    <property>
        <name>gateway.cluster.config.monitor.ambari.interval</name>
        <value>60</value>
        <description>The interval (in seconds) for polling Ambari for cluster configuration changes.</description>
    </property>
    <!-- @since 2.0.0 WebShell configs -->
    <!-- must have websocket enabled to use webshell -->
    <property>
        <name>gateway.webshell.feature.enabled</name>
        <value>false</value>
        <description>Enable/Disable webshell feature.</description>
    </property>
    <property>
        <name>gateway.webshell.max.concurrent.sessions</name>
        <value>20</value>
        <description>Maximum number of total concurrent webshell sessions</description>
    </property>
    <property>
        <name>gateway.webshell.audit.logging.enabled</name>
        <value>false</value>
        <description>[Experimental Feature] Enable/Disable webshell command audit logging.
            NOTE: Turning this on might log secrets that might be part of
            command line arguments, please consider this before turning this on.</description>
    </property>
    <property>
        <name>gateway.webshell.read.buffer.size</name>
        <value>1024</value>
        <description>Web Shell buffer size for reading</description>
    </property>

    <!-- @since 2.0.0 websocket JWT validation configs -->
    <property>
        <name>gateway.websocket.JWT.validation.feature.enabled</name>
        <value>true</value>
        <description>Enable/Disable websocket JWT validation at websocket layer.</description>
    </property>

    <!-- @since 1.5.0 homepage logout -->
    <property>
        <name>knox.homepage.logout.enabled</name>
        <value>true</value>
        <description>Enable/disable logout from the Knox Homepage.</description>
    </property>

    <!-- @since 1.6.0 token management related properties -->
    <property>
        <name>gateway.knox.token.eviction.grace.period</name>
        <value>0</value>
        <description>A duration (in seconds) beyond a token’s expiration to wait before evicting its state. This configuration only applies when server-managed token state is enabled either in gateway-site or at the topology level.</description>
    </property>

    <!-- Knox Admin related config -->
    <property>
        <name>gateway.knox.admin.groups</name>
        <value>admin</value>
    </property>

    <!-- DEMO LDAP config for Hadoop Group Provider -->
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping</name>
        <value>org.apache.hadoop.security.LdapGroupsMapping</value>
    </property>
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping.ldap.bind.user</name>
        <value>uid=guest,ou=people,dc=hadoop,dc=apache,dc=org</value>
    </property>
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping.ldap.bind.password</name>
        <value>guest-password</value>
    </property>
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping.ldap.url</name>
        <value>ldap://localhost:33389</value>
    </property>
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping.ldap.base</name>
        <value></value>
    </property>
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping.ldap.search.filter.user</name>
        <value>(&amp;(|(objectclass=person)(objectclass=applicationProcess))(cn={0}))</value>
    </property>
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping.ldap.search.filter.group</name>
        <value>(objectclass=groupOfNames)</value>
    </property>
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping.ldap.search.attr.member</name>
        <value>member</value>
    </property>
    <property>
        <name>gateway.group.config.hadoop.security.group.mapping.ldap.search.attr.group.name</name>
        <value>cn</value>
    </property>
    <property>
        <name>gateway.dispatch.whitelist.services</name>
        <value>DATANODE,HBASEUI,HDFSUI,JOBHISTORYUI,NODEUI,YARNUI,SPARK3HISTORYUI,knoxauth</value>
        <description>The comma-delimited list of service roles for which the gateway.dispatch.whitelist should be applied.</description>
    </property>
<property>
    <name>gateway.dispatch.whitelist</name>

          <value>^https?:\/\/(localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$|^https:\/\/<host>:8443.*$|^https:\/\/<host>:50070.*$|^https:\/\/<host>:60010.*$|^https:\/\/<host>:8088.*$|^https:\/\/<host>:18080.*$</value>


    <!--<value>^https?:\/\/.*$</value>-->
    <!--<value>DEFAULT</value> -->
            <description>The whitelist to be applied for dispatches associated with the service roles specified by gateway.dispatch.whitelist.services.
    If the value is DEFAULT, a domain-based whitelist will be derived from the Knox host.</description>
</property>
<property>
        <name>gateway.frontend.url</name>
        <value>https://<host>:8443/</value>
</property>

<property>
    <name>gateway.xforwarded.enabled</name>
    <value>true</value>
</property>
<property>
    <name>gateway.server.header.enabled</name>
    <value>true</value>
</property>

    <property>
        <name>gateway.xforwarded.header.context.append.servicename</name>
        <value>LIVYSERVER</value>
        <description>Add service name to x-forward-context header for the list of services defined above.</description>
    </property>

</configuration>

My spark version

I think that there is a better way to achieve this without having to do all the manual workarounds ?

Thanks for your help

Re: Help wanted 404 errors on static files from YarnUI and Sparkhistory UI

Reply via email to