Re: Force redirect to a service instead of proxying

[thomas.mau...@etu.umontpellier.fr]

Hello, For the force redirect part this behavior is due to Hadoop itself, a html meta tag refresh the page to dfshealth automatically https://github.com/apache/hadoop/blob/2fd7cf53facec3aa649f1f2cc53f8e21c209e178/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/index.html#L2 I like this behavior since I don’t have to bother with rewrite rules to proxy the Hadoop frontend then, I can let Hadoop port open instead of putting it behind Knox and protect it with the JWTRedirectAuthenticationHandler in core-site config. That way when an unauthenticated user is trying to directly access one of Hadoop component he gets redirected to Knox so page. I was able to reproduce the same behavior with spark AuthenticationFilter and force the redirection to this service by putting a simple nginx server with this meta tag pointing to a loadbalancer in front of my spark instances. This way I can force the redirection to one of those servers.  This is for now a workaround that seems to work pretty well but kind of ugly, in the end the best solution would be to not have this 302 error on the spark API call that I mentioned in other threads.  Le 17 août 2024 à 22:49, larry mccay <larry.mc...@gmail.com> a écrit :  FYI... "I saw that HDFSUI actually does that and redirect me to the Namenode instead of proxying the frontend but I didn' t find anything regarding that in the rewrite rules" This sounds like a bug, not something that we want to try and do elsewhere. IIUC, you are saying that there is a rewrite rule that ends up redirecting the browser to the proxied service directly and not back through the Knox gateway. This would definitely be a bug in the HDFSUI rewrite rules. On Wed, Jul 31, 2024 at 7:43 AM thomas.mau...@etu.umontpellier.fr <thomas.mau...@etu.umontpellier.fr> wrote: Ok I see, because here I was trying to bypass the rewrite rules because of the issue I mentioned on this email https://lists.apache.org/thread/zj90xo30kqzb2nvopx64dh86drlrmf0r where I basically have an error on the jquery call made by sparkhistoryui to fetch the json of all the jobs. Wondering if you have any clue about that at all ? I tried to debug that but can’t seem to manage to find a pattern matching this case for any IN or OUT rule  Le 31 juil. 2024 à 13:29, Sandeep Moré <moresand...@gmail.com> a écrit :  I don't think we support request forwarding in Knox, the requests go through rewrite rules and for complex UIs with _javascript_s that links to other URLs request forwarding does not work :(  Perhaps you can try to create another service for Spark with simpler rewrite rules?  On Tue, Jul 30, 2024 at 11:19 AM thomas.mau...@etu.umontpellier.fr <thomas.mau...@etu.umontpellier.fr> wrote: Hello Sandeep, Thanks for your answer, here in this example the rewrite rule is using the host specified in the query params, but for some services as Sparkhistoryui we don’t have that, so I’m wondering if you have any idea how I could force this redirection then ? Le 30 juil. 2024 à 16:58, Sandeep Moré <moresand...@gmail.com> a écrit :  This is how it is done https://github.com/apache/knox/blob/master/gateway-service-definitions/src/main/resources/services/hdfsui/3.0.0/rewrite.xml#L23 <rule dir="IN" name="DATANODE/datanode/inbound/master/query2" pattern="*://*:*/**/hdfs/datanode/?host={host}?{**}">     <rewrite template="{host}/?{**}"/>   </rule> On Tue, Jul 30, 2024 at 10:45 AM Thomas Mauran <thomas.mau...@etu.umontpellier.fr> wrote: Hello, I am using Apache Knox and am having trouble with some rewrites rules, to avoid those is there a way to tell Knox to just redirect the user to the service he asked for instead of proxying the requests on the KnoxUI on port :8443 ? I saw that HDFSUI actually does that and redirect me to the Namenode instead of proxying the frontend but I didn' t find anything regarding that in the rewrite rules Thanks for your answers