Re: Knox wrong query param encoding help

[Theyaa Matti]

I created the class as follows:

package org.apache.knox.gateway.dispatch;

import javax.servlet.http.HttpServletRequest;
import java.net.URI;
import java.net.URLDecoder;
import org.apache.hadoop.gateway.dispatch.DefaultDispatch;

/**
 * Dispatch which decodes the outgoing URLs (to services).
 * This is useful in cases where the url is picked up
 * from the query parameter and is already encoded.
 *
 * @since 1.1.0
 */
public class URLDecodingDispatch extends DefaultDispatch {

    public URLDecodingDispatch() {
        super();
    }

    @Override
    public URI getDispatchUrl(final HttpServletRequest request) {
        String decoded;

        try {
            decoded =
URLDecoder.decode(request.getRequestURL().toString(), "UTF-8" );
        } catch (final Exception e) {
      /* fall back in case of exception */
            decoded = request.getRequestURL().toString();
        }

        final StringBuffer str = new StringBuffer(decoded);
        final String query = request.getQueryString();
        if ( query != null ) {
            str.append('?');
            str.append(query);
        }
        final URI url = URI.create(str.toString());
        return url;
    }
}


packaged it into a jar and uploaded it to the knox lib dir. I can see Knox
loading the jar file at start:
resource.FileResource (FileResource.java:checkFileAlias(152)) - ALIAS
abs=/knox/bin/../lib/knox-1.0-SNAPSHOT.jar
can=/knox/lib/knox-1.0-SNAPSHOT.jar

I modified the service.xml for yarnui as follows.

<dispatch classname="org.apache.knox.gateway.dispatch.URLDecodingDispatch"
ha-classname="org.apache.hadoop.gateway.rm.dispatch.RMUIHaDispatch"/>



Deleted everything under deployments and restarted knox.


I do not see any changes in the behavior and do not see that class being
called. Do you please know what I am missing?


Regards,






On Thu, Sep 6, 2018 at 9:38 AM Sandeep Moré <moresand...@gmail.com> wrote:

> URLDecodingDispatch extends DefaultDispatch so you should be there.
>
> Best,
> Sandeep
>
> On Thu, Sep 6, 2018 at 9:32 AM Theyaa Matti <theyaama...@gmail.com> wrote:
>
>> That should be a good idea, but I still have an issue with adding the
>> custom dispatch to the yarnui service.xml. I already have an existing class
>> there <dispatch 
>> classname="org.apache.hadoop.gateway.dispatch.DefaultDispatch"
>> ha-classname="org.apache.hadoop.gateway.rm.dispatch.RMUIHaDispatch"/>
>>
>> Should I replace the DefaultDispatch or I can have more than one there?
>>
>> Regards,
>>
>>
>> On Tue, Sep 4, 2018 at 10:14 PM Dhruv Goyal <777.dh...@gmail.com> wrote:
>>
>>> In that case you might have to build the jar yourself or preferably use
>>> your custom dispatch similar to URLDecodingDispatch.
>>>
>>> Regards
>>> Dhruv
>>>
>>> On Wednesday, September 5, 2018, Theyaa Matti <theyaama...@gmail.com>
>>> wrote:
>>>
>>>> Sorry 0.12.0
>>>>
>>>> On Tue, Sep 4, 2018 at 2:30 PM Theyaa Matti <theyaama...@gmail.com>
>>>> wrote:
>>>>
>>>>> I am using Knox 1.12.0
>>>>>
>>>>> On Tue, Sep 4, 2018 at 2:28 PM Sandeep Moré <moresand...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> You can try
>>>>>> using "org.apache.knox.gateway.dispatch.URLDecodingDispatch" dispatch in
>>>>>> service.xml ( Knox 1.1.0)
>>>>>>
>>>>>> Best,
>>>>>> Sandeep
>>>>>>
>>>>>> On Tue, Sep 4, 2018 at 2:09 PM Theyaa Matti <theyaama...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I am having issues with Knox encoding the following URL when parsing
>>>>>>> html content.
>>>>>>>
>>>>>>> /proxy/application_222233323323_0001/stages/stage?id=0&amp;attempt=0
>>>>>>>
>>>>>>>
>>>>>>> This URL is generated by Yarn to track a running spark job. When
>>>>>>> applying regular rules to the above URL, Knox encodes the URL to:
>>>>>>>
>>>>>>>
>>>>>>> /proxy/application_222233323323
>>>>>>> _0001/stages/stage?amp%3Battempt=0&id=0
>>>>>>>
>>>>>>>
>>>>>>> Which makes the URL unusable and leads to a 404. To make the URL
>>>>>>> work I have to remove "amp%3B" and then it works.
>>>>>>>
>>>>>>>
>>>>>>> Below are the filter and the rule I am using for this URL and I
>>>>>>> appreciate your help/comments for a remediation.
>>>>>>>
>>>>>>>
>>>>>>> <filter name="YARNUI/yarn/outbound/apps1">
>>>>>>>     <content type="*/html">
>>>>>>>         <apply path="/proxy/[^']+/stages/stage[^']*" 
>>>>>>> rule="YARNUI/yarn/outbound/apps/history3"/>
>>>>>>>     </content>
>>>>>>> </filter>
>>>>>>>
>>>>>>>
>>>>>>> <rule dir="OUT" name="YARNUI/yarn/outbound/apps/history3">
>>>>>>>     <match pattern="/proxy/{*}/stages/stage?{**}"/>
>>>>>>>     <rewrite 
>>>>>>> template="{$frontend[url]}/yarn/proxy/{*}/stages/stage?{**}"/>
>>>>>>> </rule>
>>>>>>>
>>>>>>>
>>>>>>> Thank you for your help.
>>>>>>>
>>>>>>>
>>>>>>>