You can extend RMUIHaDispatch class and override getDispatchUrl() method
from URLDecodingDispatch
Best,
Sandeep
On Thu, Sep 6, 2018 at 2:28 PM Theyaa Matti <theyaama...@gmail.com> wrote:
> Yes I am using HA for resource manager and in that case what should I do
> to the fix this issue?
>
> Best,
>
> Theyaa.
>
>
> On Thu, Sep 6, 2018 at 2:24 PM Sandeep Moré <moresand...@gmail.com> wrote:
>
>> Are you using HA setup ? in which case org.apache.
>> hadoop.gateway.rm.dispatch.RMUIHaDispatch dispatch will be used.
>>
>> On Thu, Sep 6, 2018 at 2:19 PM Theyaa Matti <theyaama...@gmail.com>
>> wrote:
>>
>>> I created the class as follows:
>>>
>>> package org.apache.knox.gateway.dispatch;
>>>
>>> import javax.servlet.http.HttpServletRequest;
>>> import java.net.URI;
>>> import java.net.URLDecoder;
>>> import org.apache.hadoop.gateway.dispatch.DefaultDispatch;
>>>
>>> /**
>>> * Dispatch which decodes the outgoing URLs (to services).
>>> * This is useful in cases where the url is picked up
>>> * from the query parameter and is already encoded.
>>> *
>>> * @since 1.1.0
>>> */
>>> public class URLDecodingDispatch extends DefaultDispatch {
>>>
>>> public URLDecodingDispatch() {
>>> super();
>>> }
>>>
>>> @Override
>>> public URI getDispatchUrl(final HttpServletRequest request) {
>>> String decoded;
>>>
>>> try {
>>> decoded = URLDecoder.decode(request.getRequestURL().toString(),
>>> "UTF-8" );
>>> } catch (final Exception e) {
>>> /* fall back in case of exception */
>>> decoded = request.getRequestURL().toString();
>>> }
>>>
>>> final StringBuffer str = new StringBuffer(decoded);
>>> final String query = request.getQueryString();
>>> if ( query != null ) {
>>> str.append('?');
>>> str.append(query);
>>> }
>>> final URI url = URI.create(str.toString());
>>> return url;
>>> }
>>> }
>>>
>>>
>>> packaged it into a jar and uploaded it to the knox lib dir. I can see
>>> Knox loading the jar file at start:
>>> resource.FileResource (FileResource.java:checkFileAlias(152)) - ALIAS
>>> abs=/knox/bin/../lib/knox-1.0-SNAPSHOT.jar
>>> can=/knox/lib/knox-1.0-SNAPSHOT.jar
>>>
>>> I modified the service.xml for yarnui as follows.
>>>
>>> <dispatch classname="org.apache.knox.gateway.dispatch.URLDecodingDispatch"
>>> ha-classname="org.apache.hadoop.gateway.rm.dispatch.RMUIHaDispatch"/>
>>>
>>>
>>>
>>> Deleted everything under deployments and restarted knox.
>>>
>>>
>>> I do not see any changes in the behavior and do not see that class being
>>> called. Do you please know what I am missing?
>>>
>>>
>>> Regards,
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Sep 6, 2018 at 9:38 AM Sandeep Moré <moresand...@gmail.com>
>>> wrote:
>>>
>>>> URLDecodingDispatch extends DefaultDispatch so you should be there.
>>>>
>>>> Best,
>>>> Sandeep
>>>>
>>>> On Thu, Sep 6, 2018 at 9:32 AM Theyaa Matti <theyaama...@gmail.com>
>>>> wrote:
>>>>
>>>>> That should be a good idea, but I still have an issue with adding the
>>>>> custom dispatch to the yarnui service.xml. I already have an existing
>>>>> class
>>>>> there <dispatch
>>>>> classname="org.apache.hadoop.gateway.dispatch.DefaultDispatch"
>>>>> ha-classname="org.apache.hadoop.gateway.rm.dispatch.RMUIHaDispatch"/>
>>>>>
>>>>> Should I replace the DefaultDispatch or I can have more than one there?
>>>>>
>>>>> Regards,
>>>>>
>>>>>
>>>>> On Tue, Sep 4, 2018 at 10:14 PM Dhruv Goyal <777.dh...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> In that case you might have to build the jar yourself or preferably
>>>>>> use your custom dispatch similar to URLDecodingDispatch.
>>>>>>
>>>>>> Regards
>>>>>> Dhruv
>>>>>>
>>>>>> On Wednesday, September 5, 2018, Theyaa Matti <theyaama...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Sorry 0.12.0
>>>>>>>
>>>>>>> On Tue, Sep 4, 2018 at 2:30 PM Theyaa Matti <theyaama...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I am using Knox 1.12.0
>>>>>>>>
>>>>>>>> On Tue, Sep 4, 2018 at 2:28 PM Sandeep Moré <moresand...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> You can try
>>>>>>>>> using "org.apache.knox.gateway.dispatch.URLDecodingDispatch" dispatch
>>>>>>>>> in
>>>>>>>>> service.xml ( Knox 1.1.0)
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Sandeep
>>>>>>>>>
>>>>>>>>> On Tue, Sep 4, 2018 at 2:09 PM Theyaa Matti <theyaama...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I am having issues with Knox encoding the following URL when
>>>>>>>>>> parsing html content.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> /proxy/application_222233323323_0001/stages/stage?id=0&attempt=0
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This URL is generated by Yarn to track a running spark job. When
>>>>>>>>>> applying regular rules to the above URL, Knox encodes the URL to:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> /proxy/application_222233323323
>>>>>>>>>> _0001/stages/stage?amp%3Battempt=0&id=0
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Which makes the URL unusable and leads to a 404. To make the URL
>>>>>>>>>> work I have to remove "amp%3B" and then it works.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Below are the filter and the rule I am using for this URL and I
>>>>>>>>>> appreciate your help/comments for a remediation.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <filter name="YARNUI/yarn/outbound/apps1">
>>>>>>>>>> <content type="*/html">
>>>>>>>>>> <apply path="/proxy/[^']+/stages/stage[^']*"
>>>>>>>>>> rule="YARNUI/yarn/outbound/apps/history3"/>
>>>>>>>>>> </content>
>>>>>>>>>> </filter>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <rule dir="OUT" name="YARNUI/yarn/outbound/apps/history3">
>>>>>>>>>> <match pattern="/proxy/{*}/stages/stage?{**}"/>
>>>>>>>>>> <rewrite
>>>>>>>>>> template="{$frontend[url]}/yarn/proxy/{*}/stages/stage?{**}"/>
>>>>>>>>>> </rule>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thank you for your help.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
