Yes I am using HA for resource manager and in that case what should I do to
the fix this issue?
Best,
Theyaa.
On Thu, Sep 6, 2018 at 2:24 PM Sandeep Moré <moresand...@gmail.com> wrote:
> Are you using HA setup ? in which case org.apache.
> hadoop.gateway.rm.dispatch.RMUIHaDispatch dispatch will be used.
>
> On Thu, Sep 6, 2018 at 2:19 PM Theyaa Matti <theyaama...@gmail.com> wrote:
>
>> I created the class as follows:
>>
>> package org.apache.knox.gateway.dispatch;
>>
>> import javax.servlet.http.HttpServletRequest;
>> import java.net.URI;
>> import java.net.URLDecoder;
>> import org.apache.hadoop.gateway.dispatch.DefaultDispatch;
>>
>> /**
>> * Dispatch which decodes the outgoing URLs (to services).
>> * This is useful in cases where the url is picked up
>> * from the query parameter and is already encoded.
>> *
>> * @since 1.1.0
>> */
>> public class URLDecodingDispatch extends DefaultDispatch {
>>
>> public URLDecodingDispatch() {
>> super();
>> }
>>
>> @Override
>> public URI getDispatchUrl(final HttpServletRequest request) {
>> String decoded;
>>
>> try {
>> decoded = URLDecoder.decode(request.getRequestURL().toString(),
>> "UTF-8" );
>> } catch (final Exception e) {
>> /* fall back in case of exception */
>> decoded = request.getRequestURL().toString();
>> }
>>
>> final StringBuffer str = new StringBuffer(decoded);
>> final String query = request.getQueryString();
>> if ( query != null ) {
>> str.append('?');
>> str.append(query);
>> }
>> final URI url = URI.create(str.toString());
>> return url;
>> }
>> }
>>
>>
>> packaged it into a jar and uploaded it to the knox lib dir. I can see
>> Knox loading the jar file at start:
>> resource.FileResource (FileResource.java:checkFileAlias(152)) - ALIAS
>> abs=/knox/bin/../lib/knox-1.0-SNAPSHOT.jar
>> can=/knox/lib/knox-1.0-SNAPSHOT.jar
>>
>> I modified the service.xml for yarnui as follows.
>>
>> <dispatch classname="org.apache.knox.gateway.dispatch.URLDecodingDispatch"
>> ha-classname="org.apache.hadoop.gateway.rm.dispatch.RMUIHaDispatch"/>
>>
>>
>>
>> Deleted everything under deployments and restarted knox.
>>
>>
>> I do not see any changes in the behavior and do not see that class being
>> called. Do you please know what I am missing?
>>
>>
>> Regards,
>>
>>
>>
>>
>>
>>
>> On Thu, Sep 6, 2018 at 9:38 AM Sandeep Moré <moresand...@gmail.com>
>> wrote:
>>
>>> URLDecodingDispatch extends DefaultDispatch so you should be there.
>>>
>>> Best,
>>> Sandeep
>>>
>>> On Thu, Sep 6, 2018 at 9:32 AM Theyaa Matti <theyaama...@gmail.com>
>>> wrote:
>>>
>>>> That should be a good idea, but I still have an issue with adding the
>>>> custom dispatch to the yarnui service.xml. I already have an existing class
>>>> there <dispatch
>>>> classname="org.apache.hadoop.gateway.dispatch.DefaultDispatch"
>>>> ha-classname="org.apache.hadoop.gateway.rm.dispatch.RMUIHaDispatch"/>
>>>>
>>>> Should I replace the DefaultDispatch or I can have more than one there?
>>>>
>>>> Regards,
>>>>
>>>>
>>>> On Tue, Sep 4, 2018 at 10:14 PM Dhruv Goyal <777.dh...@gmail.com>
>>>> wrote:
>>>>
>>>>> In that case you might have to build the jar yourself or preferably
>>>>> use your custom dispatch similar to URLDecodingDispatch.
>>>>>
>>>>> Regards
>>>>> Dhruv
>>>>>
>>>>> On Wednesday, September 5, 2018, Theyaa Matti <theyaama...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Sorry 0.12.0
>>>>>>
>>>>>> On Tue, Sep 4, 2018 at 2:30 PM Theyaa Matti <theyaama...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I am using Knox 1.12.0
>>>>>>>
>>>>>>> On Tue, Sep 4, 2018 at 2:28 PM Sandeep Moré <moresand...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> You can try
>>>>>>>> using "org.apache.knox.gateway.dispatch.URLDecodingDispatch" dispatch
>>>>>>>> in
>>>>>>>> service.xml ( Knox 1.1.0)
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Sandeep
>>>>>>>>
>>>>>>>> On Tue, Sep 4, 2018 at 2:09 PM Theyaa Matti <theyaama...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I am having issues with Knox encoding the following URL when
>>>>>>>>> parsing html content.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> /proxy/application_222233323323_0001/stages/stage?id=0&attempt=0
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This URL is generated by Yarn to track a running spark job. When
>>>>>>>>> applying regular rules to the above URL, Knox encodes the URL to:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> /proxy/application_222233323323
>>>>>>>>> _0001/stages/stage?amp%3Battempt=0&id=0
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Which makes the URL unusable and leads to a 404. To make the URL
>>>>>>>>> work I have to remove "amp%3B" and then it works.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Below are the filter and the rule I am using for this URL and I
>>>>>>>>> appreciate your help/comments for a remediation.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <filter name="YARNUI/yarn/outbound/apps1">
>>>>>>>>> <content type="*/html">
>>>>>>>>> <apply path="/proxy/[^']+/stages/stage[^']*"
>>>>>>>>> rule="YARNUI/yarn/outbound/apps/history3"/>
>>>>>>>>> </content>
>>>>>>>>> </filter>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <rule dir="OUT" name="YARNUI/yarn/outbound/apps/history3">
>>>>>>>>> <match pattern="/proxy/{*}/stages/stage?{**}"/>
>>>>>>>>> <rewrite
>>>>>>>>> template="{$frontend[url]}/yarn/proxy/{*}/stages/stage?{**}"/>
>>>>>>>>> </rule>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thank you for your help.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
