I have made the necessary modifications to use my custom dispatcher.
However I am getting NPE from HttpServletRequest.getRequestURL() when the
queryString contains special characters like "amp%3B". In this case the
request object is null.
Any suggestions?
Regards,
On Thu, Sep 6, 2018 at 2:36 PM Sandeep Moré <moresand...@gmail.com> wrote:
> You can extend RMUIHaDispatch class and override getDispatchUrl() method
> from URLDecodingDispatch
>
> Best,
> Sandeep
>
> On Thu, Sep 6, 2018 at 2:28 PM Theyaa Matti <theyaama...@gmail.com> wrote:
>
>> Yes I am using HA for resource manager and in that case what should I do
>> to the fix this issue?
>>
>> Best,
>>
>> Theyaa.
>>
>>
>> On Thu, Sep 6, 2018 at 2:24 PM Sandeep Moré <moresand...@gmail.com>
>> wrote:
>>
>>> Are you using HA setup ? in which case org.apache.
>>> hadoop.gateway.rm.dispatch.RMUIHaDispatch dispatch will be used.
>>>
>>> On Thu, Sep 6, 2018 at 2:19 PM Theyaa Matti <theyaama...@gmail.com>
>>> wrote:
>>>
>>>> I created the class as follows:
>>>>
>>>> package org.apache.knox.gateway.dispatch;
>>>>
>>>> import javax.servlet.http.HttpServletRequest;
>>>> import java.net.URI;
>>>> import java.net.URLDecoder;
>>>> import org.apache.hadoop.gateway.dispatch.DefaultDispatch;
>>>>
>>>> /**
>>>> * Dispatch which decodes the outgoing URLs (to services).
>>>> * This is useful in cases where the url is picked up
>>>> * from the query parameter and is already encoded.
>>>> *
>>>> * @since 1.1.0
>>>> */
>>>> public class URLDecodingDispatch extends DefaultDispatch {
>>>>
>>>> public URLDecodingDispatch() {
>>>> super();
>>>> }
>>>>
>>>> @Override
>>>> public URI getDispatchUrl(final HttpServletRequest request) {
>>>> String decoded;
>>>>
>>>> try {
>>>> decoded =
>>>> URLDecoder.decode(request.getRequestURL().toString(), "UTF-8" );
>>>> } catch (final Exception e) {
>>>> /* fall back in case of exception */
>>>> decoded = request.getRequestURL().toString();
>>>> }
>>>>
>>>> final StringBuffer str = new StringBuffer(decoded);
>>>> final String query = request.getQueryString();
>>>> if ( query != null ) {
>>>> str.append('?');
>>>> str.append(query);
>>>> }
>>>> final URI url = URI.create(str.toString());
>>>> return url;
>>>> }
>>>> }
>>>>
>>>>
>>>> packaged it into a jar and uploaded it to the knox lib dir. I can see
>>>> Knox loading the jar file at start:
>>>> resource.FileResource (FileResource.java:checkFileAlias(152)) - ALIAS
>>>> abs=/knox/bin/../lib/knox-1.0-SNAPSHOT.jar
>>>> can=/knox/lib/knox-1.0-SNAPSHOT.jar
>>>>
>>>> I modified the service.xml for yarnui as follows.
>>>>
>>>> <dispatch classname="org.apache.knox.gateway.dispatch.URLDecodingDispatch"
>>>> ha-classname="org.apache.hadoop.gateway.rm.dispatch.RMUIHaDispatch"/>
>>>>
>>>>
>>>>
>>>> Deleted everything under deployments and restarted knox.
>>>>
>>>>
>>>> I do not see any changes in the behavior and do not see that class
>>>> being called. Do you please know what I am missing?
>>>>
>>>>
>>>> Regards,
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Sep 6, 2018 at 9:38 AM Sandeep Moré <moresand...@gmail.com>
>>>> wrote:
>>>>
>>>>> URLDecodingDispatch extends DefaultDispatch so you should be there.
>>>>>
>>>>> Best,
>>>>> Sandeep
>>>>>
>>>>> On Thu, Sep 6, 2018 at 9:32 AM Theyaa Matti <theyaama...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> That should be a good idea, but I still have an issue with adding the
>>>>>> custom dispatch to the yarnui service.xml. I already have an existing
>>>>>> class
>>>>>> there <dispatch
>>>>>> classname="org.apache.hadoop.gateway.dispatch.DefaultDispatch"
>>>>>> ha-classname="org.apache.hadoop.gateway.rm.dispatch.RMUIHaDispatch"/>
>>>>>>
>>>>>> Should I replace the DefaultDispatch or I can have more than one
>>>>>> there?
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 4, 2018 at 10:14 PM Dhruv Goyal <777.dh...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> In that case you might have to build the jar yourself or preferably
>>>>>>> use your custom dispatch similar to URLDecodingDispatch.
>>>>>>>
>>>>>>> Regards
>>>>>>> Dhruv
>>>>>>>
>>>>>>> On Wednesday, September 5, 2018, Theyaa Matti <theyaama...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Sorry 0.12.0
>>>>>>>>
>>>>>>>> On Tue, Sep 4, 2018 at 2:30 PM Theyaa Matti <theyaama...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I am using Knox 1.12.0
>>>>>>>>>
>>>>>>>>> On Tue, Sep 4, 2018 at 2:28 PM Sandeep Moré <moresand...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> You can try
>>>>>>>>>> using "org.apache.knox.gateway.dispatch.URLDecodingDispatch"
>>>>>>>>>> dispatch in
>>>>>>>>>> service.xml ( Knox 1.1.0)
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Sandeep
>>>>>>>>>>
>>>>>>>>>> On Tue, Sep 4, 2018 at 2:09 PM Theyaa Matti <
>>>>>>>>>> theyaama...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I am having issues with Knox encoding the following URL when
>>>>>>>>>>> parsing html content.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> /proxy/application_222233323323_0001/stages/stage?id=0&attempt=0
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This URL is generated by Yarn to track a running spark job. When
>>>>>>>>>>> applying regular rules to the above URL, Knox encodes the URL to:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> /proxy/application_222233323323
>>>>>>>>>>> _0001/stages/stage?amp%3Battempt=0&id=0
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Which makes the URL unusable and leads to a 404. To make the URL
>>>>>>>>>>> work I have to remove "amp%3B" and then it works.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Below are the filter and the rule I am using for this URL and I
>>>>>>>>>>> appreciate your help/comments for a remediation.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> <filter name="YARNUI/yarn/outbound/apps1">
>>>>>>>>>>> <content type="*/html">
>>>>>>>>>>> <apply path="/proxy/[^']+/stages/stage[^']*"
>>>>>>>>>>> rule="YARNUI/yarn/outbound/apps/history3"/>
>>>>>>>>>>> </content>
>>>>>>>>>>> </filter>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> <rule dir="OUT" name="YARNUI/yarn/outbound/apps/history3">
>>>>>>>>>>> <match pattern="/proxy/{*}/stages/stage?{**}"/>
>>>>>>>>>>> <rewrite
>>>>>>>>>>> template="{$frontend[url]}/yarn/proxy/{*}/stages/stage?{**}"/>
>>>>>>>>>>> </rule>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thank you for your help.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
