We are using HDP-2.6.5 which comes with Knox Version v0.12, Also the HDP’s Knox packaging is a little different.
> On 24-Jul-2018, at 7:57 PM, Dhruv Goyal <[email protected]> wrote: > > Hi, > Thanks Sandeep, > > I tried using the same dispatch but still there was no luck I got this > exception: > > 2018-07-24 14:23:51,128 DEBUG hadoop.gateway > (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: > https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532438630&endTime=1532442230&topN=5&topNFunction=max&isBottomN=false > > <https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532438630&endTime=1532442230&topN=5&topNFunction=max&isBottomN=false>, > direction: IN via explicit rule: GRAFANAUI/grafana/inbound/query to URL: > http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532438630&endTime=1532442230&topNFunction=max&topN=5&isBottomN=false > > <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532438630&endTime=1532442230&topNFunction=max&topN=5&isBottomN=false> > 2018-07-24 14:23:51,129 ERROR hadoop.gateway > (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter: > java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in > escape (%) pattern - For input string: "._" > java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in > escape (%) pattern - For input string: "._" > at java.net.URLDecoder.decode(URLDecoder.java:194) > at > org.apache.hadoop.gateway.dispatch.PassAllHeadersNoEncodingDispatch.getDispatchUrl(PassAllHeadersNoEncodingDispatch.java:39) > > Is this because we are using Knox version v0.12 ? > > Regards > Dhruv > >> On 24-Jul-2018, at 6:48 PM, Sandeep Moré <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hello Dhruv, >> >> In your service definition you might want to use the >> PassAllHeadersNoEncodingDispatch >> <https://github.com/apache/knox/blob/90d4339973870fbaa9c48eeae6896fefec95035d/gateway-adapter/src/main/java/org/apache/hadoop/gateway/dispatch/PassAllHeadersNoEncodingDispatch.java> >> dispatch. >> We have a Unit Test >> <https://github.com/apache/knox/blob/64a762f531e87923d3b4ff10141b69ee78904b17/gateway-spi/src/test/java/org/apache/knox/gateway/dispatch/PassAllHeadersNoEncodingDispatchTest.java#L67> >> for this case in case you want to test it out. >> >> Handling this in rewrite rules would be tricky, the dispatch route is pretty >> simple and should work. >> >> Best, >> Sandeep >> >> On Tue, Jul 24, 2018 at 5:14 AM Dhruv Goyal <[email protected] >> <mailto:[email protected]>> wrote: >> Hi, >> >> I am trying to integrate Knox with a custom service. >> Knox version- v0.12 >> >> I have many rest calls which contain % as the special characters and I want >> to integrate these calls with Knox. >> >> Examples: >> 1./api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false >> >> 2. >> /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Timer.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false >> >> 3./api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.%.CacheMisses&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false >> >> 4. >> /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=dfs.NNTopUserOpCounts.windowMs=300000.op=__%.user=%&hostname=%&appId=namenode&instanceId=&startTime=1532406804&endTime=1532417604 >> >> There other similar calls, here in all these calls we have `%` character in >> query parameter, we are trying to use a single rule: >> >> <rule dir="IN" name="GRAFANAUI/grafana/inbound/timerq" >> pattern="*://*:*/**/grafana/*/*/*/*/*/*/timeline/metrics?{metricNames=*%*}?{**}"> >> <rewrite >> template="{$serviceUrl[GRAFANAUI]}/api/datasources/proxy/1/ws/v1/timeline/metrics?{metricNames=*%2525*}?{**}"/> >> </rule> >> >> In the rewrite template we are trying to replace % with %2525, this is >> required since we are facing some url decoding issues. >> >> So we have the query parameter "metricNames” with pattern {metricNames=*%*} >> which works, but in the rewrite template we are using {metricNames=*%2525*} >> But this wildcard isnt working, is there some other wildcard which can be >> used instead of ‘*’ in the rewrite template. >> Or is there any other way in which we can avoid url decoder issue. >> >> >> Exception observed: >> >> >> 2018-07-23 06:47:39,276 DEBUG hadoop.gateway >> (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: >> https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false >> >> <https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false>, >> direction: IN via explicit rule: GRAFANAUI/grafana/inbound/query to URL: >> http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false >> >> <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false> >> 2018-07-23 06:47:39,277 ERROR hadoop.gateway >> (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter: >> java.lang.IllegalArgumentException: Malformed escape pair at index 118: >> http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false >> >> <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false> >> java.lang.IllegalArgumentException: Malformed escape pair at index 118: >> http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false >> >> <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false> >> at java.net.URI.create(URI.java:852) >> at >> org.apache.hadoop.gateway.dispatch.AbstractGatewayDispatch.getDispatchUrl(AbstractGatewayDispatch.java:88) >> at >> org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:132) >> at >> org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:115) >> at >> org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) >> at >> org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) >> at >> org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) >> >> >> >> >> >> Thanks & Regards >> Dhruv Goyal >
