Ah ! got it now and the pattern seems perfectly fine.
In which case I think just the following should work (without any
additional rules):
service.xml
<routes>
<route path="/solr/**/**?{**}" >
<rewrite apply="SOLR/solr/inbound/query" to="request.url"/>
</route>
</routes>
And in rewrite.xml
<!--Only supporting Solr queries via Knox -->
<rule dir="IN" name="SOLR/solr/inbound/query"
pattern="*://*:*/**/solr/{collection}/{query}?{**}">
<rewrite template="{$serviceUrl[SOLR]}/{collection}/{query}?{**}"/>
</rule>
Let me know if you run into issues, I can take a look at the code.
Best,
Sandeep
On Fri, Nov 18, 2016 at 11:05 AM, McParland, John <[email protected]>
wrote:
> Hi Sandeep,
>
> thanks for your help. A part of the difficulty I'm having is that the
> "KnoxSolrIntegration" part of the URL is variable - the user sending a
> request to Knox needs to specify it (it happens to the be the collection in
> Solr which they wish to query).
>
> E.g. in a normal Solr Query;
>
> curl -X GET 'http://hdp24sandbox:8983/solr/KnoxIntegrationConfig/
> select?q=*.*&wt=json&indent=true
>
> All of the pieces after the "/solr/" are variable - the user needs to
> specify it regardless of whether going through Knox or directly to Solr.
>
> So my belief is a Knox equivalent of the above curl command is
>
> curl -k -u guest:guest-password -X GET 'https://hdp24sandbox:8443/
> gateway/sandbox/solr/KnoxIntegrationConfig/select?
> q=*.*&wt=json&indent=true'
>
> Therefore, I thought the pattern to match (in rewrite.xml) would be
>
> *://*.*//**/solr/**/**?{**}
>
> As in my head
>
> - first * = protocol (http/https)
> - second * = domain name / IP address of Knox server
> - third * = port of Knox on it's server
>
> - first ** = knox context and topology (gateway/sandbox in my example)
> - second ** = Solr collection to search (KnoxIntegrationConfig in my
> example, but something I expect the caller to provide)
> - third ** = query type for Solr (e.g. "select")
> - {**} = the search parameters for Solr (q=*.*&wt=json&indent=true in my
> example)
>
> I suspect either (or both)
> a) I've not captured my pattern correctly (but I think so....)
> b) I haven't linked the service to the re-write rule to the routes in the
> service definition correctly [as I'm not sure if/how this works from the
> tutorial]
>
> Thanks,
>
> John McParland MIET CEng | System Architect, ODSC
> Health Local and Scotland | CGI
> CGI Ltd (UK)
> Second Floor, Inovo Building, 121 George St, Glasgow, UK, G1 1RD
> M: +44 7920 183 019
> [email protected] | www.cgi-group.co.uk
>
> CGI IT UK Limited. A CGI Group Inc. Company
> Registered Office: 250 Brook Drive, Green Park, Reading RG2 6UA,
> United Kingdom. Registered in England & Wales - Number 947968
>
> CONFIDENTIALITY NOTICE: Proprietary/Confidential Information belonging to
> CGI Group Inc. and its affiliates may be contained in this message. If you
> are not a recipient indicated or intended in this message (or responsible
> for delivery of this message to such person), or you think for any reason
> that this message may have been addressed to you in error, you may not use
> or copy or deliver this message to anyone else. In such case, you should
> destroy this message and are asked to notify the sender by reply e-mail.
>
> ________________________________________
> From: Sandeep More [[email protected]]
> Sent: 18 November 2016 14:11
> To: [email protected]
> Subject: Re: Re-write rules not being applied to new service
>
> Hello John,
>
> I think you are missing the definitions for the path
> '/solr/KnoxSolrIntegration'
>
> Looking at your definition files you could perhaps have something like
> this:
>
> service.xml
>
> <routes>
> <route path="/solr/**/**?{**}" >
> <rewrite apply="SOLR/solr/inbound/query"
> to="request.url"/>
> </route>
>
> <route path="/solr/KnoxSolrIntegration**" >
> <rewrite apply="SOLR/solr/inbound/KnoxSolrIntegration"
> to="request.url"/>
> </route>
> </routes>
>
> And in rewrite.xml
>
> <!--Only supporting Solr queries via Knox -->
> <rule dir="IN" name="SOLR/solr/inbound/query"
> pattern="*://*:*/**/solr/{collection}/{query}?{**}">
> <rewrite template="{$serviceUrl[SOLR]}/
> {collection}/{query}?{**}"/>
> </rule>
> <rule dir="IN" name="SOLR/solr/inbound/KnoxSolrIntegration"
> pattern="*://*:*/**/solr/KnoxSolrIntegration**">
> <rewrite template="{$serviceUrl[SOLR]}/KnoxSolrIntegration{**}"/>
> </rule>
>
> Looks like you are missing the <route> for 'SOLR/solr/inbound/query' in
> your service.xml file (like in the above example).
>
> Couple of things that helped me were
> 1) Try to keep the rules as tight as possible e.g.
> /solr/KnoxSolrIntegration**
> instead of /solr/** (provided an example with rule #2 in the above example)
> 2) In your Service.xml file you have multiple <routes>, you could
> consolidate them into single <routes>
>
> I haven't tested locally it so apologies in advance if it does not work.
>
>
> Hope that helps !
>
> Best,
> Sandeep
>
> On Fri, Nov 18, 2016 at 7:13 AM, McParland, John <[email protected]>
> wrote:
>
> > Hi all,
> >
> > I'm working to add support for Solr's API to Apache Knox. To do so, I've
> > created a service.xml and rewrite.xml, and deployed to
> > ${GATEWAY_HOME}/data/conf/services/solr/5.5.0
> >
> > See the files here: https://github.com/mcparlandjc
> > gi/knox/tree/KNOX-528/gateway-service-definitions/src/main/r
> > esources/services/solr/5.5.0
> > I also wrote a test to check that the re-write rules work as expected
> (see
> > https://github.com/mcparlandjcgi/knox/blob/KNOX-528/gateway-
> > provider-rewrite/src/test/java/org/apache/hadoop/
> > gateway/filter/rewrite/api/UrlRewriteProcessorTest.java#L281)
> >
> > I have defined the Solr service in my sandbox topology like so;
> >
> > <service>
> > <role>SOLR</role>
> > <url>http://sandbox.hortonworks.com:8983/solr</url>
> > </service>
> >
> > However when I attempt to execute a Solr Query via Knox, I find that the
> > rule isn't applied and indeed, no matching rule is found at all.
> >
> > 2016-11-18 12:04:23,221 TRACE gateway.access (AccessHandler.java:log(49))
> > - |||158.234.220.6|GET|/gateway/sandbox/solr/KnoxSolrIntegrati
> > on/select?q=*.*&wt=json&indent=true|-1|200|0|6
> > 2016-11-18 12:04:23,231 TRACE http.request (TraceRequest.java:
> traceRequestDetails(66))
> > - ||82e1c7f7-4eb5-499d-8f29-7602652079f1|Request=GET
> > /gateway/sandbox/solr/KnoxSolrIntegration/select?q=*
> .*&wt=json&indent=true
> > Header[User-Agent]=curl/7.47.0
> > Header[Authorization]=Basic Z3Vlc3Q6Z3Vlc3QtcGFzc3dvcmQ=
> > Header[Accept]=*/*
> > Header[Host]=hdp24sandbox:8443
> > 2016-11-18 12:04:23,786 DEBUG hadoop.gateway
> (GatewayFilter.java:doFilter(116))
> > - Received request: GET /solr/KnoxSolrIntegration/select
> > 2016-11-18 12:04:23,888 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: hdp24sandbox:8443, direction: IN
> > 2016-11-18 12:04:23,889 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: 8443, direction: IN
> > 2016-11-18 12:04:23,889 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: https, direction: IN
> > 2016-11-18 12:04:25,052 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: https://hdp24sandbox:8443/gateway/sandbox/solr/
> KnoxSolrIntegrati
> > on/select?q=*.*&wt=json&indent=true, direction: IN
> > 2016-11-18 12:04:25,062 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: 158.234.220.6, direction: IN
> > 2016-11-18 12:04:25,062 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: https, direction: IN
> > 2016-11-18 12:04:25,063 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: 8443, direction: IN
> > 2016-11-18 12:04:25,063 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: hdp24sandbox:8443, direction: IN
> > 2016-11-18 12:04:25,063 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: hdp24sandbox, direction: IN
> > 2016-11-18 12:04:25,063 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: /gateway/sandbox, direction: IN
> > 2016-11-18 12:04:25,064 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: curl/7.47.0, direction: IN
> > 2016-11-18 12:04:25,069 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: Basic Z3Vlc3Q6Z3Vlc3QtcGFzc3dvcmQ=, direction: IN
> > 2016-11-18 12:04:25,070 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: */*, direction: IN
> > 2016-11-18 12:04:25,070 TRACE hadoop.gateway (UrlRewriteProcessor.java:
> rewrite(177))
> > - No rule matching URL: hdp24sandbox:8443, direction: IN
> > 2016-11-18 12:04:25,072 DEBUG hadoop.gateway (DefaultDispatch.java:
> executeOutboundRequest(120))
> > - Dispatch request: GET https://hdp24sandbox:8443/gate
> > way/sandbox/solr/KnoxSolrIntegration/select?q=*.*&wt=json&indent=true
> >
> > What I expect at the end is the Dispatch request to go to
> >
> > http://sandbox.hortonworks.com:8983/solr/KnoxSolrIntegration
> > /select?q=*.*&wt=json&indent=true
> >
> > Is there a missing relationship or problem in my service/rules which
> > prevents the rule being matched with the request URL?
> >
> > Thanks,
> >
> > John McParland MIET CEng | System Architect, ODSC
> > Health Local and Scotland | CGI
> > CGI Ltd (UK)
> > Second Floor, Inovo Building, 121 George St, Glasgow, UK, G1 1RD
> > M: +44 7920 183 019
> > [email protected] | www.cgi-group.co.uk
> >
> > CGI IT UK Limited. A CGI Group Inc. Company
> > Registered Office: 250 Brook Drive, Green Park, Reading RG2 6UA,
> > United Kingdom. Registered in England & Wales - Number 947968
> >
> > CONFIDENTIALITY NOTICE: Proprietary/Confidential Information belonging to
> > CGI Group Inc. and its affiliates may be contained in this message. If
> you
> > are not a recipient indicated or intended in this message (or responsible
> > for delivery of this message to such person), or you think for any reason
> > that this message may have been addressed to you in error, you may not
> use
> > or copy or deliver this message to anyone else. In such case, you should
> > destroy this message and are asked to notify the sender by reply e-mail.
> >
>
