Thanks for the blog Kevin, it's a very useful breakdown of the process that even I can understand. This covers everything I said about the documentation being tricky so perhaps just linking this blog somewhere in the developer guide would be helpful.
On a side note I noticed a couple of typos in there: "This will need to match the topology file’s for this service" - rogue apostrophe in files under the service.xml section "Since and are not present a default..." - missing a word, under the sandbox.xml section at the end. Kind Regards, Adam On Mon, 16 Nov 2015 at 23:47 Kevin Minder <[email protected]> wrote: > Inspired by this question I created this blog. > http://kminder.github.io/knox/2015/11/16/adding-a-service-to-knox.html > > > > > On 11/16/15, 11:34 AM, "Kevin Minder" <[email protected]> > wrote: > > >I will note that the <route path=“”/> choice can be more subtle that it > seems. It also intrinsically affects the way the rewrite rules are written. > > > >The routes in service.xml need be unique across all of the services that > may be deployed within the gateway. Therefore it is important to carefully > consider the top level of the URL hierarchy. In the sample below I’ve used > /weather. This will lead to gateway URLs such as > 'https://host:port/gateway/sandbox/weather'. > This choice affects your rewrite rules because you may need to rewrite this > prefix out (e.g. weather) since the real service may not use it. In the > case of OpenWeatherMap their API URLs starts with /data we the rewrite > rules have to remove it. > > > > > ><rule ... pattern="*://*:*/**/weather/{path=**}?{**}"> > > <rewrite template="{$serviceUrl[WEATHER]}/{path=**}?{**}”/> > > > >Note: The $serviceUrl[WEATHER] function simply looks up the <url> from > <service> with <role> WEATHER in the topology file. > > > > > > > >On 11/16/15, 11:11 AM, "Kevin Minder" <[email protected]> > wrote: > > > >>Adam, > >>The details from Sumit seem correct. As another example I took the > OpenWeatherMap example from the DevGuide and turned it into a config based > example and tested it. Here is the result below. This really needs to be > in the dev-guide with a detailed breakdown of how everything fits > together. Anyway I hope with Sumit’s input and this example you can > progress. If you’re still stuck I’m going to keep working on this > OpenWeatherMap sample to provide the detailed breakdown so that may help. > >> > >>These are the external/interal curl commands I’ve verified work. > >>curl ' > http://api.openweathermap.org/data/2.5/weather?zip=95054,us&appid=2de143494c0b295cca9337e1e96b00e0 > ' > >>curl -ku guest:guest-password ' > https://localhost:8443/gateway/sandbox/weather/data/2.5/weather?zip=95054,us&appid=2de143494c0b295cca9337e1e96b00e0 > ' > >> > >>This it the service.xml file: > <GATEWAY_HOME>/data/services/weather/0.0.1/service.xml > >><service role="WEATHER" name="weather" version="0.0.1"> > >> <routes> > >> <route path="/weather/**"/> > >> </routes> > >></service> > >> > >>This is the rewrite.xml file: > <GATEWAY_HOME>/data/services/weather/0.0.1/rewrite.xml > >><rules> > >> <rule dir="IN" name="WEATHER/weather/inbound" > pattern="*://*:*/**/weather/{path=**}?{**}"> > >> <rewrite template="{$serviceUrl[WEATHER]}/{path=**}?{**}"/> > >> </rule> > >></rules> > >> > >> > >>This is the topology file: <GATEWAY_HOME>/conf/topologies/sandbox.ml > >><topology> > >> ... > >> <service> > >> <role>WEATHER</role> > >> <url>http://api.openweathermap.org:80</url> > >> </service> > >></topology> > >> > >> > >> > >>Kevin. > >> > >> > >> > >>On 11/16/15, 10:43 AM, "Sumit Gupta" <[email protected]> > wrote: > >> > >>>Hey Adam, > >>> > >>>We clearly need to improve our docs :). A simpler example to follow may > be > >>>something like the falcon service and rewrite files in the codebase. In > >>>any case, you can try something like this for service xml > >>> > >>><service role="HELLO" name="hello" version="0.0.1"> > >>> <routes> > >>> <route path="/hello/**"/> > >>> </routes> > >>></service> > >>> > >>> > >>> > >>>And this for rewrite: > >>> > >>><rules> > >>> <rule dir="IN" name="HELLO/greeting/inbound/root" > >>>pattern="*://*:*/**/hello/{**}"> > >>> <rewrite template="{$serviceUrl[HELLO]}/{**}"/> > >>> </rule> > >>></rules> > >>> > >>> > >>>And in the topology file you only need the host and port info. > >>> > >>><service> > >>> <role>HELLO</role> > >>> <url>http://sandbox.hortonworks.com:8095</url> > >>></service> > >>> > >>> > >>> > >>>And then for following curl for knox should work. > >>> > >>> > >>>curl -iku guest:guest-password -X GET > >>>'https://localhost:8443/gateway/default/hello/greeting' > >>> > >>> > >>>Please note that in this case I simply Œroute¹ anything with your > desired > >>>service url part (Œhello') and rewrite the host and port information in > it > >>>so that it goes to the right place. Rewrite files can certainly be very > >>>elaborate and complex. This is the simplest one I could think off to get > >>>you going. > >>> > >>>Kevin can probably add more color to this and maybe even simplify it > >>>further. > >>> > >>>Thanks, > >>>Sumit. > >>> > >>> > >>> > >>>On 11/16/15, 10:01 AM, "Adam Davidson" > >>><[email protected]> wrote: > >>> > >>>>Hi Sumit and Kevin, > >>>> > >>>>This is the command I'm using to access the demo REST app inside the > >>>>sandbox: > >>>>*curl 'http://localhost:8095/greeting <http://localhost:8095/greeting > >'* > >>>>This returns a simple JSON response like *'{"id":1,"content":"Hello, > >>>>World!"}' . *BTW, it's the same app as described in a Spring Boot > tutorial > >>>>here - https://spring.io/guides/gs/rest-service/ > >>>> > >>>>From outside the sandbox I am using the following: > >>>>*curl -kv -u admin:admin-password > >>>>'https://127.0.0.1:8443/gateway/default/greeting/v1 > >>>><https://127.0.0.1:8443/gateway/default/greeting/v1>'* > >>>>Which is returning a 404. I have a version number in there as I was > >>>>getting > >>>>null pointer exceptions when I left it out and assumed it was > mandatory to > >>>>have some kind of versioning in the URL. > >>>> > >>>>I can also run this WebHDFS command from outside the gateway: > >>>>'curl -k -u admin:admin-password* > >>>>'https://127.0.0.1:8443/gateway/default/webhdfs/v1?op=LISTSTATUS > >>>><https://127.0.0.1:8443/gateway/default/webhdfs/v1?op=LISTSTATUS>'* > >>>>This gives the appropriate response. > >>>> > >>>>Thanks for the quick follow-up, I also realise now this question may > have > >>>>been better suited to the user mailing-list so apologies for that. > >>>> > >>>>Best Regards, > >>>>Adam > >>>> > >>>>On Mon, 16 Nov 2015 at 14:49 Sumit Gupta <[email protected]> > >>>>wrote: > >>>> > >>>>> Hi Adam, > >>>>> > >>>>> Thanks for providing the the configuration and file contents. Can you > >>>>>also > >>>>> provide the exact URLs you are trying with the curl commands, both > >>>>>inside > >>>>> and outside the cluster? > >>>>> > >>>>> Thanks, > >>>>> Sumit > >>>>> > >>>>> On 11/16/15, 9:38 AM, "Adam Davidson" > >>>>> <[email protected]> wrote: > >>>>> > >>>>> >Hi, > >>>>> > > >>>>> >As part of some client work I'm trying to add a service to a Knox > >>>>>topology > >>>>> >in the Hortonworks HDP 2.3.1 sandbox; this includes Knox 0.6.0. My > >>>>> >understanding is that I simply need to add the <service> to the > >>>>>topology > >>>>> >XML and then create directories under <knox home>/data/services with > >>>>>new > >>>>> >service and rewrite XML files. > >>>>> > > >>>>> >In this test case the REST service is a similar demo app running on > >>>>>port > >>>>> >8095. I've created the service and rewrite XMLs as before yet the > >>>>>gateway > >>>>> >is throwing a 404 when I try and access the API from outside the > >>>>>sandbox. > >>>>> >The usual calls to WebHDFS through the gateway still work. My demo > >>>>>REST > >>>>> >service is running and can be accessed through curl on the inside of > >>>>>the > >>>>> >gateway. > >>>>> > > >>>>> >I've added the following to the default.xml topology (and also > >>>>> >knox-sample.xml, another topology in the HDP sandbox) > >>>>> ><service> > >>>>> > <role>HELLO</role> > >>>>> > <url>http://sandbox.hortonworks.com:8095/greeting</url> > >>>>> ></service> > >>>>> > > >>>>> >This is my service.xml in data/services/hello/0.0.1 > >>>>> ><service role="HELLO" name="hello" version="0.0.1"> > >>>>> > <routes> > >>>>> > <route path="/greeting/v1/?**"> > >>>>> > <rewrite apply="HELLO/greeting/inbound/root" > >>>>> >to="request.url"/> > >>>>> > </route> > >>>>> > </routes> > >>>>> ></service> > >>>>> > > >>>>> >And finally this is my rewrite.xml > >>>>> ><rules> > >>>>> > <rule dir="IN" name="HELLO/greeting/inbound/root" > >>>>> >pattern="*://*:*/**/greeting/v1/?{**}"> > >>>>> > <rewrite template="{$serviceUrl[HELLO]}?/v1/{**}"/> > >>>>> > </rule> > >>>>> ></rules> > >>>>> > > >>>>> >Am I missing anything in terms of configuration? I must confess I'm > not > >>>>> >sure what's going on in rewrite.xml and struggle to get much out of > the > >>>>> >documentation. > >>>>> > > >>>>> >Best Regards, > >>>>> >Adam > >>>>> > > >>>>> >-- > >>>>> > > >>>>> > > >>>>> >*NOTICE AND DISCLAIMER* > >>>>> > > >>>>> >This email (including attachments) is confidential. If you are not > the > >>>>> >intended recipient, notify the sender immediately, delete this email > >>>>>from > >>>>> >your system and do not disclose or use for any purpose. > >>>>> > > >>>>> >Business Address: Eagle House, 163 City Road, London, EC1V 1NR. > United > >>>>> >Kingdom > >>>>> >Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. > >>>>> >United > >>>>> >Kingdom > >>>>> >Big Data Partnership Limited is a company registered in England & > Wales > >>>>> >with Company No 7904824 > >>>>> > >>>>> > >>>> > >>>>-- > >>>> > >>>> > >>>>*NOTICE AND DISCLAIMER* > >>>> > >>>>This email (including attachments) is confidential. If you are not the > >>>>intended recipient, notify the sender immediately, delete this email > from > >>>>your system and do not disclose or use for any purpose. > >>>> > >>>>Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United > >>>>Kingdom > >>>>Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. > >>>>United > >>>>Kingdom > >>>>Big Data Partnership Limited is a company registered in England & Wales > >>>>with Company No 7904824 > >>> > >>> > -- *NOTICE AND DISCLAIMER* This email (including attachments) is confidential. If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United Kingdom Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United Kingdom Big Data Partnership Limited is a company registered in England & Wales with Company No 7904824
