Hey Adam, Thanks for being my editor! These were both cases of unescaped XML in markdown. Thanks again! Kevin. P.S. I’ll eventually fold this into the dev guide but I wanted to get this “on paper” while inspired.
On 11/17/15, 4:57 AM, "Adam Davidson" <[email protected]> wrote: >Thanks for the blog Kevin, it's a very useful breakdown of the process that >even I can understand. This covers everything I said about the >documentation being tricky so perhaps just linking this blog somewhere in >the developer guide would be helpful. > >On a side note I noticed a couple of typos in there: >"This will need to match the topology file’s for this service" - rogue >apostrophe in files under the service.xml section >"Since and are not present a default..." - missing a word, under the >sandbox.xml section at the end. > >Kind Regards, >Adam > >On Mon, 16 Nov 2015 at 23:47 Kevin Minder <[email protected]> >wrote: > >> Inspired by this question I created this blog. >> http://kminder.github.io/knox/2015/11/16/adding-a-service-to-knox.html >> >> >> >> >> On 11/16/15, 11:34 AM, "Kevin Minder" <[email protected]> >> wrote: >> >> >I will note that the <route path=“”/> choice can be more subtle that it >> seems. It also intrinsically affects the way the rewrite rules are written. >> > >> >The routes in service.xml need be unique across all of the services that >> may be deployed within the gateway. Therefore it is important to carefully >> consider the top level of the URL hierarchy. In the sample below I’ve used >> /weather. This will lead to gateway URLs such as >> 'https://host:port/gateway/sandbox/weather'. >> This choice affects your rewrite rules because you may need to rewrite this >> prefix out (e.g. weather) since the real service may not use it. In the >> case of OpenWeatherMap their API URLs starts with /data we the rewrite >> rules have to remove it. >> > >> > >> ><rule ... pattern="*://*:*/**/weather/{path=**}?{**}"> >> > <rewrite template="{$serviceUrl[WEATHER]}/{path=**}?{**}”/> >> > >> >Note: The $serviceUrl[WEATHER] function simply looks up the <url> from >> <service> with <role> WEATHER in the topology file. >> > >> > >> > >> >On 11/16/15, 11:11 AM, "Kevin Minder" <[email protected]> >> wrote: >> > >> >>Adam, >> >>The details from Sumit seem correct. As another example I took the >> OpenWeatherMap example from the DevGuide and turned it into a config based >> example and tested it. Here is the result below. This really needs to be >> in the dev-guide with a detailed breakdown of how everything fits >> together. Anyway I hope with Sumit’s input and this example you can >> progress. If you’re still stuck I’m going to keep working on this >> OpenWeatherMap sample to provide the detailed breakdown so that may help. >> >> >> >>These are the external/interal curl commands I’ve verified work. >> >>curl ' >> http://api.openweathermap.org/data/2.5/weather?zip=95054,us&appid=2de143494c0b295cca9337e1e96b00e0 >> ' >> >>curl -ku guest:guest-password ' >> https://localhost:8443/gateway/sandbox/weather/data/2.5/weather?zip=95054,us&appid=2de143494c0b295cca9337e1e96b00e0 >> ' >> >> >> >>This it the service.xml file: >> <GATEWAY_HOME>/data/services/weather/0.0.1/service.xml >> >><service role="WEATHER" name="weather" version="0.0.1"> >> >> <routes> >> >> <route path="/weather/**"/> >> >> </routes> >> >></service> >> >> >> >>This is the rewrite.xml file: >> <GATEWAY_HOME>/data/services/weather/0.0.1/rewrite.xml >> >><rules> >> >> <rule dir="IN" name="WEATHER/weather/inbound" >> pattern="*://*:*/**/weather/{path=**}?{**}"> >> >> <rewrite template="{$serviceUrl[WEATHER]}/{path=**}?{**}"/> >> >> </rule> >> >></rules> >> >> >> >> >> >>This is the topology file: <GATEWAY_HOME>/conf/topologies/sandbox.ml >> >><topology> >> >> ... >> >> <service> >> >> <role>WEATHER</role> >> >> <url>http://api.openweathermap.org:80</url> >> >> </service> >> >></topology> >> >> >> >> >> >> >> >>Kevin. >> >> >> >> >> >> >> >>On 11/16/15, 10:43 AM, "Sumit Gupta" <[email protected]> >> wrote: >> >> >> >>>Hey Adam, >> >>> >> >>>We clearly need to improve our docs :). A simpler example to follow may >> be >> >>>something like the falcon service and rewrite files in the codebase. In >> >>>any case, you can try something like this for service xml >> >>> >> >>><service role="HELLO" name="hello" version="0.0.1"> >> >>> <routes> >> >>> <route path="/hello/**"/> >> >>> </routes> >> >>></service> >> >>> >> >>> >> >>> >> >>>And this for rewrite: >> >>> >> >>><rules> >> >>> <rule dir="IN" name="HELLO/greeting/inbound/root" >> >>>pattern="*://*:*/**/hello/{**}"> >> >>> <rewrite template="{$serviceUrl[HELLO]}/{**}"/> >> >>> </rule> >> >>></rules> >> >>> >> >>> >> >>>And in the topology file you only need the host and port info. >> >>> >> >>><service> >> >>> <role>HELLO</role> >> >>> <url>http://sandbox.hortonworks.com:8095</url> >> >>></service> >> >>> >> >>> >> >>> >> >>>And then for following curl for knox should work. >> >>> >> >>> >> >>>curl -iku guest:guest-password -X GET >> >>>'https://localhost:8443/gateway/default/hello/greeting' >> >>> >> >>> >> >>>Please note that in this case I simply Œroute¹ anything with your >> desired >> >>>service url part (Œhello') and rewrite the host and port information in >> it >> >>>so that it goes to the right place. Rewrite files can certainly be very >> >>>elaborate and complex. This is the simplest one I could think off to get >> >>>you going. >> >>> >> >>>Kevin can probably add more color to this and maybe even simplify it >> >>>further. >> >>> >> >>>Thanks, >> >>>Sumit. >> >>> >> >>> >> >>> >> >>>On 11/16/15, 10:01 AM, "Adam Davidson" >> >>><[email protected]> wrote: >> >>> >> >>>>Hi Sumit and Kevin, >> >>>> >> >>>>This is the command I'm using to access the demo REST app inside the >> >>>>sandbox: >> >>>>*curl 'http://localhost:8095/greeting <http://localhost:8095/greeting >> >'* >> >>>>This returns a simple JSON response like *'{"id":1,"content":"Hello, >> >>>>World!"}' . *BTW, it's the same app as described in a Spring Boot >> tutorial >> >>>>here - https://spring.io/guides/gs/rest-service/ >> >>>> >> >>>>From outside the sandbox I am using the following: >> >>>>*curl -kv -u admin:admin-password >> >>>>'https://127.0.0.1:8443/gateway/default/greeting/v1 >> >>>><https://127.0.0.1:8443/gateway/default/greeting/v1>'* >> >>>>Which is returning a 404. I have a version number in there as I was >> >>>>getting >> >>>>null pointer exceptions when I left it out and assumed it was >> mandatory to >> >>>>have some kind of versioning in the URL. >> >>>> >> >>>>I can also run this WebHDFS command from outside the gateway: >> >>>>'curl -k -u admin:admin-password* >> >>>>'https://127.0.0.1:8443/gateway/default/webhdfs/v1?op=LISTSTATUS >> >>>><https://127.0.0.1:8443/gateway/default/webhdfs/v1?op=LISTSTATUS>'* >> >>>>This gives the appropriate response. >> >>>> >> >>>>Thanks for the quick follow-up, I also realise now this question may >> have >> >>>>been better suited to the user mailing-list so apologies for that. >> >>>> >> >>>>Best Regards, >> >>>>Adam >> >>>> >> >>>>On Mon, 16 Nov 2015 at 14:49 Sumit Gupta <[email protected]> >> >>>>wrote: >> >>>> >> >>>>> Hi Adam, >> >>>>> >> >>>>> Thanks for providing the the configuration and file contents. Can you >> >>>>>also >> >>>>> provide the exact URLs you are trying with the curl commands, both >> >>>>>inside >> >>>>> and outside the cluster? >> >>>>> >> >>>>> Thanks, >> >>>>> Sumit >> >>>>> >> >>>>> On 11/16/15, 9:38 AM, "Adam Davidson" >> >>>>> <[email protected]> wrote: >> >>>>> >> >>>>> >Hi, >> >>>>> > >> >>>>> >As part of some client work I'm trying to add a service to a Knox >> >>>>>topology >> >>>>> >in the Hortonworks HDP 2.3.1 sandbox; this includes Knox 0.6.0. My >> >>>>> >understanding is that I simply need to add the <service> to the >> >>>>>topology >> >>>>> >XML and then create directories under <knox home>/data/services with >> >>>>>new >> >>>>> >service and rewrite XML files. >> >>>>> > >> >>>>> >In this test case the REST service is a similar demo app running on >> >>>>>port >> >>>>> >8095. I've created the service and rewrite XMLs as before yet the >> >>>>>gateway >> >>>>> >is throwing a 404 when I try and access the API from outside the >> >>>>>sandbox. >> >>>>> >The usual calls to WebHDFS through the gateway still work. My demo >> >>>>>REST >> >>>>> >service is running and can be accessed through curl on the inside of >> >>>>>the >> >>>>> >gateway. >> >>>>> > >> >>>>> >I've added the following to the default.xml topology (and also >> >>>>> >knox-sample.xml, another topology in the HDP sandbox) >> >>>>> ><service> >> >>>>> > <role>HELLO</role> >> >>>>> > <url>http://sandbox.hortonworks.com:8095/greeting</url> >> >>>>> ></service> >> >>>>> > >> >>>>> >This is my service.xml in data/services/hello/0.0.1 >> >>>>> ><service role="HELLO" name="hello" version="0.0.1"> >> >>>>> > <routes> >> >>>>> > <route path="/greeting/v1/?**"> >> >>>>> > <rewrite apply="HELLO/greeting/inbound/root" >> >>>>> >to="request.url"/> >> >>>>> > </route> >> >>>>> > </routes> >> >>>>> ></service> >> >>>>> > >> >>>>> >And finally this is my rewrite.xml >> >>>>> ><rules> >> >>>>> > <rule dir="IN" name="HELLO/greeting/inbound/root" >> >>>>> >pattern="*://*:*/**/greeting/v1/?{**}"> >> >>>>> > <rewrite template="{$serviceUrl[HELLO]}?/v1/{**}"/> >> >>>>> > </rule> >> >>>>> ></rules> >> >>>>> > >> >>>>> >Am I missing anything in terms of configuration? I must confess I'm >> not >> >>>>> >sure what's going on in rewrite.xml and struggle to get much out of >> the >> >>>>> >documentation. >> >>>>> > >> >>>>> >Best Regards, >> >>>>> >Adam >> >>>>> > >> >>>>> >-- >> >>>>> > >> >>>>> > >> >>>>> >*NOTICE AND DISCLAIMER* >> >>>>> > >> >>>>> >This email (including attachments) is confidential. If you are not >> the >> >>>>> >intended recipient, notify the sender immediately, delete this email >> >>>>>from >> >>>>> >your system and do not disclose or use for any purpose. >> >>>>> > >> >>>>> >Business Address: Eagle House, 163 City Road, London, EC1V 1NR. >> United >> >>>>> >Kingdom >> >>>>> >Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. >> >>>>> >United >> >>>>> >Kingdom >> >>>>> >Big Data Partnership Limited is a company registered in England & >> Wales >> >>>>> >with Company No 7904824 >> >>>>> >> >>>>> >> >>>> >> >>>>-- >> >>>> >> >>>> >> >>>>*NOTICE AND DISCLAIMER* >> >>>> >> >>>>This email (including attachments) is confidential. If you are not the >> >>>>intended recipient, notify the sender immediately, delete this email >> from >> >>>>your system and do not disclose or use for any purpose. >> >>>> >> >>>>Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United >> >>>>Kingdom >> >>>>Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. >> >>>>United >> >>>>Kingdom >> >>>>Big Data Partnership Limited is a company registered in England & Wales >> >>>>with Company No 7904824 >> >>> >> >>> >> > >-- > > >*NOTICE AND DISCLAIMER* > >This email (including attachments) is confidential. If you are not the >intended recipient, notify the sender immediately, delete this email from >your system and do not disclose or use for any purpose. > >Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United >Kingdom >Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United >Kingdom >Big Data Partnership Limited is a company registered in England & Wales >with Company No 7904824
