Thanks, Lakmali. I added the following note in the docs: *When caching response messages, a hash value is generated based on the request's URI, transport headers and the payload (if available). WSO2 has a default REQUESTHASHGenerator class written to generate the hash value. See sample here <https://docs.wso2.com/download/attachments/41747169/REQUESTHASHGenerator.java?version=1&modificationDate=1414132933410&api=v2>.*
*If you want to change this default implementation (for example, to exclude certain headers), you can write a new hash generator implementation by extending the REQUESTHASHGenerator and overriding its getDigest() method. Once done, add the new class as the hashGenerator attribute of the <cache> element in the velocity_template.xml file.* Thanks, -Nirdesha On Thu, Oct 23, 2014 at 7:52 PM, Lalaji Sureshika <[email protected]> wrote: > Hi Lakmali, > > On Wed, Oct 22, 2014 at 11:15 PM, Lakmali Baminiwatta <[email protected]> > wrote: > >> Hi Lalaji, >> >> In REQUESTHASHGenerator, we excluded Date and User-Agent headers when >> generating the request hash. If there are any other or custom headers which >> need to be excluded, we can write a new Hash Generator implementation by >> extending REQUESTHASHGenerator and override 'getDigest()' method. Then we >> can use that in the cache mediator. >> > > Thanks for the info..I guess,we should include this information in to > wiki,regarding this feature.. > >> >> Thanks, >> Lakmali >> >> On 22 October 2014 22:59, Lalaji Sureshika <[email protected]> wrote: >> >>> Hi, >>> >>> >>> On Fri, Dec 6, 2013 at 2:49 AM, Lakmali Baminiwatta <[email protected]> >>> wrote: >>> >>>> Hi Sanjeewa, >>>> >>>> >>>> On 6 December 2013 07:48, Sanjeewa Malalgoda <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> >>>>> On Mon, Dec 2, 2013 at 6:23 PM, Lakmali Baminiwatta <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> On 2 December 2013 18:02, Sumedha Rubasinghe <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Dec 2, 2013 at 5:24 PM, Lakmali Baminiwatta < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Sanjeewa, >>>>>>>> >>>>>>>> >>>>>>>> On 2 December 2013 17:00, Sanjeewa Malalgoda <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Dec 1, 2013 at 11:01 PM, Lakmali Baminiwatta < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> When processing the response, if the request is REST then soap >>>>>>>>>> format, transport headers and message type are also stored. >>>>>>>>>> Similarly when >>>>>>>>>> retrieving the cached response, if the request is REST then stored >>>>>>>>>> values >>>>>>>>>> are used to compose the response message. With these modifications >>>>>>>>>> cache >>>>>>>>>> mediator returns the response correctly for non SOAP requests. >>>>>>>>>> >>>>>>>>>> But we have another issue here. Currently the request Hash value >>>>>>>>>> is derived from the SOAP message body. So for REST calls it gets the >>>>>>>>>> same >>>>>>>>>> hash value for all requests which sent without a payload. >>>>>>>>>> >>>>>>>>>> ex: Both below requests are recognized as the same request >>>>>>>>>> according to the hash value derived from DOMHASHGenerator. >>>>>>>>>> >>>>>>>>>> curl -v -H "Authorization: Bearer 1EV6Qqa_DboaBzNj2JfiyXoO1Ysa" >>>>>>>>>> http://localhost:8280/test/1.0.0?regNo=001 >>>>>>>>>> curl -v -H "Authorization: Bearer 1EV6Qqa_DboaBzNj2JfiyXoO1Ysa" >>>>>>>>>> http://localhost:8280/test/1.0.0?regNo=002 >>>>>>>>>> <http://localhost:8280/test/1.0.0?regNo=001> >>>>>>>>>> >>>>>>>>>> So right now we need to figure out a mechanism to generate hash >>>>>>>>>> value for REST calls. >>>>>>>>>> >>>>>>>>> How if we consider full request path(including query params and >>>>>>>>> etc) + headers. For above example something like follows (full >>>>>>>>> request path >>>>>>>>> and "," separated transport headers list). We can compute hash value >>>>>>>>> of >>>>>>>>> that string and store it. >>>>>>>>> >>>>>>>>> *"http://localhost:8280/test/1.0.0?regNo=002,Authorization >>>>>>>>> <http://localhost:8280/test/1.0.0?regNo=002,Authorization>: Bearer >>>>>>>>> 1EV6Qqa_DboaBzNj2JfiyXoO1Ysa"* >>>>>>>>> >>>>>>>>> Here we cannot only consider full request path as we take some >>>>>>>>> decisions at backend based on headers. WDYT? >>>>>>>>> >>>>>>>> >>>>>>>> Yeah, need to consider request path + headers. But I think we don't >>>>>>>> need hash Authorization headers, as request hits the cache mediator >>>>>>>> after >>>>>>>> going through the handlers. So in default case, at the time of hashing >>>>>>>> we >>>>>>>> don't have the authorization header. So AFAIU, the only concern which >>>>>>>> comes >>>>>>>> up here is, same request invoked using different keys may get the >>>>>>>> response >>>>>>>> from the cache. I believe that is ok. WDYT? >>>>>>>> >>>>>>> >>>>>>> >>>>>>> We simply store response values against a given request pattern. But >>>>>>> Authorisation header should not be part of it. Otherwise it will become >>>>>>> a >>>>>>> token based cache. But are whole lot of other header parameter like >>>>>>> content-type, accept header, accept language ( >>>>>>> http://en.wikipedia.org/wiki/List_of_HTTP_header_fields). >>>>>>> >>>>>>> Let's ignore Authorisation header for time being. >>>>>>> >>>>>>> +1. So for now let's generate the hash based on request path, Accept >>>>>> & Content-Type headers and payload (if available). >>>>>> >>>>> +1. IMO we should use all other headers except authentication headers >>>>> (Ex. sometimes response may depend on client type or sometimes it may >>>>> depend on any other custom header). >>>>> >>>> >>>> We can use all the headers [1]. But there are some headers which seems >>>> not correct to use for hashing the request. >>>> >>>> ex: >>>> Date - This is unique for each request. If this is used for generting >>>> the hash, cache will be never used. >>>> User-Agent - This is specific to the user client/browser who sent the >>>> request. So different caches for different users. >>>> >>>> So I think we need to figure out which headers to skip or which headers >>>> to use for hash generation. >>>> >>> >>> Did we consider , above unique request headers to include in >>> hashing the request or are we generating the cache digest based on all >>> transport headers..? >>> >>>> >>>> >>>> [1]http://en.wikipedia.org/wiki/List_of_HTTP_header_fields >>>> >>>> Thanks, >>>> Lakmali >>>> >>>>> >>>>> Thanks, >>>>> sanjeewa. >>>>> >>>>>> >>>>>> Thanks, >>>>>> Lakmali >>>>>> >>>>>>> >>>>>>> I am writing a new Digest Generator implementation by extending >>>>>>> DOMHASHGenerator. This basically does what DOMHASHGenerator had been >>>>>>> doing >>>>>>> and additionally it digests request path and available transport >>>>>>> headers as >>>>>>> well. >>>>>>> >>>>>>> Thanks, >>>>>>> Lakmali >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> sanjeewa. >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Lakmali >>>>>>>>> >>>>>>>>> >>>>>>>>> On 29 November 2013 00:18, Lakmali Baminiwatta <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Currently Cache mediator stores the response message as a byte >>>>>>>>>> array and builds the soap envelope from that when sending the cached >>>>>>>>>> response[1]. When building the response, it checks whether the >>>>>>>>>> request >>>>>>>>>> format is in SOAP11 or SOAP12 and do the build accordingly. >>>>>>>>>> >>>>>>>>>> For SOAP requests, the SOAP format for response and request >>>>>>>>>> messages are same since most of the time a WSDL is used. So above >>>>>>>>>> logic >>>>>>>>>> works. >>>>>>>>>> >>>>>>>>>> But the response message for REST calls may be in different SOAP >>>>>>>>>> formats than the request. This results in errors when building the >>>>>>>>>> message. >>>>>>>>>> >>>>>>>>>> ex: >>>>>>>>>> >>>>>>>>>> Below REST call's request message context is SOAP12. But the >>>>>>>>>> response SOAP envelope is SOAP11. Hence building the message throws >>>>>>>>>> an >>>>>>>>>> exception. >>>>>>>>>> >>>>>>>>>> curl -v -H "Authorization: Bearer 1EV6Qqa_DboaBzNj2JfiyXoO1Ysa" >>>>>>>>>> http://localhost:8280/test/1.0.0?regNo=001 >>>>>>>>>> >>>>>>>>>> Further Content-Type of the responses can be different (ex: >>>>>>>>>> application/xml, application/json). So I think we should store the >>>>>>>>>> actual >>>>>>>>>> SOAP format and the Content-Type of the response than just storing >>>>>>>>>> the >>>>>>>>>> response envelope. One option would be to store them in >>>>>>>>>> CachableResponse >>>>>>>>>> object[2] (May be in a property map?) >>>>>>>>>> >>>>>>>>>> Any better way to fix this? >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/dependencies/synapse/2.1.2-wso2v2/modules/core/src/main/java/org/apache/synapse/mediators/builtin/CacheMediator.java >>>>>>>>>> [2] >>>>>>>>>> https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/dependencies/commons/caching/4.0.2/modules/core/src/main/java/org/wso2/caching/CachableResponse.java >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Lakmali >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 22 November 2013 15:59, Sumedha Rubasinghe <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Dinusha, >>>>>>>>>>> Did you identify what needs to be fixed? I think ESB team is >>>>>>>>>>> busy with a release. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Nov 21, 2013 at 7:09 PM, Dinusha Senanayaka < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> We are trying to add response caching as a default feature in >>>>>>>>>>>> API Manager using Cache mediator. There, we found that Cache >>>>>>>>>>>> mediator could >>>>>>>>>>>> handle only SOAP-1.1 or SOAP-1.2 messages. >>>>>>>>>>>> >>>>>>>>>>>> - If backend returns a POX response, it throws the following >>>>>>>>>>>> exception [1] when request is served from the cache. >>>>>>>>>>>> - If backend returns a JSON response, and once the response get >>>>>>>>>>>> added to cache, client does not receive a response until cache >>>>>>>>>>>> expires. (No >>>>>>>>>>>> backend errors) >>>>>>>>>>>> >>>>>>>>>>>> Is it possible to fix those for API Manager-1.6.0 release ? >>>>>>>>>>>> >>>>>>>>>>>> [1] [2013-11-21 13:33:18,044] ERROR - NativeWorkerPool Uncaught >>>>>>>>>>>> exception >>>>>>>>>>>> org.apache.axiom.soap.SOAPProcessingException: Transport level >>>>>>>>>>>> information does not match with SOAP Message namespace URI >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.identifySOAPVersion(StAXSOAPModelBuilder.java:187) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.<init>(StAXSOAPModelBuilder.java:170) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.axis2.saaj.SOAPPartImpl.<init>(SOAPPartImpl.java:194) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.axis2.saaj.SOAPMessageImpl.<init>(SOAPMessageImpl.java:112) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.axis2.saaj.MessageFactoryImpl.createMessage(MessageFactoryImpl.java:132) >>>>>>>>>>>> at >>>>>>>>>>>> org.wso2.caching.util.SOAPMessageHelper.buildSOAPEnvelopeFromBytes(SOAPMessageHelper.java:77) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.mediators.builtin.CacheMediator.processRequestMessage(CacheMediator.java:319) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.mediators.builtin.CacheMediator.mediate(CacheMediator.java:170) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:77) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:47) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.mediators.filters.FilterMediator.mediate(FilterMediator.java:160) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:77) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:47) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:131) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.rest.Resource.process(Resource.java:297) >>>>>>>>>>>> at org.apache.synapse.rest.API.process(API.java:341) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:76) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:63) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:337) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:378) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:184) >>>>>>>>>>>> at >>>>>>>>>>>> org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172) >>>>>>>>>>>> at >>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>>>>>>>> at >>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>>>>>>>> at java.lang.Thread.run(Thread.java:619) >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Dinusha. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Dinusha Dilrukshi >>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>> WSO2 Inc.: http://wso2.com/ >>>>>>>>>>>> Mobile: +94725255071 >>>>>>>>>>>> Blog: http://dinushasblog.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> /sumedha >>>>>>>>>>> m: +94 773017743 >>>>>>>>>>> b : bit.ly/sumedha >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Lakmali Baminiwatta >>>>>>>>>> Software Engineer >>>>>>>>>> WSO2, Inc.: http://wso2.com >>>>>>>>>> lean.enterprise.middleware >>>>>>>>>> mobile: +94 71 2335936 >>>>>>>>>> blog : lakmali.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Lakmali Baminiwatta >>>>>>>>> Software Engineer >>>>>>>>> WSO2, Inc.: http://wso2.com >>>>>>>>> lean.enterprise.middleware >>>>>>>>> mobile: +94 71 2335936 >>>>>>>>> blog : lakmali.com >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> *Sanjeewa Malalgoda* >>>>>>>> Senior Software Engineer >>>>>>>> WSO2 Inc. >>>>>>>> Mobile : +94713068779 >>>>>>>> >>>>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Lakmali Baminiwatta >>>>>>> Software Engineer >>>>>>> WSO2, Inc.: http://wso2.com >>>>>>> lean.enterprise.middleware >>>>>>> mobile: +94 71 2335936 >>>>>>> blog : lakmali.com >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /sumedha >>>>>>> b : bit.ly/sumedha >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Lakmali Baminiwatta >>>>>> Software Engineer >>>>>> WSO2, Inc.: http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> mobile: +94 71 2335936 >>>>>> blog : lakmali.com >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *Sanjeewa Malalgoda* >>>>> Senior Software Engineer >>>>> WSO2 Inc. >>>>> Mobile : +94713068779 >>>>> >>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Lakmali Baminiwatta >>>> Software Engineer >>>> WSO2, Inc.: http://wso2.com >>>> lean.enterprise.middleware >>>> mobile: +94 71 2335936 >>>> blog : lakmali.com >>>> >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> Lalaji Sureshika >>> WSO2, Inc.; http://wso2.com/ >>> email: [email protected]; cell: +94 71 608 6811 >>> blog: http://lalajisureshika.blogspot.com >>> >>> >>> >> >> >> -- >> Lakmali Baminiwatta >> Senior Software Engineer >> WSO2, Inc.: http://wso2.com >> lean.enterprise.middleware >> mobile: +94 71 2335936 >> blog : lakmali.com >> >> > > > -- > Lalaji Sureshika > WSO2, Inc.; http://wso2.com/ > email: [email protected]; cell: +94 71 608 6811 > blog: http://lalajisureshika.blogspot.com > > > -- Thanks, Nirdesha Munasinghe, WSO2 Inc. Web:http://wso2.com Mobile: +94 776321920
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
