Is this going to be a rewrite or improvement to existing cache mediator? If this is a rewrite, will it be included in next EI release?
Also can we also implement cache eviction as part of this effort? Ideally, we should be able to configure how many maximum responses we can cache (not sure whether it's there already, it was not working sometimes back) and what's the maximum message size (it's already there) we can cache to avoid OOM. Then we need to have cache eviction in place to evict some cached items when cache is full, in-order to give space for most recent responses. Thanks. On Wed, Jul 5, 2017 at 6:02 AM, Riyafa Abdul Hameed <riy...@wso2.com> wrote: > Hi, > Please find the requirements doc below: > https://docs.google.com/document/d/1iPtArrW6C- > VgzAzjjSsLmG9aqAFEubmFgNkQhoDvqz8/edit?usp=sharing > Any suggestions would be highly appreciated. > Thank you. > Riyafa > > On Tue, Jul 4, 2017 at 5:37 PM, Riyafa Abdul Hameed <riy...@wso2.com> > wrote: > >> Hi all, >> >> Thank you for the suggestions. Yes the mediator should also work with the >> call mediator. >> >> Further on a discussion with the APIM team I have identified the >> following issues which points to requiring a rewrite of the cache mediator: >> >> 1. >> >> Current cache mediator[1] caches responses for all the HTTP Request >> Methods which include GET, POST, PUT and DELETE. Ideally a cache mediator >> should cache only the response for GET method and caching for POST method >> should be configurable (because POST is the only method used in SOAP >> implementations). The caching for PUT and DELETE methods should be >> completely omitted. The reason for caching responses only for GET is >> because in the REST world other methods (POST, PUT and DELETE) are used >> for >> creating, modifying or deleting existing records and hence the response >> would only mention the success or failure of the request. Caching these >> responses is meaningless and error prone. >> 2. >> >> Current cache mediator caches all the responses whether they are >> success or failure responses. Ideally the mediator should cache only >> success (200 OK) responses and not the failures >> 3. >> >> Current mediator hashes the payload (both DOMHASHGenerator and >> REQUESTHASHGenerator) which is a huge overhead for messages with larger >> bodies. The mediator should hash only the recipient (To) address of the >> request and the HTTP headers which can be easily handled when only >> responses for GET methods are hashed. Payload should be hashed only if PUT >> method is configured. >> 4. >> >> Current mediator does not allow configurations for omitting certain >> headers when hashing. For example there’s no point in hashing the >> timestamp >> header--it will almost always be different from message to message. >> 5. >> >> The current mediator does not honor the http headers[2]. For example >> the max-age header specifies the maximum time in seconds that the fetched >> response is allowed to be reused from the time of the request. Although >> cache timeout is allowed to be configured this header is not honored when >> it is specified in a message. >> 6. >> >> Although it is possible to set the maximum number of elements to be >> cached in bytes it might result in this set value exceeding the available >> memory at runtime. >> 7. >> >> The current mediator does not invoke the handleResponseInFlow or >> handleResponseOutFlow methods in handlers[3] >> >> >> >> [1] https://docs.wso2.com/display/EI611/Cache+Mediator >> >> [2] https://developers.google.com/web/fundamentals/performance/o >> ptimizing-content-efficiency/http-caching >> >> [3] https://docs.wso2.com/display/EI611/Writing+a+Synapse+Handler >> >> On Tue, Jul 4, 2017 at 4:28 PM, Isuru Udana <isu...@wso2.com> wrote: >> >>> Hi, >>> >>> On Tue, Jul 4, 2017 at 3:10 PM, Vijitha Ekanayake <vijit...@wso2.com> >>> wrote: >>> >>>> Hi Riyafa, >>>> >>>> Apart from the issues that you have mentioned, we may need to consider >>>> following points while refactoring the cache mediator implementation. >>>> >>>> 1). Verify the cache mediator functionality with different >>>> configuration parameters and fix if there is anything broken. >>>> 2). Implement a mechanism to support caching when using the call >>>> mediator. >>>> 3). Feasibility to introduce native JSON caching support to cache >>>> mediator. >>>> 4). Feasibility to introduce service level caching which isn't support >>>> at the moment. >>>> >>> IMO, service level caching is not a requirement as long as we can >>> achieve all the requirements through the Cache Mediator. >>> >>> Thanks. >>> >>>> >>>> >>>> Thanks. >>>> >>>> >>>> On Tue, Jul 4, 2017 at 12:06 PM, Riyafa Abdul Hameed <riy...@wso2.com> >>>> wrote: >>>> >>>>> Dear all, >>>>> >>>>> By going through the issues faced by the customers in the past I >>>>> discovered the following issues: >>>>> >>>>> 1. Continue the execution on cache hit. Reported as an issue in >>>>> github[1] >>>>> 2. Issue in processing JSON array (payload) with a single element >>>>> when response caching is enabled where expected response is: >>>>> >>>>> [ >>>>> { >>>>> "msg":"Hello", >>>>> "services":[ >>>>> "elec", >>>>> "patrol" >>>>> ], >>>>> "test":"World." >>>>> }, >>>>> { >>>>> "msg":"Hi", >>>>> "services":[ >>>>> "water" >>>>> ], >>>>> "test":"Sri Lanka." >>>>> } >>>>> ] >>>>> >>>>> but received response is: >>>>> >>>>> [ >>>>> { "msg": "Hello", "services": [ "elec", "patrol" ], "test": "World." } >>>>> >>>>> , >>>>> { "msg": "Hi", "services": "water", "test": "Sri Lanka." } >>>>> >>>>> ] >>>>> >>>>> This issue has been fixed in the carbon mediation[2]. >>>>> >>>>> 3. When a xml body with processing instructions is stored >>>>> in cache and sent back as json it fails with an exception. Issue was >>>>> reported in 2015 and already has a fix in current EI. >>>>> >>>>> Since the last two issues have been already fixed, I would like to >>>>> know what other issues if any needs to be addressed and if a complete >>>>> rewrite of the cache mediator would be required. >>>>> >>>>> [1] https://github.com/wso2/product-ei/issues/695 >>>>> >>>>> [2]https://github.com/riyafa/carbon-mediation/commit/7aaf597 >>>>> 988a333e1cad36dc0b5057e24fb779a5c >>>>> >>>>> >>>>> Thank you. >>>>> >>>>> Riyafa >>>>> >>>>> -- >>>>> Riyafa Abdul Hameed >>>>> Software Engineer, WSO2 Lanka (Pvt) Ltd <http://wso2.com/> >>>>> >>>>> Email: riy...@wso2.com <riyafa...@cse.mrt.ac.lk> >>>>> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/> >>>>> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa> >>>>> <http://twitter.com/Riyafa1> >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> Dev@wso2.org >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> Vijitha Ekanayake >>>> Software Engineer*, *WSO2, Inc.; http://wso2.com/ >>>> Mobile : +94 777 24 73 39 | +94 718 74 44 08 >>>> lean.enterprise.middleware >>>> >>> >>> >>> >>> -- >>> *Isuru Udana* >>> Senior Technical Lead >>> WSO2 Inc.; http://wso2.com >>> email: isu...@wso2.com cell: +94 77 3791887 <+94%2077%20379%201887> >>> blog: http://mytecheye.blogspot.com/ >>> >> >> >> >> -- >> Riyafa Abdul Hameed >> Software Engineer, WSO2 Lanka (Pvt) Ltd <http://wso2.com/> >> >> Email: riy...@wso2.com <riyafa...@cse.mrt.ac.lk> >> Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/> >> <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa> >> <http://twitter.com/Riyafa1> >> > > > > -- > Riyafa Abdul Hameed > Software Engineer, WSO2 Lanka (Pvt) Ltd <http://wso2.com/> > > Email: riy...@wso2.com <riyafa...@cse.mrt.ac.lk> > Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/> > <http://facebook.com/riyafa.ahf> <http://lk.linkedin.com/in/riyafa> > <http://twitter.com/Riyafa1> > > _______________________________________________ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Rajkumar Rajaratnam Committer & PMC Member, Apache Stratos Senior Software Engineer, WSO2
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
