We are not doing this to avoid vendor specific queries. The real reason why we want to go down this path, we have no straight forward way of paginating at DB level along with taking into a account the latest version of a given API. To support this at DB level properly means we need to really complicate things and even then we are not sure we can actually meet the need. Being able to avoid vendor specific queries is just a nice side effect.
Having thought about the concern Jo raised, I think we can simply avoid using a cache for searching. The cache will only be used for things like API listing. Search results will always be served from the DB and paginated at Java level. We can dispense with the cache altogether for now and add later on if performance is not enough. The main thing is that we want to move pagination logic to the Java level. On 16 November 2016 at 21:14, Lahiru Cooray <[email protected]> wrote: > +1 for Jo's opinion on using RBMS based pagination. > > I think we are trying here to make the code unnecessarily complex by > introducing caches. As discussed this may become more complex when we do > sorting/searching operations. > Also regarding the performance, I don't see a significant performance gain > by introducing our own cache as RDBMS's normally maintain query caches. > > Only benefit I see here is avoiding the vendor specific queries. > How about addressing this concern properly by introducing a factory method > and giving the flexibility to plug a new vendor easily when needed. > > On Wed, Nov 16, 2016 at 3:23 PM, Uvindra Dias Jayasinha <[email protected]> > wrote: > >> Hi Jo, >> >> On 16 November 2016 at 14:47, Joseph Fonseka <[email protected]> wrote: >> >>> Hi Uvindra >>> >>> On Wed, Nov 16, 2016 at 1:48 PM, Uvindra Dias Jayasinha < >>> [email protected]> wrote: >>>> >>>> >>>> Searching and sorting will be done at DB level via SQL. It is only the >>>> pagination of the returned result set that will be done at Java level. >>>> >>> >>> So does this mean for different search request you will be caching >>> separate sets of data ? If that is so will it exhaust the memory and how >>> will API search perform under a load. And since the servers will be running >>> on containers available memory would be even less. >>> >> >> Thats a good point, definitely should not be caching different search >> request results.. Will discuss this with the team and get back to you. >> >>> >>> Why would this be more efficient than Java? We achieve a similar thing >>>> by using a local cache so there is no need to have an intermediate in >>>> memory DB. What was proposed makes sense when you consider that we a >>>> dealing with only a very small amount of data at DB level now. >>>> >>> >>> Considering pagination is done using DBMS, in-memory db will help to >>> work only with a one data set as well as needing to maintain one set of >>> pagination queries. >>> >> >> Hmmm I still dont understand the difference between in memory DB and in >> memory cache. Technically in both cases we are working with an in memory >> dataset. >> >>> >>> >>>> In the future if we foresee a requirement for DB level pagination we >>>> can implement that within the DAO layer, but for now this is more than >>>> enough to meet our needs. This way we can avoid introducing unwanted >>>> complexity upfront since there is no rationale for doing so. >>>> >>> >>> Agreed, but if the complexity is to support pagination for all the DBMSs >>> we can at-least focus on adding db level pagination for commonly used DBMSs >>> (ie MySQL, Oracle). >>> >> >> Its not a complexity to support for multiple DB's, we can do that but its >> an additional effort we can avoid by going down this path. >> >>> >>> Regards >>> Jo >>> >>> >>> >>> >>> >>> >>>> >>>> >>>>> Thanks & Regards >>>>> Jo >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>>>> With regards, >>>>>>> *Manu*ranga Perera. >>>>>>> >>>>>>> phone : 071 7 70 20 50 >>>>>>> mail : [email protected] >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Uvindra >>>>>> >>>>>> Mobile: 777733962 >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> -- >>>>> *Joseph Fonseka* >>>>> WSO2 Inc.; http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> mobile: +94 772 512 430 >>>>> skype: jpfonseka >>>>> >>>>> * <http://lk.linkedin.com/in/rumeshbandara>* >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Uvindra >>>> >>>> Mobile: 777733962 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> >>> -- >>> *Joseph Fonseka* >>> WSO2 Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> mobile: +94 772 512 430 >>> skype: jpfonseka >>> >>> * <http://lk.linkedin.com/in/rumeshbandara>* >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Regards, >> Uvindra >> >> Mobile: 777733962 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Lahiru Cooray* > Software Engineer > WSO2, Inc.;http://wso2.com/ > lean.enterprise.middleware > > Mobile: +94 715 654154 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Regards, Uvindra Mobile: 777733962
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
