Had a chat with Jo and NuwanD, Jo elaborated more on his point of having an additional numeric column to determine the latest version. How this will work is,
1. Before inserting a new API version we use the existing API Version comparator class to sort all available versions of the API into a collection. 2. Now the collection has the APIs sorted from least to greatest version and we can use the index of each respective object in the collection to populate the numeric column field in the DB table. 3. We now have a numeric column at DB level which indicates the ascending order of the APIs by version. We will need to update the corresponding numeric column for all versions of a given API when a new API is inserted. When retrieving data we can now simply get the API with the max value of the numeric column to find its latest version. So there is a performance gain when doing things this way as opposed to figuring out the latest version every single time using the version comparator class if we were to do this purely at Java level. There are Pros and Cons with both DB and Java level pagination. At the moment we thought it is better to delay the decision of committing to a specific implementation. We can decide once we integrate with the user core. Thanks all for the feedback On 17 November 2016 at 10:05, Uvindra Dias Jayasinha <[email protected]> wrote: > We are not doing this to avoid vendor specific queries. > > The real reason why we want to go down this path, we have no straight > forward way of paginating at DB level along with taking into a account the > latest version of a given API. To support this at DB level properly means > we need to really complicate things and even then we are not sure we can > actually meet the need. Being able to avoid vendor specific queries is just > a nice side effect. > > Having thought about the concern Jo raised, I think we can simply avoid > using a cache for searching. The cache will only be used for things like > API listing. Search results will always be served from the DB and paginated > at Java level. > > We can dispense with the cache altogether for now and add later on if > performance is not enough. The main thing is that we want to move > pagination logic to the Java level. > > > On 16 November 2016 at 21:14, Lahiru Cooray <[email protected]> wrote: > >> +1 for Jo's opinion on using RBMS based pagination. >> >> I think we are trying here to make the code unnecessarily complex by >> introducing caches. As discussed this may become more complex when we do >> sorting/searching operations. >> Also regarding the performance, I don't see a significant performance >> gain by introducing our own cache as RDBMS's normally maintain query caches. >> >> Only benefit I see here is avoiding the vendor specific queries. >> How about addressing this concern properly by introducing a factory >> method and giving the flexibility to plug a new vendor easily when needed. >> >> On Wed, Nov 16, 2016 at 3:23 PM, Uvindra Dias Jayasinha <[email protected] >> > wrote: >> >>> Hi Jo, >>> >>> On 16 November 2016 at 14:47, Joseph Fonseka <[email protected]> wrote: >>> >>>> Hi Uvindra >>>> >>>> On Wed, Nov 16, 2016 at 1:48 PM, Uvindra Dias Jayasinha < >>>> [email protected]> wrote: >>>>> >>>>> >>>>> Searching and sorting will be done at DB level via SQL. It is only the >>>>> pagination of the returned result set that will be done at Java level. >>>>> >>>> >>>> So does this mean for different search request you will be caching >>>> separate sets of data ? If that is so will it exhaust the memory and how >>>> will API search perform under a load. And since the servers will be running >>>> on containers available memory would be even less. >>>> >>> >>> Thats a good point, definitely should not be caching different search >>> request results.. Will discuss this with the team and get back to you. >>> >>>> >>>> Why would this be more efficient than Java? We achieve a similar thing >>>>> by using a local cache so there is no need to have an intermediate in >>>>> memory DB. What was proposed makes sense when you consider that we a >>>>> dealing with only a very small amount of data at DB level now. >>>>> >>>> >>>> Considering pagination is done using DBMS, in-memory db will help to >>>> work only with a one data set as well as needing to maintain one set of >>>> pagination queries. >>>> >>> >>> Hmmm I still dont understand the difference between in memory DB and in >>> memory cache. Technically in both cases we are working with an in memory >>> dataset. >>> >>>> >>>> >>>>> In the future if we foresee a requirement for DB level pagination we >>>>> can implement that within the DAO layer, but for now this is more than >>>>> enough to meet our needs. This way we can avoid introducing unwanted >>>>> complexity upfront since there is no rationale for doing so. >>>>> >>>> >>>> Agreed, but if the complexity is to support pagination for all the >>>> DBMSs we can at-least focus on adding db level pagination for commonly used >>>> DBMSs (ie MySQL, Oracle). >>>> >>> >>> Its not a complexity to support for multiple DB's, we can do that but >>> its an additional effort we can avoid by going down this path. >>> >>>> >>>> Regards >>>> Jo >>>> >>>> >>>> >>>> >>>> >>>> >>>>> >>>>> >>>>>> Thanks & Regards >>>>>> Jo >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>>>> With regards, >>>>>>>> *Manu*ranga Perera. >>>>>>>> >>>>>>>> phone : 071 7 70 20 50 >>>>>>>> mail : [email protected] >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Uvindra >>>>>>> >>>>>>> Mobile: 777733962 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> -- >>>>>> *Joseph Fonseka* >>>>>> WSO2 Inc.; http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> mobile: +94 772 512 430 >>>>>> skype: jpfonseka >>>>>> >>>>>> * <http://lk.linkedin.com/in/rumeshbandara>* >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Uvindra >>>>> >>>>> Mobile: 777733962 >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> -- >>>> *Joseph Fonseka* >>>> WSO2 Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> mobile: +94 772 512 430 >>>> skype: jpfonseka >>>> >>>> * <http://lk.linkedin.com/in/rumeshbandara>* >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Regards, >>> Uvindra >>> >>> Mobile: 777733962 >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Lahiru Cooray* >> Software Engineer >> WSO2, Inc.;http://wso2.com/ >> lean.enterprise.middleware >> >> Mobile: +94 715 654154 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Regards, > Uvindra > > Mobile: 777733962 > -- Regards, Uvindra Mobile: 777733962
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
