Another option:
Java function just accepts a sting and returns a integer.
eg:
int verstionStringToInt(String version){
String[] versionParts = version..replaceAll("[^\\d\\.]+", ""
).split(".");
int major = Integer.parse(versionParts[0]);
...
return (major >> 20 ) + ( minor >> 10) + patch;
}
And we store the int. Then we don't need to sort and all versions at time
of insert.
On Thu, Nov 17, 2016 at 8:47 AM, Uvindra Dias Jayasinha <[email protected]>
wrote:
> Had a chat with Jo and NuwanD,
>
> Jo elaborated more on his point of having an additional numeric column to
> determine the latest version. How this will work is,
>
> 1. Before inserting a new API version we use the existing API Version
> comparator class to sort all available versions of the API into a
> collection.
>
> 2. Now the collection has the APIs sorted from least to greatest version
> and we can use the index of each respective object in the collection to
> populate the numeric column field in the DB table.
>
> 3. We now have a numeric column at DB level which indicates the ascending
> order of the APIs by version.
>
> We will need to update the corresponding numeric column for all versions
> of a given API when a new API is inserted. When retrieving data we can now
> simply get the API with the max value of the numeric column to find its
> latest version.
>
> So there is a performance gain when doing things this way as opposed to
> figuring out the latest version every single time using the version
> comparator class if we were to do this purely at Java level.
>
>
> There are Pros and Cons with both DB and Java level pagination. At the
> moment we thought it is better to delay the decision of committing to a
> specific implementation. We can decide once we integrate with the user core.
>
> Thanks all for the feedback
>
>
>
> On 17 November 2016 at 10:05, Uvindra Dias Jayasinha <[email protected]>
> wrote:
>
>> We are not doing this to avoid vendor specific queries.
>>
>> The real reason why we want to go down this path, we have no straight
>> forward way of paginating at DB level along with taking into a account the
>> latest version of a given API. To support this at DB level properly means
>> we need to really complicate things and even then we are not sure we can
>> actually meet the need. Being able to avoid vendor specific queries is just
>> a nice side effect.
>>
>> Having thought about the concern Jo raised, I think we can simply avoid
>> using a cache for searching. The cache will only be used for things like
>> API listing. Search results will always be served from the DB and paginated
>> at Java level.
>>
>> We can dispense with the cache altogether for now and add later on if
>> performance is not enough. The main thing is that we want to move
>> pagination logic to the Java level.
>>
>>
>> On 16 November 2016 at 21:14, Lahiru Cooray <[email protected]> wrote:
>>
>>> +1 for Jo's opinion on using RBMS based pagination.
>>>
>>> I think we are trying here to make the code unnecessarily complex by
>>> introducing caches. As discussed this may become more complex when we do
>>> sorting/searching operations.
>>> Also regarding the performance, I don't see a significant performance
>>> gain by introducing our own cache as RDBMS's normally maintain query caches.
>>>
>>> Only benefit I see here is avoiding the vendor specific queries.
>>> How about addressing this concern properly by introducing a factory
>>> method and giving the flexibility to plug a new vendor easily when needed.
>>>
>>> On Wed, Nov 16, 2016 at 3:23 PM, Uvindra Dias Jayasinha <
>>> [email protected]> wrote:
>>>
>>>> Hi Jo,
>>>>
>>>> On 16 November 2016 at 14:47, Joseph Fonseka <[email protected]> wrote:
>>>>
>>>>> Hi Uvindra
>>>>>
>>>>> On Wed, Nov 16, 2016 at 1:48 PM, Uvindra Dias Jayasinha <
>>>>> [email protected]> wrote:
>>>>>>
>>>>>>
>>>>>> Searching and sorting will be done at DB level via SQL. It is only
>>>>>> the pagination of the returned result set that will be done at Java
>>>>>> level.
>>>>>>
>>>>>
>>>>> So does this mean for different search request you will be caching
>>>>> separate sets of data ? If that is so will it exhaust the memory and how
>>>>> will API search perform under a load. And since the servers will be
>>>>> running
>>>>> on containers available memory would be even less.
>>>>>
>>>>
>>>> Thats a good point, definitely should not be caching different search
>>>> request results.. Will discuss this with the team and get back to you.
>>>>
>>>>>
>>>>> Why would this be more efficient than Java? We achieve a similar thing
>>>>>> by using a local cache so there is no need to have an intermediate in
>>>>>> memory DB. What was proposed makes sense when you consider that we a
>>>>>> dealing with only a very small amount of data at DB level now.
>>>>>>
>>>>>
>>>>> Considering pagination is done using DBMS, in-memory db will help to
>>>>> work only with a one data set as well as needing to maintain one set of
>>>>> pagination queries.
>>>>>
>>>>
>>>> Hmmm I still dont understand the difference between in memory DB and in
>>>> memory cache. Technically in both cases we are working with an in memory
>>>> dataset.
>>>>
>>>>>
>>>>>
>>>>>> In the future if we foresee a requirement for DB level pagination we
>>>>>> can implement that within the DAO layer, but for now this is more than
>>>>>> enough to meet our needs. This way we can avoid introducing unwanted
>>>>>> complexity upfront since there is no rationale for doing so.
>>>>>>
>>>>>
>>>>> Agreed, but if the complexity is to support pagination for all the
>>>>> DBMSs we can at-least focus on adding db level pagination for commonly
>>>>> used
>>>>> DBMSs (ie MySQL, Oracle).
>>>>>
>>>>
>>>> Its not a complexity to support for multiple DB's, we can do that but
>>>> its an additional effort we can avoid by going down this path.
>>>>
>>>>>
>>>>> Regards
>>>>> Jo
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>> Thanks & Regards
>>>>>>> Jo
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>>> With regards,
>>>>>>>>> *Manu*ranga Perera.
>>>>>>>>>
>>>>>>>>> phone : 071 7 70 20 50
>>>>>>>>> mail : [email protected]
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Architecture mailing list
>>>>>>>>> [email protected]
>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Uvindra
>>>>>>>>
>>>>>>>> Mobile: 777733962
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Architecture mailing list
>>>>>>>> [email protected]
>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> --
>>>>>>> *Joseph Fonseka*
>>>>>>> WSO2 Inc.; http://wso2.com
>>>>>>> lean.enterprise.middleware
>>>>>>>
>>>>>>> mobile: +94 772 512 430
>>>>>>> skype: jpfonseka
>>>>>>>
>>>>>>> * <http://lk.linkedin.com/in/rumeshbandara>*
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Architecture mailing list
>>>>>>> [email protected]
>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Uvindra
>>>>>>
>>>>>> Mobile: 777733962
>>>>>>
>>>>>> _______________________________________________
>>>>>> Architecture mailing list
>>>>>> [email protected]
>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> --
>>>>> *Joseph Fonseka*
>>>>> WSO2 Inc.; http://wso2.com
>>>>> lean.enterprise.middleware
>>>>>
>>>>> mobile: +94 772 512 430
>>>>> skype: jpfonseka
>>>>>
>>>>> * <http://lk.linkedin.com/in/rumeshbandara>*
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Architecture mailing list
>>>>> [email protected]
>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Uvindra
>>>>
>>>> Mobile: 777733962
>>>>
>>>> _______________________________________________
>>>> Architecture mailing list
>>>> [email protected]
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> *Lahiru Cooray*
>>> Software Engineer
>>> WSO2, Inc.;http://wso2.com/
>>> lean.enterprise.middleware
>>>
>>> Mobile: +94 715 654154
>>>
>>> _______________________________________________
>>> Architecture mailing list
>>> [email protected]
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> Regards,
>> Uvindra
>>
>> Mobile: 777733962
>>
>
>
>
> --
> Regards,
> Uvindra
>
> Mobile: 777733962
>
> _______________________________________________
> Architecture mailing list
> [email protected]
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>
--
With regards,
*Manu*ranga Perera.
phone : 071 7 70 20 50
mail : [email protected]
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
