Hi everyone,
I would like to share the initial results of our testing and get your
feedback.
We modified the code for the post interest to savings scheduler job,
specifically the findByStatus method in SavingsRepositoryWrapper where it
queries the active savings accounts. We noticed that it the current design
depended on lazy load fetching to populate the transactions and charges
list of each savings accounts. From previous experience with other systems
this has been a cause of various slow downs so we focused on modifying this
part. We decided to query the savings transaction and charges in bulk to
reduce the number of database calls. See below for our implementation.
We also removed the CascadeType.ALL and FetchType.Lazy settings for the
transactions and charges list of the SavingsAccount entity as we are
already manually fetching their contents. We will do further testing as
this may have an impact in other modules.
@Transactional(readOnly=true)
public Page<SavingsAccount> findByStatus(Integer status, Pageable
pageable) {
logger.info("findByStatus - Start querying savings account");
Page<SavingsAccount> accounts =
this.repository.findByStatus(status, pageable);
List<Long> idList = new ArrayList<Long>();
Map<Long, SavingsAccount> accountsMap = new HashMap<Long,
SavingsAccount>();
if(accounts != null) {
for(SavingsAccount account : accounts) {
account.setCharges(new HashSet<>());
account.setTransactions(new ArrayList<>());
idList.add(account.getId());
accountsMap.put(account.getId(), account);
}
List<SavingsAccountTransaction>
savingsAccountTransactionList =
savingsAccountTransactionRepository.findBySavingsAccountIdList(idList);
if(savingsAccountTransactionList != null) {
for(SavingsAccountTransaction transaction :
savingsAccountTransactionList) {
SavingsAccount account =
accountsMap.get(transaction.getSavingsAccount().getId());
account.getTransactions().add(transaction);
}
}
Set<SavingsAccountCharge> savingsAccountChargeList =
savingsAccountChargeRepository.findBySavingsAccountIdList(idList);
if(savingsAccountChargeList != null) {
for(SavingsAccountCharge charges : savingsAccountChargeList) {
SavingsAccount account =
accountsMap.get(charges.savingsAccount().getId());
account.getCharges().add(charges);
}
}
}
logger.info("findByStatus - Finished querying savings
account");// loadLazyCollections(accounts);
return accounts;
}
We were able to reduce the post interest to savings schedule job run time
from 15-20 hours to 6 hours with these modifications. After this we will
look at how to reduce the run time for the saving/updating part.
I would like to ask if anyone has any alternative solutions or if you have
any feedback on how we implemented it?
Regards,
Joseph
On Sun, Oct 20, 2019 at 11:02 PM Joseph Cabral <joseph.cabra...@gmail.com>
wrote:
> Hi Michael,
>
> Of course, we will give feedback if we are able to make improvements to
> our scheduler job run time.
>
> But if anyone else has any experience in load testing or running Fineract
> in high load environments I am open to suggestions.
>
> Thanks!
>
> Joseph
>
> On Sun, Oct 20, 2019 at 7:32 PM Michael Vorburger <m...@vorburger.ch>
> wrote:
>
>> Joseph,
>>
>> On Sun, 20 Oct 2019, 00:28 Joseph Cabral, <joseph.cabra...@gmail.com>
>> wrote:
>>
>>> Hi Nayan,
>>>
>>> Thank you for the tips! I tried asking for advice here first because we
>>> are wary of doing any code change since we were under the assumption that
>>> Fineract had already been used in many high load situations in production
>>> and we have just set it up wrong or there are some settings we could
>>> change.
>>>
>>> We will first try adding some indexes to the database but it looks like
>>> we have to do some code change for this.
>>>
>>
>> Will you be contributing any performance related improvements you make
>> back to the community?
>>
>> Thanks again!
>>>
>>> Joseph
>>>
>>> On Sun, Oct 20, 2019 at 2:21 AM Nayan Ambali <nayan.amb...@gmail.com>
>>> wrote:
>>>
>>>> Joseph,
>>>>
>>>> Previously I had done Mifos platform load testing for the community,
>>>> based on my experience below are my recommendations
>>>>
>>>> *without code change*
>>>> 1. Use SSD/High IOPS storage for Database
>>>> 2. If you are on AWS, go for Aurora instead of MySQL
>>>> 3. Look at database usage for this batch job and see if there is
>>>> opportunity index some columns for better performance
>>>>
>>>> *with code change*
>>>> 1. Process the data parallel either on with multi threading on single
>>>> node or we can go for multiple nodes
>>>> 2. Query optimisation
>>>> 3. Data fetch and commit batch size
>>>> there are many other opportunities to improve the same
>>>>
>>>> -
>>>> at your service
>>>>
>>>> Nayan Ambali
>>>> +91 9591996042
>>>> skype: nayangambali
>>>>
>>>>
>>>> On Sat, Oct 19, 2019 at 2:20 PM Joseph Cabral <
>>>> joseph.cabra...@gmail.com> wrote:
>>>>
>>>>> Hi Everyone,
>>>>>
>>>>> I would like to ask if anyone has done load testing of the
>>>>> MifosX/Fineract scheduler jobs? Specifically the post interest to savings
>>>>> job.
>>>>>
>>>>> We created a new Savings Product (ADB Monthly - 10% Interest) with the
>>>>> following settings:
>>>>> Nominal interest rate (annual): 10%
>>>>> Balance required for interest calculation: $1,000.00
>>>>> Interest compounding period: Monthly
>>>>> Interest posting period: Monthly
>>>>> Interested calculated using: Average Daily Balance
>>>>>
>>>>> We populated the m_savings_account table using the savings product
>>>>> above with 1.2M new savings accounts which we then deposited an initial
>>>>> balance of $10,000 each. We then edited the post interest to savings job
>>>>> to
>>>>> post the interest even though it is not yet the end of the month.
>>>>>
>>>>> On consecutive tests, we averaged around 15 to 20 hours to complete
>>>>> the job.
>>>>>
>>>>> We are using a 4vCPU 128GB server to do the load testing. We deployed
>>>>> MifosX/Fineract in a docker container with Tomcat 7. The MySQL database is
>>>>> deployed in a separate docker container on the same machine.
>>>>>
>>>>> Any tips or ideas on what we can do to improve the run time of the
>>>>> job? Our target for 1.2M savings accounts is less than 1 hour.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Joseph
>>>>>
>>>>
_______________________________________________
Mifos-users mailing list
Mifos-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mifos-users
