Tedyyy-Albur commented on code in PR #5120:
URL: https://github.com/apache/fineract/pull/5120#discussion_r2467321562
##########
fineract-provider/src/main/java/org/apache/fineract/portfolio/savings/jobs/postinterestforsavings/PostInterestForSavingTasklet.java:
##########
@@ -166,22 +166,6 @@ private void postInterest(List<SavingsAccountData>
savingsAccounts, int threadPo
List<Future<Void>> responses = new ArrayList<>();
posters.forEach(poster -> responses.add(taskExecutor.submit(poster)));
- Long maxId = maxSavingsIdInList;
- if (!queue.isEmpty()) {
- maxId = Math.max(maxSavingsIdInList,
queue.element().get(queue.element().size() - 1).getId());
- }
-
- while (queue.size() <= QUEUE_SIZE) {
- log.debug("Fetching while threads are running!..:: this is not
supposed to run........");
- savingsAccounts =
Collections.synchronizedList(this.savingAccountReadPlatformService
-
.retrieveAllSavingsDataForInterestPosting(backdatedTxnsAllowedTill, pageSize,
ACTIVE.getValue(), maxId));
- if (savingsAccounts.isEmpty()) {
- break;
- }
- maxId = savingsAccounts.get(savingsAccounts.size() - 1).getId();
- log.debug("Add to the Queue");
- queue.add(savingsAccounts);
- }
Review Comment:
@adamsaghy @galovics
> **why it only occurs while processing large amounts of data. if this is
duplicate, why it was not occurred for "small amount of data"**
The problem is only observed with large volumes because concurrent
processing exposed a race condition between the `fetchData` thread (which fills
the queue) and the redundant block at the end of the method.
In small scenarios, execution is sequential, and the second block rarely
executes before the process terminates, so the duplication was not noticed.
> **Do we know why this was there originally? -> How we can be sure to not
break anything?**
No, we don't know, but it's likely this block was added as a backup measure
to ensure the queue never became empty if the upload thread (`fetchData`)
stopped.
`fetchData` handles incremental account uploads in a controlled manner.
Keeping both mechanisms active led to race conditions and duplicate processing
under high load.
Therefore, removing the block does not remove any legitimate functionality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
