[
https://issues.apache.org/jira/browse/FINERACT-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18132773#comment-18132773
]
Piotr Wargulak commented on FINERACT-3004:
------------------------------------------
See Apache Dev mailing list thread: FINERACT-3004: COB silently rolls back a
whole chunk when one loan's business step fails: loans stay locked with no
error and the job reports success
> COB silently rolls back a whole chunk when one loan's business step fails:
> loans stay locked with no error and the job reports success
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FINERACT-3004
> URL: https://issues.apache.org/jira/browse/FINERACT-3004
> Project: Apache Fineract
> Issue Type: Bug
> Reporter: Piotr Wargulak
> Assignee: Piotr Wargulak
> Priority: Major
> Attachments: FeignWorkingCapitalLoanCobBusinessStepFailureTest.java,
> LoanCobBusinessStepFailureTest.java
>
>
> When a business step throws for one loan during a scheduled Loan COB or
> Working Capital Loan COB run, every loan of that chunk is left behind without
> any trace:
> * the loans keep their {{LOAN_COB_CHUNK_PROCESSING}} lock with {{error}}
> empty, so the lock is never overrulable;
> * none of them gets {{last_closed_business_date}} moved, and none of the
> chunk's work is kept, including the loans that had no problem;
> * the step ends {{COMPLETED}} and the job run is recorded as {{{}success{}}}.
> What happens, step by step (Spring Batch 6.0.5, {{{}ChunkOrientedStep{}}}):
> # The item processor runs {{{}COBBusinessStepServiceImpl#run{}}}, which is
> {{@Transactional}} and joins the chunk transaction. The step's exception
> leaves that method, so the JPA transaction manager marks the chunk
> transaction rollback-only ("Participating transaction failed - marking
> existing transaction as rollback-only").
> # The worker step is fault tolerant with an immediate retry policy, so
> {{doProcess}} retries the item in the same transaction and persistence
> context. Two outcomes:
> ** the retry succeeds (it can: the first attempt's changes are still in the
> persistence context) and nothing is reported at all;
> ** the retries are used up: {{onProcessError}} is called, the item is
> skipped ({{{}doSkipInProcess{}}}), and the chunk goes on.
> # After the chunk, {{doExecute}} finds
> {{{}transactionStatus.isRollbackOnly(){}}}, rolls back and returns
> ({{{}ChunkOrientedStep:380-389{}}}). No exception, no listener, no scan; the
> next chunk starts and the step completes.
> A process skip therefore cannot work while processing writes in the chunk
> transaction: the skipped item has already doomed the transaction of the items
> processed with it.
> In the retries-used-up case,
> {{AbstractLoanItemListener#updateAccountLockWithError}} also failed for us
> with {{UnexpectedRollbackException}} (Working Capital, on our build); we have
> not established why its {{REQUIRES_NEW}} transaction is rollback-only.
> *Steps to reproduce*
> _(a GL closure on the COB date makes one loan's business step fail)_
> Two attached integration tests, each with two loans of one chunk: one whose
> step posts a journal entry on the COB date, and one with no accounting, which
> posts nothing and so is not concerned by the closure.
> * {{LoanCobBusinessStepFailureTest}} (Loan COB, based on
> {{{}LoanAccrualTransactionOnChargeSubmittedDateTest{}}}): business date
> 2023-03-03 with {{charge-accrual-date}} = {{{}submitted-date{}}}; a
> single-repayment loan with periodic accrual accounting and a penalty and a
> fee charge, and a second loan on a product from the same template with
> accounting none. Business date 2023-03-04, {{POST /glclosures}} for office 1
> on 2023-03-03, run the Loan COB job.
> {{LoanAccrualsProcessingServiceImpl#addPeriodicAccruals}} throws
> {{{}JournalEntryInvalidException{}}}.
> * {{FeignWorkingCapitalLoanCobBusinessStepFailureTest}} (Working Capital
> Loan COB): a loan on an accrual ({{{}ACC_DEF_REV_AM{}}}) product with a
> discount and a repayment on 2026-01-02, and a loan on a product without
> accounting. Business date 2026-01-03, closure on 2026-01-02, run the Working
> Capital Loan COB job.
> {{WorkingCapitalLoanDiscountFeeAmortizationServiceImpl#processDiscountFeeAmortization}}
> throws {{{}JournalEntryInvalidException{}}}.
> On develop 537e19aaa9 (PostgreSQL) both tests fail on the loan with nothing
> to post: {{expected: <2023-03-03> but was: <null>}} and {{{}expected:
> <2026-01-02> but was: <null>{}}}. After the runs both loans of each job keep
> {{LOAN_COB_CHUNK_PROCESSING}} with no error, and both job runs are recorded
> as {{{}success{}}}. The application log shows the exception, then
> "Transactional code has requested rollback".
> {*}Possible directions{*}:
> * run each loan's business steps in their own transaction (a nested one, or
> REQUIRES_NEW plus writing the loan in it), so one loan's failure does not
> doom the chunk; this changes the semantics FINERACT-2684 settled;
> * on a process failure, roll the chunk back and process it again without the
> failed loan, recording the error on its lock in an independent transaction
> (as Spring Batch 5's fault-tolerant chunk processor did for a process skip);
> * at the very least, fail the step when a chunk is rolled back this way, so
> the job does not report success while loans are left locked.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
