ravjot28 opened a new pull request, #20615:
URL: https://github.com/apache/hudi/pull/20615

   ### Describe the issue this Pull Request addresses
   
   Closes #20609
   
   `LazyConcatenatingIterator` closes an inner iterator before it drops its 
reference to it. If that `close()` throws,
   or if opening the next inner iterator throws, the closed iterator stays 
referenced, so the next `hasNext()` or
   `close()` closes it a second time; a failed `close()` of the outer iterator 
also leaves it marked as open.
   
   ### Summary and Changelog
   
   An inner iterator of `LazyConcatenatingIterator` is closed at most once, and 
`close()` marks the iterator closed even
   when an inner close fails.
   
   - `close()` sets `closed` and clears the queued suppliers before closing the 
current inner iterator.
   - `hasNext()` drops the reference to an exhausted inner iterator before 
closing it, then opens the next one; the
     separate `init()` step is folded into this loop.
   - New tests in `TestLazyConcatenatingIterator`: an inner close that fails 
while advancing, an outer close whose
     inner close fails, and a supplier that fails after the previous iterator 
was closed.
   
   ### Impact
   
   No API or config change. The exception of a failed inner close still reaches 
the caller. If the caller then calls
   `hasNext()` again, the iterator continues with the next inner iterator 
instead of closing the failed one again. This
   is intended: an inner iterator is closed only once it is exhausted, so 
continuing skips none of its records. Callers
   that stop at the exception see no difference. The iterator is used by Spark 
and Java
   clustering and by metadata table reads.
   
   **Backport:** please include this fix in the next 1.2.x patch release. The 
commit cherry-picks cleanly onto `release-1.2.1` and `branch-1.2.x` (where the 
class is in `org.apache.hudi.util`), and its tests pass on `branch-1.2.x`.
   
   ### Risk Level
   
   low. Successful iteration is unchanged; failure paths no longer close an 
inner iterator twice. The existing tests
   of the class pass unchanged.
   
   Tests, run on master with JDK 11 and passing checkstyle checks for main and 
test sources, with
   `mvn test -pl hudi-common -am -Dtest=TestLazyConcatenatingIterator`:
   - Without the main code change: the three new tests of 
`TestLazyConcatenatingIterator` fail
     (`testFailedCloseOfExhaustedIteratorIsNotRepeated` and 
`testFailedCloseStillClosesTheIterator` with the inner
     iterator's second close throwing again, 
`testFailedSupplierDoesNotCloseThePreviousIteratorAgain` with
     `expected: <1> but was: <2>` closes).
   - With the change: `TestLazyConcatenatingIterator` 5/5 pass.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to