ravjot28 opened a new issue, #20609:
URL: https://github.com/apache/hudi/issues/20609

   ### Bug Description
   
   **What happened:**
   
   `LazyConcatenatingIterator` closes an inner iterator before it drops its 
reference to it. When that `close()` throws,
   or when opening the next inner iterator throws, the closed iterator stays 
referenced. The next `hasNext()` or
   `close()` calls its `close()` again, and a failed `close()` of the outer 
iterator leaves it marked as open.
   
   **What you expected:**
   
   Each inner iterator is closed at most once, and after `close()` the iterator 
is closed even if an inner close failed.
   
   **Steps to reproduce:**
   
   1. Concatenate two iterators whose first throws from `close()`.
   2. Read the first iterator to its end: `hasNext()` closes it and throws.
   3. Call `hasNext()` again: the first iterator's `close()` runs a second time 
and throws again; the second iterator
      is never reached.
   4. Separately: with the first iterator open, call `close()` twice: the first 
iterator's `close()` runs twice.
   
   **Root cause (master 9a7728c5862):**
   
   
`hudi-common/src/main/java/org/apache/hudi/common/util/collection/LazyConcatenatingIterator.java`:
   - `close()` (`:49-58`) calls `itr.close()` (`:52`) before clearing `itr` and 
before setting `closed` (`:56`).
   - `hasNext()` (`:61-76`) calls `itr.close()` (`:68`) and then opens the next 
iterator (`:71`) while `itr` still points
     at the closed one.
   
   The class is used by Spark clustering (`MultipleSparkJobExecutionStrategy`,
   `SingleSparkJobConsistentHashingExecutionStrategy`), Java clustering 
(`JavaExecutionStrategy`) and metadata table
   reads (`HoodieBackedTableMetadata`), where the inner iterators hold file 
readers.
   
   ### Environment
   
   **Hudi version:** master (1.3.0-SNAPSHOT at 9a7728c5862), 1.2.1
   **Query engine:** any (hudi-common)
   **Relevant configs:** none
   
   ### Logs and Stack Trace
   
   The second close surfaces as whatever the inner reader throws when closed 
twice, after the first failure.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to