singhpk234 commented on code in PR #16768:
URL: https://github.com/apache/iceberg/pull/16768#discussion_r3454500714
##########
core/src/main/java/org/apache/iceberg/rest/ScanTaskIterable.java:
##########
@@ -103,11 +104,25 @@ private void submitFixedWorkers() {
@Override
public CloseableIterator<FileScanTask> iterator() {
- return new ScanTasksIterator();
+ ScanTasksIterator iter = new ScanTasksIterator();
+ addCloseable(iter);
+ return iter;
}
@Override
- public void close() throws IOException {}
+ public void close() throws IOException {
+ if (shutdown.compareAndSet(false, true)) {
+ LOG.info(
+ "ScanTaskIterable is closing. Clearing {} queued tasks, {} plan
tasks, and {} initial file scan tasks.",
+ taskQueue.size(),
+ planTasks.size(),
+ initialFileScanTasks.size());
+ super.close();
+ taskQueue.clear();
+ planTasks.clear();
+ initialFileScanTasks.clear();
Review Comment:
Lets keep the iterator close logic in the iterator and rely on closableGroup
traking the iterator is calling iterator close instead ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
