harishjp commented on PR #5548: URL: https://github.com/apache/hive/pull/5548#issuecomment-2481417515
> > > This method is not even called upstream! > > > > > > I'm sorry, I do not understand which method is not called upstream? close or filterBatch? > > CloseableIterator#close(). Instead CloseableIterable#close is invoked. I've refactored the IcebergRecordReader#nextKeyValue not to use iceber's `CloseableIterable.concat` and looking into positional-delete readers leak - seems to be some race condition in deletes loading. > > I can repro the issue locally, and I do not see how proposed change even supposed to help. `deleteInputIterable` is just a closeable wrapper over `srcIterator`, and `deleteOutputIterable` is the result of filter transform on the same `srcIterator`. Closing `srcIterator` should be enough to release the data-files connection. <img alt="Screenshot 2024-11-17 at 18 45 37" width="641" src="https://private-user-images.githubusercontent.com/2761528/386984282-4177082a-50da-4adc-88a0-0764a528b2ea.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzE4NjgwMzAsIm5iZiI6MTczMTg2NzczMCwicGF0aCI6Ii8yNzYxNTI4LzM4Njk4NDI4Mi00MTc3MDgyYS01MGRhLTRhZGMtODhhMC0wNzY0YTUyOGIyZWEucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MTExNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDExMTdUMTgyMjEwWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR 1cmU9NzZmNTU3ZmE5MWVmZjQ5MzRiZjYwMDlmMjhhNGQxZTVmYWM3OTM1ODY1MmNhZDllZWNjYTZiYTdiMjNjNWQyOCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.mjkFu77sHkXdmLGeyfPX2HaRG1khS_Nojrt07NKpFWY"> Thank you for the clarification, yes you are right. This patch is not required for master. I do not know if there is a leak in the master, I'll close this ticket and patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
