JingsongLi commented on PR #9472: URL: https://github.com/apache/paimon/pull/9472#issuecomment-5467433288
I think this PR can be simplified and kept focused: - Keep the close forwarding in `PluginFileIO` and `ResolvingFileIO`. - In `RESTTokenFileIO`, stop closing cache values from the removal listener for now, instead of adding the lease/reference-counting API and changing the multipart, Lance, and Vortex callers in this PR. - Handle safe cleanup of REST-token cache entries in a follow-up after lazy listings have an explicit close contract. Forwarding `close()` does make the current removal listener unsafe: cache eviction can close a delegate while a returned stream or iterator is still active. However, the lease implementation in this PR still cannot make that lifecycle correct. `LeasedRemoteIterator` releases only after exhaustion or an exception, while `RemoteIterator` is not closeable. A `LIMIT`, task cancellation, or any caller that abandons a listing early permanently leaks a reference, so the cached delegate can never be closed after token rotation or eviction. Removing the listener preserves the previous effective behavior for the `PluginFileIO` / `ResolvingFileIO` paths, avoids closing a delegate while it is in use, and reduces this PR to the wrapper ownership fix. A follow-up can make `RemoteIterator` closeable with a default no-op for compatibility, propagate close through `ObjectTable` / `IteratorRecordReader` / `FileIO.listFiles`, and then re-enable reference-counted cache cleanup with an early-close regression test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
