wombatu-kun opened a new pull request, #9472: URL: https://github.com/apache/paimon/pull/9472
### Purpose Split out of #8962 at @JingsongLi's request. Answers [this review comment](https://github.com/apache/paimon/pull/8962#discussion_r3703449620). `PluginFileIO` and `ResolvingFileIO` never forwarded `close()` to their delegates, unlike `CachingFileIO`, so `OSSFileIO.close()` has never run and `RESTCatalog`'s `try (FileIO fileIO = fileIOFromOptions(path))` releases nothing. Both forward now, and `close()` is terminal so the wrappers cannot silently rebuild a delegate nobody will release. That also makes `RESTTokenFileIO`'s removal listener close a delegate for real. Its cache is JVM wide and hands out raw `FileIO`s and streams, so an eviction could close an OSS file system while another table is still reading through it. Values are reference counted now: the listener hands back only the cache's own reference, and the delegate is closed when the last lease goes. Leases cover each operation and the lifetime of every stream and listing handed out, and the caller's lease is taken before the put, since the admission policy can evict a just-inserted entry. `BaseMultiPartUploadCommitter`, `LanceUtils` and `VortexUtils` work inside a lease instead of unwrapping the cached instance. `fileIO()` is deprecated, since a raw reference carries no lifetime to track, but kept because engines cast the result to their own implementation. Cut from current master and shares no file with #8962 or #9471, so the three can be reviewed in any order. ### Tests `RESTTokenFileIOTest` (new): an eviction keeps a leased `FileIO` alive and closes an unleased one; an open input, output or two-phase stream keeps its delegate alive until the stream ends; the wrapper carries `VectoredReadable` through and does not fake it; a failed open hands the lease back; a spent entry is rebuilt; a lease closed twice releases once; the multipart committer holds a lease across the call. `PluginFileIOTest`, `ResolvingFileIOTest`: close releases every delegate under the plugin classloader, keeps going when one fails, and use after close is rejected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
