foxtail463 opened a new pull request, #64371: URL: https://github.com/apache/doris/pull/64371
Issue Number: close #64369 Problem Summary: RemoteFSPhantomManager currently closes the underlying Hadoop FileSystem when DFSFileSystem is GCed. If a list/read/write operation is still using objects derived from that Hadoop FileSystem, phantom cleanup can close the resource underneath the active operation and cause intermittent IO failures. Solution: Introduce DFSFileSystemResource with reference-counted leases. DFS operations and HDFS input/output wrappers acquire a lease while using the Hadoop FileSystem, and close() / phantom cleanup only request resource closure. The actual Hadoop FileSystem is closed after all active leases are released. Remove the old phantom reference wrapper and adapt tests to the lease API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
