Mehakmeet Singh created HADOOP-18781:
----------------------------------------
Summary: ABFS Output stream thread pools getting shutdown during
GC.
Key: HADOOP-18781
URL: https://issues.apache.org/jira/browse/HADOOP-18781
Project: Hadoop Common
Issue Type: Bug
Components: fs/azure
Reporter: Mehakmeet Singh
Assignee: Mehakmeet Singh
Applications using AzureBlobFileSystem to create the AbfsOutputStream can use
the AbfsOutputStream for the purpose of writing, however, the OutputStream
doesn't hold any reference to the fs instance that created it, which can make
the FS instance eligible for GC, when this occurs, AzureblobFileSystem's
`finalize()` method gets called which in turn closes the FS, and in turn call
the close for AzureBlobFileSystemStore, which uses the same Threadpool that is
used by the AbfsOutputStream. This leads to the closing of the thread pool
while the writing is happening in the background and leads to hanging while
writing.
*Solution:*
Pass a backreference of AzureBlobFileSystem into AzureBlobFileSystemStore and
AbfsOutputStream as well.
Same should be done for AbfsInputStream as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
