Mehakmeet Singh created HADOOP-18781:
----------------------------------------

             Summary: ABFS Output stream thread pools getting shutdown during 
GC.
                 Key: HADOOP-18781
                 URL: https://issues.apache.org/jira/browse/HADOOP-18781
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs/azure
            Reporter: Mehakmeet Singh
            Assignee: Mehakmeet Singh


Applications using AzureBlobFileSystem to create the AbfsOutputStream can use 
the AbfsOutputStream for the purpose of writing, however, the OutputStream 
doesn't hold any reference to the fs instance that created it, which can make 
the FS instance eligible for GC, when this occurs, AzureblobFileSystem's 
`finalize()` method gets called which in turn closes the FS, and in turn call 
the close for AzureBlobFileSystemStore, which uses the same Threadpool that is 
used by the AbfsOutputStream. This leads to the closing of the thread pool 
while the writing is happening in the background and leads to hanging while 
writing.

 

*Solution:*
Pass a backreference of AzureBlobFileSystem into AzureBlobFileSystemStore and 
AbfsOutputStream as well.

 

Same should be done for AbfsInputStream as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Reply via email to