[ https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17037270#comment-17037270 ]
Alex Parrill commented on HIVE-20979: ------------------------------------- This patch isn't in 3.1.1 or 3.1.2, despite what the fix versions says. It's relatively easy to work around at least; just subclass whatever writer you want to use, and override the {{close}} method to call {{fs.close()}} after calling {{super.close()}}. > Fix memory leak in hive streaming > --------------------------------- > > Key: HIVE-20979 > URL: https://issues.apache.org/jira/browse/HIVE-20979 > Project: Hive > Issue Type: Bug > Components: Streaming > Affects Versions: 3.1.1 > Reporter: Shubham Chaurasia > Assignee: Shubham Chaurasia > Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0, 3.1.1 > > Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch, > HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch, HIVE-20979.5.patch > > > {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via > }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes > all the handlers to accumulate and hence a memory leak.}} > 2) AbstractRecordWriter creates an instance of FileSystem but does not close > it which in turn causes a leak due to accumulation in FileSystem$Cache#map > -- This message was sent by Atlassian Jira (v8.3.4#803005)