[
https://issues.apache.org/jira/browse/YARN-7168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiandan Yang updated YARN-7168:
--------------------------------
Description:
In our cluster, when found NodeManager frequently FullGC when decommissioning
NodeManager, and we found the biggest object is dataQueue of DataStreamer, it
has almost 6w DFSPacket, and every DFSPacket is about 64k, as shown below.
The root reason is that the size of dataQueue and ackQueue in DataStreamer has
no limit when writer thread is interrupted. I know NodeManager may stop writing
when interruped, but DFSOutputStream also could do something to avoid fullgc
!mat.jpg|memory_analysis!
was:
In our cluster, when found NodeManager frequently FullGC when decommissioning
NodeManager, and we found the biggest object is dataQueue of DataStreamer, it
has almost 6w DFSPacket, and every DFSPacket is about 64k.
!mat.jpg|memory_analysis!
The root reason is that the size of dataQueue and ackQueue in DataStreamer has
no limit when writer thread is interrupted. I know NodeManager may stop writing
when interruped, but DFSOutputStream also could do something to avoid fullgc
> The size of dataQueue and ackQueue in DataStreamer has no limit when writer
> thread is interrupted
> -------------------------------------------------------------------------------------------------
>
> Key: YARN-7168
> URL: https://issues.apache.org/jira/browse/YARN-7168
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
> Reporter: Jiandan Yang
> Attachments: mat.jpg
>
>
> In our cluster, when found NodeManager frequently FullGC when decommissioning
> NodeManager, and we found the biggest object is dataQueue of DataStreamer, it
> has almost 6w DFSPacket, and every DFSPacket is about 64k, as shown below.
> The root reason is that the size of dataQueue and ackQueue in DataStreamer
> has no limit when writer thread is interrupted. I know NodeManager may stop
> writing when interruped, but DFSOutputStream also could do something to avoid
> fullgc
> !mat.jpg|memory_analysis!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
