Jiandan Yang created YARN-7168:
-----------------------------------
Summary: The size of dataQueue and ackQueue in DataStreamer has no
limit when writer thread is interrupted
Key: YARN-7168
URL: https://issues.apache.org/jira/browse/YARN-7168
Project: Hadoop YARN
Issue Type: Bug
Components: client
Reporter: Jiandan Yang
In our cluster, when found NodeManager frequently FullGC when decommissioning
NodeManager, and we found the biggest object is dataQueue of DataStreamer, it
has almost 6w DFSPacket, and every DFSPacket is about 64k.
!mat.jpg|memory_analysis!
The root reason is that the size of dataQueue and ackQueue in DataStreamer has
no limit when writer thread is interrupted. I know NodeManager may stop writing
when interruped, but DFSOutputStream also could do something to avoid fullgc
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
