Jason Lowe created HADOOP-8942:
----------------------------------
Summary: Thundering herd of RPCs with large responses leads to OOM
Key: HADOOP-8942
URL: https://issues.apache.org/jira/browse/HADOOP-8942
Project: Hadoop Common
Issue Type: Bug
Components: ipc
Affects Versions: 0.23.3
Reporter: Jason Lowe
When a large number of clients are all making calls with large amounts of
response data then the IPC server can exhaust memory. See MAPREDUCE-4730 for
an example of this.
There does not appear to be any flow control between the server's handler
threads and the responder thread. If a handler thread cannot write out all of
the response data without blocking, it queues up the remainder for the
responder thread and goes back to the next call in the call queue. If there
are enough clients, this can cause the handler threads to overwhelm the heap by
queueing response data faster than it can be processed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
