song fengfei created SPARK-17286:
------------------------------------
Summary: The fetched data of Shuffle read stored in the
ManagedBuffer, so its underlying data stored in off-memory or file?
Key: SPARK-17286
URL: https://issues.apache.org/jira/browse/SPARK-17286
Project: Spark
Issue Type: Question
Components: Block Manager, Input/Output, Shuffle
Reporter: song fengfei
The fetched data of Shuffle read stored in the ManagedBuffer, so its
underlying data stored in off-memory or file? if stored in off-memory, Whether
it is also managed by spark memorymanager ? and if a map output is too big,
placed it directly in the memory isn't very easily lead to oom? I did not
understand this piece, hoping to get the answer here,thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
