Biao Ma created SPARK-16593:
-------------------------------
Summary: Provide a pre-fetch mechanism to accelerate shuffle
stage.
Key: SPARK-16593
URL: https://issues.apache.org/jira/browse/SPARK-16593
Project: Spark
Issue Type: Improvement
Components: Spark Core
Reporter: Biao Ma
Priority: Minor
Currently, the `NettyBlockRpcServer` will reading data through BlockManager,
while the block is not cached in memory, the data should be read from DISK
first, then into MEM. I wonder if we implement a mechanism add a message
contains the blockIds that the same as the openBlock message but one loop
ahead, then the `NettyBlockRpcServer ` will load the block ready for transfer
to the reduce side.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
