[ https://issues.apache.org/jira/browse/SPARK-16593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Biao Ma updated SPARK-16593: ---------------------------- Summary: Provide a pre-fetch mechanism to accelerate shuffle stage. (was: a) > Provide a pre-fetch mechanism to accelerate shuffle stage. > ---------------------------------------------------------- > > Key: SPARK-16593 > URL: https://issues.apache.org/jira/browse/SPARK-16593 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Reporter: Biao Ma > Priority: Minor > Labels: features > > Currently, the `NettyBlockRpcServer` will reading data through BlockManager, > while the block is not cached in memory, the data should be read from DISK > first, then into MEM. I wonder if we implement a mechanism add a message > contains the blockIds that the same as the openBlock message but one loop > ahead, then the `NettyBlockRpcServer ` will load the block ready for transfer > to the reduce side. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org