Repository: spark Updated Branches: refs/heads/master ebd648058 -> 1ae51f6dc
[SPARK-4183] Enable NettyBlockTransferService by default Note that we're turning this on for at least the first part of the QA period as a trial. We want to enable this (and deprecate the NioBlockTransferService) as soon as possible in the hopes that NettyBlockTransferService will be more stable and easier to maintain. We will turn it off if we run into major issues. Author: Aaron Davidson <[email protected]> Closes #3049 from aarondav/enable-netty and squashes the following commits: bb981cc [Aaron Davidson] [SPARK-4183] Enable NettyBlockTransferService by default Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1ae51f6d Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1ae51f6d Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/1ae51f6d Branch: refs/heads/master Commit: 1ae51f6dc7e79a88dfe54a2c488b540fdbf51212 Parents: ebd6480 Author: Aaron Davidson <[email protected]> Authored: Sat Nov 1 13:15:24 2014 -0700 Committer: Patrick Wendell <[email protected]> Committed: Sun Nov 2 18:14:57 2014 -0800 ---------------------------------------------------------------------- docs/configuration.md | 10 ++++++++++ 1 file changed, 10 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/1ae51f6d/docs/configuration.md ---------------------------------------------------------------------- diff --git a/docs/configuration.md b/docs/configuration.md index 099972c..685101e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -371,6 +371,16 @@ Apart from these, the following properties are also available, and may be useful map-side aggregation and there are at most this many reduce partitions. </td> </tr> +<tr> + <td><code>spark.shuffle.blockTransferService</code></td> + <td>netty</td> + <td> + Implementation to use for transferring shuffle and cached blocks between executors. There + are two implementations available: <code>netty</code> and <code>nio</code>. Netty-based + block transfer is intended to be simpler but equally efficient and is the default option + starting in 1.2. + </td> +</tr> </table> #### Spark UI --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
