[
https://issues.apache.org/jira/browse/SPARK-45792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782931#comment-17782931
]
Alessandro Bellina commented on SPARK-45792:
--------------------------------------------
Added a link to an issue that needs to be solved in order for this SPIP to move
forward.
> SPIP: ShuffleManager short name registration via SparkPlugin
> ------------------------------------------------------------
>
> Key: SPARK-45792
> URL: https://issues.apache.org/jira/browse/SPARK-45792
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 4.0.0
> Reporter: Alessandro Bellina
> Priority: Minor
> Fix For: 4.0.0
>
>
> We would like to make it possible for a SparkPlugin to expose custom
> ShuffleManager implementations using short names, in addition to default
> configurations for each short name in order to improve ease of use.
> Today, users leveraging a spark plugin need to set a class name under
> spark.plugins and if the plugin also provides a ShuffleManager, the user
> needs to set spark.shuffle.manager to a fully qualified class name.
> Additionally, users need to make sure they have set Spark configurations that
> are required or recommended for the shuffle manager instance. This can be
> cumbersome for the user, adding barriers to the pluggable interface that
> Spark provides.
> Spark provides a short name for SortShuffleManager (“sort”) today. The idea
> in this SPIP is to make this set of short names extensible by a SparkPlugin.
> SPIP:
> https://docs.google.com/document/d/1flijDjMMAAGh2C2k-vg1u651RItaRquLGB_sVudxf6I/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
