sunchao commented on code in PR #57355:
URL: https://github.com/apache/spark/pull/57355#discussion_r3634653602
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala:
##########
@@ -346,7 +354,8 @@ object ShuffleExchangeExec {
outputAttributes: Seq[Attribute],
newPartitioning: Partitioning,
serializer: Serializer,
- writeMetrics: Map[String, SQLMetric])
+ writeMetrics: Map[String, SQLMetric],
+ customWriteMetrics: Map[String, SQLMetric])
Review Comment:
[P1] Preserve the legacy helper overloads
This replaces the five-argument `prepareShuffleDependency` JVM entry point
with a six-argument method; the same change below replaces
`createShuffleWriteProcessor(Map)` with a two-argument method. Current
downstream accelerators still call the old methods (for example, [Apache
DataFusion Comet calls the one-argument processor
helper](https://github.com/apache/datafusion-comet/blob/52140f3930e7e11504d5df893a82c2adb08c406b/spark/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/CometShuffleExchangeExec.scala#L850)),
so source builds fail, and already-compiled plugins will hit
`NoSuchMethodError`. Please keep explicit legacy overloads that delegate with
`Map.empty` alongside the new overloads. A Scala default argument would restore
source compatibility but not the old JVM descriptors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
