SteNicholas commented on code in PR #3492:
URL: https://github.com/apache/celeborn/pull/3492#discussion_r2427749008
##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala:
##########
@@ -1258,17 +1243,15 @@ class PushDataHandler(val workerSource: WorkerSource)
extends BaseMessageHandler
// During worker shutdown, worker will return HARD_SPLIT for all existed
partition.
// This should before return exception to make current push request revive
and retry.
- val isPartitionSplitEnabled =
fileWriter.getCurrentFileInfo.isPartitionSplitEnabled
-
if (shutdown.get() && (messageType == Type.REGION_START || messageType ==
- Type.PUSH_DATA_HAND_SHAKE) && isPartitionSplitEnabled) {
+ Type.PUSH_DATA_HAND_SHAKE)) {
Review Comment:
@RexXiong, celeborn removed
`celeborn.client.shuffle.mapPartition.split.enabled` to enable shuffle
partition split at default for MapPartition since 0.6.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
