mbutrovich commented on code in PR #3845:
URL: https://github.com/apache/datafusion-comet/pull/3845#discussion_r3023113873
##########
docs/source/contributor-guide/native_shuffle.md:
##########
@@ -81,10 +81,18 @@ Native shuffle (`CometExchange`) is selected when all of
the following condition
└─────────────────────────────────────────────────────────────────────────────┘
│ │
▼ ▼
-┌───────────────────────────────────┐ ┌───────────────────────────────────┐
-│ MultiPartitionShuffleRepartitioner │ │ SinglePartitionShufflePartitioner │
-│ (hash/range partitioning) │ │ (single partition case) │
-└───────────────────────────────────┘ └───────────────────────────────────┘
+┌───────────────────────────────────────────────────────────────────────┐
+│ Partitioner Selection │
+│ Controlled by spark.comet.exec.shuffle.partitionerMode │
+├───────────────────────────┬───────────────────────────────────────────┤
+│ immediate (default) │ buffered │
+│ ImmediateModePartitioner │ MultiPartitionShuffleRepartitioner │
+│ (hash/range/round-robin) │ (hash/range/round-robin) │
+│ Writes IPC blocks as │ Buffers all rows in memory │
Review Comment:
Could you clarify what "writes" means here? It's still buffering, I think,
so it's not a distinction from the buffered side. It's buffering as compressed
IPC now, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
