This is an automated email from the ASF dual-hosted git repository.
jackylee pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 06956efef5 [GLUTEN-10506] Include compression time in the shuffle
write metric for Uniffle (#10507)
06956efef5 is described below
commit 06956efef5f39b83352e0baa5af5e7a615377a70
Author: Junfan Zhang <[email protected]>
AuthorDate: Fri Aug 22 17:43:22 2025 +0800
[GLUTEN-10506] Include compression time in the shuffle write metric for
Uniffle (#10507)
---
.../spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/backends-velox/src-uniffle/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java
b/backends-velox/src-uniffle/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java
index 3240a50064..0951e0b809 100644
---
a/backends-velox/src-uniffle/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java
+++
b/backends-velox/src-uniffle/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java
@@ -245,7 +245,9 @@ public class VeloxUniffleColumnarShuffleWriter<K, V>
extends RssShuffleWriter<K,
// bytesWritten is calculated in uniffle side:
WriteBufferManager.createShuffleBlock
// shuffleWriteMetrics.incBytesWritten(splitResult.getTotalBytesWritten());
shuffleWriteMetrics.incWriteTime(
- splitResult.getTotalWriteTime() + splitResult.getTotalPushTime());
+ splitResult.getTotalWriteTime()
+ + splitResult.getTotalPushTime()
+ + splitResult.getTotalCompressTime());
// partitionLengths is calculate in uniffle side
long pushMergedDataTime = System.nanoTime();
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
