This is an automated email from the ASF dual-hosted git repository.
zuston pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/uniffle.git
The following commit(s) were added to refs/heads/master by this push:
new 872926122 improvement(spark): Always reset decompression buffer with
explicit position and limit (#2634)
872926122 is described below
commit 872926122a67d6dc8da8fb5a9286d6dd0b86bf95
Author: Junfan Zhang <[email protected]>
AuthorDate: Tue Sep 30 10:17:39 2025 +0800
improvement(spark): Always reset decompression buffer with explicit
position and limit (#2634)
### What changes were proposed in this pull request?
Refactor the uncompression buffer to reset by the explicit position=0 and
limit=len
### Why are the changes needed?
This may be related with the bug #2630
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests
---
.../org/apache/spark/shuffle/reader/RssShuffleDataIterator.java | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git
a/client-spark/common/src/main/java/org/apache/spark/shuffle/reader/RssShuffleDataIterator.java
b/client-spark/common/src/main/java/org/apache/spark/shuffle/reader/RssShuffleDataIterator.java
index 015fd16e6..2d462e014 100644
---
a/client-spark/common/src/main/java/org/apache/spark/shuffle/reader/RssShuffleDataIterator.java
+++
b/client-spark/common/src/main/java/org/apache/spark/shuffle/reader/RssShuffleDataIterator.java
@@ -220,10 +220,11 @@ public class RssShuffleDataIterator<K, C> extends
AbstractIterator<Product2<K, C
unCompressedBytesLength += uncompressedLen;
long decompressDuration = System.currentTimeMillis() - startDecompress;
decompressTime += decompressDuration;
+
// uncompressedData's limit is not updated by `codec.decompress`,
however this information is
- // used
- // by `createKVIterator`. Update limit here.
- uncompressedData.limit(uncompressedData.position() + uncompressedLen);
+ // used by `createKVIterator`. update position and limit
+ uncompressedData.position(0);
+ uncompressedData.limit(uncompressedLen);
} else {
uncompressedData = rawData;
}