This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 647837b3acf4 [SPARK-57474][CORE] Validate length before allocation in
`Encoders.StringArrays.decode`
647837b3acf4 is described below
commit 647837b3acf4301151eccd0527cbdbdb8dae9ca8
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Mon Jun 15 12:13:35 2026 -0700
[SPARK-57474][CORE] Validate length before allocation in
`Encoders.StringArrays.decode`
### What changes were proposed in this pull request?
`Encoders.StringArrays.decode()` now validates the count prefix before
allocating the array via `Objects.checkFromIndexSize(0, numStrings,
buf.readableBytes() / 4)`, which requires `0 <= numStrings <=
buf.readableBytes() / 4` and throws `IndexOutOfBoundsException` otherwise. The
divisor `4` is the minimum on-wire size of one element, since `Strings.encode`
always writes a 4-byte length prefix like the following.
https://github.com/apache/spark/blob/118026dc9b9693613aab96ae6f4a3c126c6c3fdf/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java#L37-L41
### Why are the changes needed?
`decode()` allocated `new String[numStrings]` from an untrusted count.
- A negative value throws an opaque `NegativeArraySizeException`.
- An oversized value can trigger `OutOfMemoryError` on a corrupt or hostile
frame.
Validating first fails fast with a clear error.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs with newly added test cases.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
Closes #56521 from dongjoon-hyun/SPARK-57474.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 0caf79030573233c6db526a4289499f8756adeaf)
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 727db8f9e8b58c8022ddc532c946f17d9aff9edf)
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 3960c2ae72fd3b88270204d4b805c450b1062898)
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit db86780f5846dcac90d7153a41377b9f770c67e7)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../apache/spark/network/protocol/Encoders.java | 1 +
.../spark/network/protocol/EncodersSuite.java | 22 ++++++++++++++++++++++
2 files changed, 23 insertions(+)
diff --git
a/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
b/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
index d69ad7c1f9ce..a91200e825d2 100644
---
a/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
+++
b/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
@@ -133,6 +133,7 @@ public class Encoders {
public static String[] decode(ByteBuf buf) {
int numStrings = buf.readInt();
+ Objects.checkFromIndexSize(0, numStrings, buf.readableBytes() / 4);
String[] strings = new String[numStrings];
for (int i = 0; i < strings.length; i ++) {
strings[i] = Strings.decode(buf);
diff --git
a/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
b/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
index 0a07ebf17ded..24b766639210 100644
---
a/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
+++
b/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
@@ -154,4 +154,26 @@ public class EncodersSuite {
buf.writeInt(Integer.MAX_VALUE);
assertThrows(IndexOutOfBoundsException.class, () ->
Encoders.LongArrays.decode(buf));
}
+
+ @Test
+ public void testStringArraysEncodeDecode() {
+ String[] arr = new String[] { "spark", "", "rocks" };
+ ByteBuf buf = Unpooled.buffer(Encoders.StringArrays.encodedLength(arr));
+ Encoders.StringArrays.encode(buf, arr);
+ assertArrayEquals(arr, Encoders.StringArrays.decode(buf));
+ }
+
+ @Test
+ public void testStringArraysDecodeShouldFailWhenLengthIsNegative() {
+ ByteBuf buf = Unpooled.buffer();
+ buf.writeInt(-1);
+ assertThrows(IndexOutOfBoundsException.class, () ->
Encoders.StringArrays.decode(buf));
+ }
+
+ @Test
+ public void testStringArraysDecodeShouldFailWhenLengthExceedsReadableBytes()
{
+ ByteBuf buf = Unpooled.buffer();
+ buf.writeInt(Integer.MAX_VALUE);
+ assertThrows(IndexOutOfBoundsException.class, () ->
Encoders.StringArrays.decode(buf));
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
