This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new d1e34ca6e967 [SPARK-57430][CORE] Validate length before allocation in
`Encoders.ByteArrays.decode`
d1e34ca6e967 is described below
commit d1e34ca6e9678031d66ed3d3ec7bca8a751bf3e7
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sun Jun 14 08:47:57 2026 -0700
[SPARK-57430][CORE] Validate length before allocation in
`Encoders.ByteArrays.decode`
`Encoders.ByteArrays.decode()` now validates the length prefix before
allocating the array via `Objects.checkFromIndexSize(0, length,
buf.readableBytes())`, which requires `0 <= length <= buf.readableBytes()` and
throws `IndexOutOfBoundsException` otherwise. After `readInt()`, the reader
index is past the prefix, so `readableBytes()` equals the remaining payload
bytes.
- Note that we use the Java built-in `Objects.checkFromIndexSize` because
we cannot use **Netty's `checkReadableBytes`**. However, both methods throws
`IndexOutOfBoundsException` identically for the error cases.
-
https://netty.io/4.2/api/io/netty/buffer/AbstractByteBuf.html#checkReadableBytes(int)
-
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/Objects.html#checkFromIndexSize(int,int,int)
`decode()` allocated `new byte[length]` from an untrusted length.
- A negative value throws an opaque `NegativeArraySizeException`.
- An oversized value can trigger `OutOfMemoryError` on a corrupt or hostile
frame.
Validating first fails fast with a clear error.
Note that `OutOfMemoryError` can happen even in a secure environment at the
first parse on an unauthenticated channel.
https://github.com/apache/spark/blob/b6450e6765a2fbe307c0b10d744da3ab4583ba44/common/network-common/src/main/java/org/apache/spark/network/crypto/AuthRpcHandler.java#L88-L90
https://github.com/apache/spark/blob/b6450e6765a2fbe307c0b10d744da3ab4583ba44/common/network-common/src/main/java/org/apache/spark/network/crypto/AuthMessage.java#L53-L64
No.
Pass the CIs with newly added test cases in `EncodersSuite`.
Generated-by: Claude Code (Claude Opus 4.8)
Closes #56493 from dongjoon-hyun/SPARK-57430.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 5c29a931066d7bd909f47aea931ce98323e8da27)
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 704bfef931d9510e8c4df778879a58527604e162)
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 04ca5e682099cf7270dec9faafc2782bd59d8658)
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 0c1c3f975d705b1dfe6d885a331f0b1dceb7244e)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../apache/spark/network/protocol/Encoders.java | 1 +
.../spark/network/protocol/EncodersSuite.java | 22 ++++++++++++++++++++++
2 files changed, 23 insertions(+)
diff --git
a/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
b/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
index 488e36004f2f..69b7c3159b2c 100644
---
a/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
+++
b/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
@@ -107,6 +107,7 @@ public class Encoders {
public static byte[] decode(ByteBuf buf) {
int length = buf.readInt();
+ Objects.checkFromIndexSize(0, length, buf.readableBytes());
byte[] bytes = new byte[length];
buf.readBytes(bytes);
return bytes;
diff --git
a/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
b/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
index 55b09b66aef6..66299c38fa03 100644
---
a/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
+++
b/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
@@ -88,4 +88,26 @@ public class EncodersSuite {
RoaringBitmap[] decodedBitmaps = Encoders.BitmapArrays.decode(buf);
assertArrayEquals(bitmaps, decodedBitmaps);
}
+
+ @Test
+ public void testByteArraysEncodeDecode() {
+ byte[] arr = new byte[] { 1, 2, 3, 4, 5 };
+ ByteBuf buf = Unpooled.buffer(Encoders.ByteArrays.encodedLength(arr));
+ Encoders.ByteArrays.encode(buf, arr);
+ assertArrayEquals(arr, Encoders.ByteArrays.decode(buf));
+ }
+
+ @Test
+ public void testByteArraysDecodeShouldFailWhenLengthIsNegative() {
+ ByteBuf buf = Unpooled.buffer();
+ buf.writeInt(-1);
+ assertThrows(IndexOutOfBoundsException.class, () ->
Encoders.ByteArrays.decode(buf));
+ }
+
+ @Test
+ public void testByteArraysDecodeShouldFailWhenLengthExceedsReadableBytes() {
+ ByteBuf buf = Unpooled.buffer();
+ buf.writeInt(Integer.MAX_VALUE);
+ assertThrows(IndexOutOfBoundsException.class, () ->
Encoders.ByteArrays.decode(buf));
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
