joshdoesthis opened a new issue, #526:
URL: https://github.com/apache/arrow-js/issues/526
### Describe the bug, including details regarding any error messages,
version, and platform.
Decoding an IPC stream with one corrupted vector length in a dictionary
batch header makes `tableFromIPC` allocate until the process aborts with
`JavaScript heap out of memory`, instead of throwing an error.
**Version and platform:** `apache-arrow` 21.2.0 (also 21.1.0), Node
v26.13.0, macOS arm64.
**Reproduction.** The bytes are a valid stream from
`tableToIPC(tableFromArrays({ id: Int32Array.from([0, 0]), name: ["", ""] }),
"stream")`. Without the patch line it decodes 2 rows. (`name` is
dictionary-encoded, so the stream holds a dictionary batch.) With it, the
dictionary batch's `buffers` vector length goes from 3 to `0x06000003`:
```js
import { tableFromIPC } from "apache-arrow";
const bytes = Buffer.from(
"/////9gAAAAQAAAAAAAKABAADgAHAAgACgAAAAAAAAEIAAAAAAAEAKj///8EAAAAAgAAAHgAAAAUAAAAEAAcAAgADwAXABgABAAQABAAAAA4AAAAFAAAAAAAAAEYAAAAAAAABRgAAAAEAAAAbmFtZQAAAAAAAAAABAAEAAQAAAAIAAgAAAAEAAgAAAAEAAAAxP///wAAAAEgAAAAEAAUAAQAAAAPABAAAAAIABAAAAAQAAAAFAAAAAAAAAIYAAAAAgAAAGlkAAAAAAAACAAMAAgABwAIAAAAAAAAASAAAAD/////sAAAABQAAAAAAAAADAAcABoAEwAUAAQADAAAAAgAAAAAAAAAAAAAAAAAAAIQAAAAAAAEAAgACgAAAAQACAAAABAAAAAAAAoAGAAMAAgABAAKAAAAFAAAAEgAAAABAAAAAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////uAAAABQAAAAAAAAADAAWABQADwAQAAQADAAAABAAAAAAAAAAAAAAAxAAAAAEAAoAGAAMAAgABAAKAAAAFAAAAFgAAAACAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAAAAAACAAAAAgAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=",
"base64"
);
bytes[335] = 6;
tableFromIPC(bytes);
```
**Observed:** with the default heap, it runs for about 56 seconds, reaches
about 2.7 GB resident and aborts:
```
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed -
JavaScript heap out of memory
```
The same happens with byte 334 set to 255, and with the same header's
`nodes` length (bytes 390 and 391). With `--max-old-space-size=256` it aborts
in a few seconds.
**Expected:** a thrown error saying the header is invalid. A vector length
in a message header can't be larger than the metadata that holds it, so the
reader could check each declared length (for example a batch's `nodes` and
`buffers`) against the bytes present before allocating.
**Why it matters:** anything that decodes IPC it didn't produce, such as an
Arrow Flight client reading `FlightData` from a server, can be stopped by one
bad message. We now run a bounds check before decoding in our own Flight
client, but every other consumer of `tableFromIPC` and `RecordBatchReader` is
still exposed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
