jbonofre opened a new pull request, #1463: URL: https://github.com/apache/arrow-java/pull/1463
## What's Changed Follow-up to #1422 (#1421). This branch is stacked on #1422, so until that is merged the diff also shows its commits. Only the last commit, "Check IPC and C Data counts before using them, and release message bodies on errors", is new here. **C Data (`ArrayImporter`)** - Check `n_children` against the number of child vectors before reading the child pointers, instead of after. - Check `n_buffers` before reading the buffer pointers. It must not be negative, and for types with a fixed buffer layout it must not exceed that layout's buffer count. Before this, a huge `n_buffers` sized a `long[]` and threw `OutOfMemoryError`, which escapes the import's `catch (Exception)`. **IPC (`MessageSerializer`)** - Reject a variadic buffer count that is negative, or larger than the number of buffers left in the batch (variadic buffers are also listed in the batch buffers). `VectorLoader` casts these counts to `int` and uses them to size a list. - `deserializeMessageBatch`: release the body when the message is rejected for its size, metadata version or header type. - `deserializeRecordBatch` / `deserializeDictionaryBatch` with an `ArrowBlock`: release the block buffer when the read is short or the metadata is malformed, before the batch is deserialized. **JSON (`JsonFileReader`)** - `read()` now sets the row count after the columns are read, as `read(VectorSchemaRoot)` already does, so vectors are not first allocated for the batch count. Tests in `RoundtripTest` (buffer and child counts) and `MessageSerializerTest` (variadic counts, unsupported messages, truncated blocks). Without the fix, the new `MessageSerializerTest` cases fail, and `n_buffers = Integer.MAX_VALUE` crashes the forked test JVM with `OutOfMemoryError: Requested array size exceeds VM limit`. Closes #1462. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
