This is an automated email from the ASF dual-hosted git repository.
pitrou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 6d06317ab1 GH-50313: [C++][Docs] Add guidance about memory bombs
(#50408)
6d06317ab1 is described below
commit 6d06317ab1fe3d5be478e4de282365d7021f073b
Author: Antoine Pitrou <[email protected]>
AuthorDate: Wed Jul 8 09:54:45 2026 +0200
GH-50313: [C++][Docs] Add guidance about memory bombs (#50408)
### Rationale for this change
Add guidance about handling or avoiding memory bombs in file formats that
involve optional compression.
### Are these changes tested?
N/A.
### Are there any user-facing changes?
No.
* GitHub Issue: #50313
Lead-authored-by: Antoine Pitrou <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Co-authored-by: Andrew Lamb <[email protected]>
Co-authored-by: Dewey Dunnington <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
---
docs/source/cpp/security.rst | 23 +++++++++++++++++++++++
docs/source/format/Columnar.rst | 2 ++
docs/source/format/Security.rst | 8 ++++++++
3 files changed, 33 insertions(+)
diff --git a/docs/source/cpp/security.rst b/docs/source/cpp/security.rst
index ee35f7b296..b622607b98 100644
--- a/docs/source/cpp/security.rst
+++ b/docs/source/cpp/security.rst
@@ -120,6 +120,22 @@ variants, but these typically fall into two categories:
arguments (this is typical of the :ref:`array builders
<cpp-api-array-builders>`
and :ref:`buffer builders <cpp-api-buffer-builders>`).
+Controlling and restricting memory allocation
+---------------------------------------------
+
+By construction, many Arrow C++ APIs can allocate large amounts of memory,
depending
+on their input parameters. Arrow C++ allows customizing the memory allocator
for
+such large data requests through the :ref:`MemoryPool <cpp_memory_pool>`
interface.
+
+You can therefore implement a MemoryPool class enforcing the restrictions
+of your choice (for example to limit the total number of allocated bytes), and
pass
+it to any Arrow C++ APIs you use.
+
+.. note::
+ Unlike memory used for Arrow data, smaller metadata structures (such as
field
+ names, etc.) instead rely on the C++ standard library allocators for
convenience.
+ They will therefore be invisible to the MemoryPool memory accounting.
+
Ingesting untrusted data
========================
@@ -144,6 +160,13 @@ from an untrusted source), you **must** follow these steps:
2. If the API returned successfully, validate the returned Arrow data in full
(see "Full validity" above)
+Furthermore, both the IPC and the Parquet format allow for powerful forms of
+compression, and can therefore exhibit large expansion factors when reading.
+If you need to guard against potential denial-of-service attacks that would
+exhaust available memory, we recommend you enforce memory allocation limits
+using a dedicated MemoryPool implementation (see "Controlling and restricting
+memory allocation" above).
+
CSV reader
----------
diff --git a/docs/source/format/Columnar.rst b/docs/source/format/Columnar.rst
index 2e81bd0f94..adf197887e 100644
--- a/docs/source/format/Columnar.rst
+++ b/docs/source/format/Columnar.rst
@@ -1391,6 +1391,8 @@ have two entries in each RecordBatch. For a RecordBatch
of this schema with
buffer 12: col2 data
buffer 13: col2 data
+.. _buffer-compression:
+
Compression
-----------
diff --git a/docs/source/format/Security.rst b/docs/source/format/Security.rst
index 8e630ea9a5..d1fa38de72 100644
--- a/docs/source/format/Security.rst
+++ b/docs/source/format/Security.rst
@@ -180,6 +180,11 @@ their own risks. For example, buffer offsets and sizes
encoded in IPC messages
may be out of bounds for the IPC stream; Flatbuffers-encoded metadata payloads
may carry incorrect offsets pointing outside of the designated metadata area.
+In addition, the IPC format provides optional :ref:`buffer compression
<buffer-compression>`
+using general-purpose compression algorithms. It is therefore possible to
craft an IPC
+stream or file that acts as a decompression bomb by consuming all available
memory,
+opening a potential channel for denial-of-service attacks.
+
Advice for users
----------------
@@ -194,6 +199,9 @@ It is **extremely recommended** to run dedicated validation
checks when decoding
the IPC format, to make sure that the decoding can not induce unwanted
behavior.
Failing those checks should return a well-known error to the caller, not crash.
+It is **recommended** to provide facilities for users to control the memory
+allocation behavior when reading an IPC file or stream (for example by making
+the allocator customizable).
Extension Types
===============
