pitrou commented on code in PR #48108:
URL: https://github.com/apache/arrow/pull/48108#discussion_r2522542457
##########
cpp/src/arrow/memory_pool.h:
##########
@@ -245,6 +246,64 @@ class ARROW_EXPORT ProxyMemoryPool : public MemoryPool {
std::unique_ptr<ProxyMemoryPoolImpl> impl_;
};
+/// EXPERIMENTAL MemoryPool wrapper with an upper limit
+class ARROW_EXPORT CappedMemoryPool : public MemoryPool {
+ public:
+ CappedMemoryPool(MemoryPool* wrapped_pool, int64_t bytes_allocated_limit)
+ : wrapped_(wrapped_pool), bytes_allocated_limit_(bytes_allocated_limit)
{}
+
+ using MemoryPool::Allocate;
+ using MemoryPool::Reallocate;
+
+ Status Allocate(int64_t size, int64_t alignment, uint8_t** out) override {
+ const auto attempted = size + wrapped_->bytes_allocated();
+ if (ARROW_PREDICT_FALSE(attempted > bytes_allocated_limit_)) {
+ return OutOfMemory(attempted);
+ }
Review Comment:
Fuzzing itself is not multi-threaded, but our fuzz targets use default
settings for Parquet reading, so there is probably some multi-threading going
on. That said, the likelihood of a race condition in this particular place is
rather low IMHO.
We have a couple fuzz files that can trigger OOM on the fuzzer, and this PR
successfully prevents OOM on these files (at least when tested locally).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
