GitHub user pitrou added a comment to the discussion: It is possible to reduce peak memory usage when using datasets (to use predicate pushdown) when reading single parquet files
Yes, and it will also depend on which platform you run it on (the system allocator is obviously system-dependent). GitHub link: https://github.com/apache/arrow/discussions/47003#discussioncomment-13765866 ---- This is an automatically sent email for user@arrow.apache.org. To unsubscribe, please send an email to: user-unsubscr...@arrow.apache.org
