thisisnic commented on issue #37766: URL: https://github.com/apache/arrow/issues/37766#issuecomment-7038864397
This still reproduces on pyarrow 26.0.0: with the reprex above scaled down to 100 writes, `table.take([0])` on the memory-mapped table allocates the full 763 MiB column to return one row, while `table.slice(0, 1)` allocates nothing. The cause is that `take` on a chunked array concatenates all the chunks first (`cpp/src/arrow/compute/kernels/vector_selection_take_internal.cc:550`). That is tracked in #25822, which is older and still open, so I'm closing this one as a duplicate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
