XiangpengHao commented on issue #6408: URL: https://github.com/apache/arrow-rs/issues/6408#issuecomment-2356034365
> On solution here would be to replace Vec<Buffer> with Arc<[Buffer]> after construction I agree, this should fix the problems. In fact, many other operations (e.g., take) also need to clone the Vec<Buffer>, changing to Arc<[Buffer]> will benefit them as well. However, I would be a bit more careful about why cloning the buffer taking so long -- often indicating the Vec<Buffer> is large, which often means gc is not being called timely. So in addition to changing to Arc<[Buffer]>, I would also examine the plan to make sure gc (in CoalesceBatchesExec) is being invoked properly. cc @WetABQ who might be interested in this discussion. I plan to work on this later this week but anyone else feel free to run faster than me! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
