adriangb commented on issue #19386: URL: https://github.com/apache/datafusion/issues/19386#issuecomment-3696707647
I agree with both of you: 1. Arrow memory pool API seems like a better way to address the root cause. 2. Doing some compute and continuing even if slowly is always better than an error (i.e. if we can literally replace and error in code with try to compact and then error if that doesn't help that's a win). 3. Long term we can be more thoughtful about when / what heuristics to use to auto compact. So right now I'm +1 for a change to that takes us from error -> continue slowly. I also am tracking the Arrow memory pool API changes and look forward to seeing them happen, but both things can happen at the same time and like @bharath-techie says are complementary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
