DDtKey opened a new issue, #5162: URL: https://github.com/apache/arrow-datafusion/issues/5162
**Describe the bug** In the `17.0.0` DF version there is an issue with possible OOM. Probably related to usage of unbounded channels (I believe it should be avoided actually) Looks like it was introduced by this PR: https://github.com/apache/arrow-datafusion/pull/4867, because it works correctly with `MemoryPool` limits before and doesn't after. **To Reproduce** MRE to achieve this behavior is large Cartesian product: CSV File example (250mb): [GDrive link](https://drive.google.com/file/d/1q_-p8BvvO2w-0IH7SyxvDIOYK44yQIKt/view?usp=share_link) SQL: `SELECT * FROM rnd rnd1 JOIN rnd rnd2 ON rnd1."s3_drive" = rnd2."s3_drive"` **Expected behavior** In 16.0.0 it worked fine with memory-pools like: `FairSpillPool::new(4 * 1024 * 1024 * 1024)` It should return`ResourcesExhausted` error (as it used to be) **Additional context** Add any other context about the problem here. A part of this was described in the discussion here: https://github.com/apache/arrow-datafusion/issues/5108#issuecomment-1412491794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
