FelixYBW commented on PR #5952: URL: https://github.com/apache/incubator-gluten/pull/5952#issuecomment-2254775734
Looks like the generator operator already limit the output rowvector size to the one we set. The OOM isn't caused by C2R, but the R2C after parquet scan, because the struct column is too large. The way to solve OOM is to decrease the batch. Another operator which may output large batch size is hashjoin. So maybe the PR is still useful.  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
