jackylee-ch commented on PR #7003: URL: https://github.com/apache/incubator-gluten/pull/7003#issuecomment-2309767363
> OOM happens with 120K values, but there is no event dropped logs. Hm, perhaps it is related to hardware performance. I tried `./bin/spark-sql --master local[192]` with the default the gluten configs and above sqls, the problem can be stably reproduced in our environment.  > BTW, maybe it's a better choice to change it to false by default, and users can turn it on as needed. Since this pr has already closed the duplicate reading detection by default, It's ok to me now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
