2010YOUY01 commented on issue #6983: URL: https://github.com/apache/arrow-datafusion/issues/6983#issuecomment-1661290723
> I made a POC on #6984 which demonstrates the issue is indeed using more cores to do the write. However, the implementation of doing repartitioning is probably not right -- I think the better approach would be to set the target partitions when writing into memory table This POC and adding predicate seem both suppress the physical optimizer bug in repartition rule by adding another execution node on top of `ParquetExec` node 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
