comphead commented on code in PR #3924: URL: https://github.com/apache/datafusion-comet/pull/3924#discussion_r3070755768
########## docs/source/user-guide/latest/tuning.md: ########## @@ -56,15 +56,29 @@ For more details about Spark off-heap memory mode, please refer to [Spark docume Comet implements multiple memory pool implementations. The type of pool can be specified with `spark.comet.exec.memoryPool`. +When Comet executes a shuffle, it creates two separate native plans within the same Spark task: the child plan Review Comment: maybe it is not a plan, and rather a stage DAG? plan is per operator: scan-> shuffle -> write stage DAG is different, the plan above split into Stage1: scan + shuffle write Stage2: shuffle read + write -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
