andygrove commented on code in PR #684: URL: https://github.com/apache/datafusion-comet/pull/684#discussion_r1684397942
########## docs/source/user-guide/tuning.md: ########## @@ -21,6 +21,17 @@ under the License. Comet provides some tuning options to help you get the best performance from your queries. +## Memory Tuning + +Comet currently doesn't share the memory allocation from Spark but owns its own memory allocation. +That's said, Comet requires additional memory to be allocated. Comet provides some memory related configurations to help you tune the memory usage. + +By default, the amount of memory is `spark.comet.memory.overhead.factor` * `spark.executor.memory`. +`COMET_MEMORY_OVERHEAD_FACTOR`'s default value is 0.2. You can increase the factor to require more +memory for Comet to use, if you see OOM error. Review Comment: As well as solving OOM, would increasing the memory help with performance? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
