[GitHub] [spark] cloud-fan commented on pull request #28780: [SPARK-31952][SQL]Fix incorrect memory spill metric when doing Aggregate
cloud-fan commented on pull request #28780: URL: https://github.com/apache/spark/pull/28780#issuecomment-647945904 shall we set `sorter.totalSpillBytes`, then we can update the metrics correctly in `sort.spill`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28780: [SPARK-31952][SQL]Fix incorrect memory spill metric when doing Aggregate
cloud-fan commented on pull request #28780: URL: https://github.com/apache/spark/pull/28780#issuecomment-647288933 > number of in-memory bytes spilled what's the difference between it and `spill (disk)`? IIUC `spill` means we dump data from memory to disk. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28780: [SPARK-31952][SQL]Fix incorrect memory spill metric when doing Aggregate
cloud-fan commented on pull request #28780: URL: https://github.com/apache/spark/pull/28780#issuecomment-644697788 I'm not familiar with this part. what does "spill (memory)" mean？ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
