maryannxue commented on issue #28109: [SPARK-31253][SQL][followup] Add metric to the split task number for skew optimization URL: https://github.com/apache/spark/pull/28109#issuecomment-611198182 I had thought about this when @cloud-fan first did the metrics for this class. But the problem is, when there is more than one skew partition, the number of total splits doesn't really say anything... e.g., the smallest skew could have 2 splits while the biggest skew could have 200...
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
