yifeih commented on issue #23914: [SPARK-27009] Add Standard Deviation to Benchmark tests URL: https://github.com/apache/spark/pull/23914#issuecomment-468083315 We're writing some microbenchmark tests to get ready for some refactoring changes that we're making to the spark shuffle code in order to plan for an API that allows for resilient external shuffle. We're planning on running these benchmarks for each PR change that we make, comparing each PR's benchmark numbers to the previous PR's. Having standard deviation would help us see whether we unexpectedly introduced any increases in variance at each stage, and whether the changes in the average times from one PR to the next are significant or not. More info on the shuffle change that we're planning for: Jira ticket here: https://issues.apache.org/jira/browse/SPARK-25299 Benchmarks PR here: https://github.com/palantir/spark/pull/498 Project plan here: https://docs.google.com/document/d/1NQW1XgJ6bwktjq5iPyxnvasV9g-XsauiRRayQcGLiik/edit#) Let me know if that makes sense or not, thanks!
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
