Jim Apple created IMPALA-7426: --------------------------------- Summary: T-test is an unreliable method for comparing non-normal distributions Key: IMPALA-7426 URL: https://issues.apache.org/jira/browse/IMPALA-7426 Project: IMPALA Issue Type: Bug Components: Infrastructure Reporter: Jim Apple Assignee: Jim Apple
https://en.wikipedia.org/wiki/Student%27s_t-test is for normally distributed variables, and many Impala benchmarks won't be formally distributed. In particular, none of them will have negative run times. We should consider https://en.wikipedia.org/wiki/Nonparametric_statistics like https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test#Comparison_to_Student's_t-test in report_benchmark_results.py. -- This message was sent by Atlassian JIRA (v7.6.3#76005)