Dear all, I'm sure you have all noticed that the Spark tests have been fairly unstable recently. I wanted to share a tool that I use to track which tests have been failing most often in order to prioritize fixing these flaky tests.
Here is an output of the tool. This spreadsheet reports the top 10 failed tests this week (ending yesterday 5/5): https://docs.google.com/spreadsheets/d/1Iv_UDaTFGTMad1sOQ_s4ddWr6KD3PuFIHmTSzL7LSb4 It is produced by a small project: https://github.com/andrewor14/spark-test-failures I have been filing JIRAs on flaky tests based on this tool. Hopefully we can collectively stabilize the build a little more as we near the release for Spark 1.4. -Andrew
