I'd be okay skipping the HiveCompatibilitySuite for core-only changes. They do often catch bugs in changes to catalyst or sql though. Same for HashJoinCompatibilitySuite/VersionsSuite.
HiveSparkSubmitSuite/CliSuite should probably stay, as they do test things like addJar that have been broken by core in the past. On Tue, Aug 25, 2015 at 1:40 PM, Patrick Wendell <[email protected]> wrote: > There is already code in place that restricts which tests run > depending on which code is modified. However, changes inside of > Spark's core currently require running all dependent tests. If you > have some ideas about how to improve that heuristic, it would be > great. > > - Patrick > > On Tue, Aug 25, 2015 at 1:33 PM, Marcelo Vanzin <[email protected]> > wrote: > > Hello y'all, > > > > So I've been getting kinda annoyed with how many PR tests have been > > timing out. I took one of the logs from one of my PRs and started to > > do some crunching on the data from the output, and here's a list of > > the 5 slowest suites: > > > > 307.14s HiveSparkSubmitSuite > > 382.641s VersionsSuite > > 398s CliSuite > > 410.52s HashJoinCompatibilitySuite > > 2508.61s HiveCompatibilitySuite > > > > Looking at those, I'm not surprised at all that we see so many > > timeouts. Is there any ongoing effort to trim down those tests > > (especially HiveCompatibilitySuite) or somehow restrict when they're > > run? > > > > Almost 1 hour to run a single test suite that affects a rather > > isolated part of the code base looks a little excessive to me. > > > > -- > > Marcelo > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
