Was talking with Eric off-list about a recent test he added.
Over the past two major release lines (1.6 and 1.7), there's been a
significant level of effort put forth by multiple devs to get the
integration tests running on "terrible" hardware. This has been a great
endeavor because our tests have never been more stable and it's even
helped us catch bugs that we would have otherwise assumed as transiently
failing (ACCUMULO-3859 is a great example).
Because we are writing a database, we're always concerned about
performance regressions, both high-level and low-level. I'd like to
propose that we recognize and accept this head-on and try to move these
specifically "high-load" and "performance related" tests to their own
execution phase that we can run specifically on nodes that meet the
necessary preconditions.
Some examples of tests:
DeleteTableDuringSplitIT
DurabilityIT
ManySplitIT
RollWALPerformanceIT
I know we can do some classification of tests via surefire/failsafe
which should roughly meet our goals (typically via an annotation on the
class). Thus, we could add a specific flag to a Maven build that would
include this subset of tests.
What do people think? Do others also think that this is worth pursuing?
- Josh
