Hi everyone, As I can understand, a lot of tests in Nexmark set are performance tests. I suggest to rename(or split) the set to performance tests.
Performance tests are much less reliable compared to post-commit tests and should have different requirements. Additionally, they are much more flaky. Splitting out performance tests to separate set will allow us to treat failures with lower priority and add more tolerance for flakes compared to what we have decided for post-commit tests <https://beam.apache.org/contribute/postcommits-policies/>. This will also be more organic to use different builder from PostcommitJobBuilder <https://github.com/apache/beam/tree/master/.test-infra/jenkins>, since we will want different requirements for perf tests. I do not believe we have a problem with this in current state, but I expect this to become an issue in the future as amount of perf tests grows. Regards, --Mikhail Have feedback <http://go/migryz-feedback>?
