A strong +1 to getting to a single CI system. CircleCI definitely has some niceties and I understand why it's currently used, but right now we get 2 CI systems for twice the price. +1 on the proposed subsets.
Derek On Mon, Jul 10, 2023 at 9:37 AM Josh McKenzie <jmcken...@apache.org> wrote: > I'm personally not thinking about CircleCI at all; I'm envisioning a world > where all of us have 1 CI *software* system (i.e. reproducible on any > env) that we use for pre-commit validation, and then post-commit happens on > reference ASF hardware. > > So: > 1: Pre-commit subset of tests (suites + matrices + env) runs. On green, > merge. > 2: Post-commit tests (all suites, matrices, env) runs. If failure, link > back to the JIRA where the commit took place > > Circle would need to remain in lockstep with the requirements for point 1 > here. > > On Mon, Jul 10, 2023, at 1:04 AM, Berenguer Blasi wrote: > > +1 to Josh which is exactly my line of thought as well. But that is only > valid if we have a solid Jenkins that will eventually run all test configs. > So I think I lost track a bit here. Are you proposing: > > 1- CircleCI: Run pre-commit a single (the most common/meaningful, TBD) > config of tests > > 2- Jenkins: Runs post-commit _all_ test configs and emails/notifies you in > case of problems? > > Or sthg different like having 1 also in Jenkins? > On 7/7/23 17:55, Andrés de la Peña wrote: > > I think 500 runs combining all configs could be reasonable, since it's > unlikely to have config-specific flaky tests. As in five configs with 100 > repetitions each. > > On Fri, 7 Jul 2023 at 16:14, Josh McKenzie <jmcken...@apache.org> wrote: > > Maybe. Kind of depends on how long we write our tests to run doesn't it? :) > > But point taken. Any non-trivial test would start to be something of a > beast under this approach. > > On Fri, Jul 7, 2023, at 11:12 AM, Brandon Williams wrote: > > On Fri, Jul 7, 2023 at 10:09 AM Josh McKenzie <jmcken...@apache.org> > wrote: > > 3. Multiplexed tests (changed, added) run against all JDK's and a > broader range of configs (no-vnode, vnode default, compression, etc) > > I think this is going to be too heavy...we're taking 500 iterations > and multiplying that by like 4 or 5? > > > > -- +---------------------------------------------------------------+ | Derek Chen-Becker | | GPG Key available at https://keybase.io/dchenbecker and | | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org | | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC | +---------------------------------------------------------------+
