Also +1 for java-based testing framework; bash just disrupts development flow too much. +1 to use docker as much as possible, ideally with testcontainers [1].
[1] https://www.testcontainers.org/ On Wed, Nov 18, 2020 at 10:23 AM Jark Wu <imj...@gmail.com> wrote: > +1 to use the Java-based testing framework and +1 for using docker images > in the future. > IIUC, the Java-based testing framework refers to the > `flink-end-to-end-tests-common` module. > The java-based framework helped us a lot when debugging the unstable e2e > tests. > > Best, > Jark > > On Wed, 18 Nov 2020 at 14:42, Yang Wang <danrtsey...@gmail.com> wrote: > > > Thanks for starting this discussion. > > > > In general, I agree with you that a java-based testing framework is > better > > than the bash-based. It will > > help a lot for the commons and utilities. > > > > Since I am trying to add a new bash-based Kubernetes HA test, I have some > > quick questions. > > * I am not sure where the java-based framework is. Do you mean > > "flink-jepsen" module or sth else? > > * Maybe it will be harder to run a cli command(e.g. flink run / > > run-application) to submit a Flink job in the java-based framework. > > * It will be harder to inject some operations. For example, kill the > > JobManager in Kubernetes. Currently, I > > am trying to use "kubectl exec" to do this. > > > > > > Best, > > Yang > > > > Robert Metzger <rmetz...@apache.org> 于2020年11月17日周二 下午11:36写道： > > > > > Hi all, > > > > > > Since we are currently testing the 1.12 release, and potentially adding > > > more automated e2e tests, I would like to bring up our end to end tests > > for > > > discussion. > > > > > > Some time ago, we introduced a Java-based testing framework, with the > > idea > > > of replacing the current bash-based end to end tests. > > > Since the introduction of the java-based framework, more bash tests > were > > > actually added, making a future migration even harder. > > > > > > *For that reason, I would like to propose that we are stopping to add > any > > > new bash end to end tests to Flink. All new end to end tests must be > > > written in Java and rely on the existing testing framework.* > > > > > > For the 1.13 release, I'm trying to find some time to revisit potential > > > improvements for the existing java e2e framework (such as using Docker > > > images everywhere), as well as a migration plan for the existing bash > > > tests. We have a large number of bash e2e tests that are just > > parameterized > > > differently. If we would start migrating them to Java, we could move a > > > larger proportion of tests over to the new Java framework, and tackle > the > > > more involved bash tests later (kerberized yarn, kubernetes, ...). > > > > > > Let me know what you think! > > > > > > Best, > > > Robert > > > > > > > > > PS: If you are wondering why I'm bringing this up now: I'm spending > > quite a > > > lot of time trying to figure out really hard to debug issues with our > > bash > > > testing infra. > > > Also, it is very difficult to introduce something generic for all tests > > > (such as a test-timeout, using docker as the preferred deployment > method > > > etc.) since the tests often don't share common tooling. > > > Speaking about tooling: there are a lot of utilities everywhere, > > sometimes > > > duplicated, with different features / stability etc. > > > I believe bash is not the right tool for a project this size (in terms > of > > > developers and lines of code) > > > > > > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
