+1 to use the Java-based testing framework and +1 for using docker images in the future. IIUC, the Java-based testing framework refers to the `flink-end-to-end-tests-common` module. The java-based framework helped us a lot when debugging the unstable e2e tests.
Best, Jark On Wed, 18 Nov 2020 at 14:42, Yang Wang <[email protected]> wrote: > Thanks for starting this discussion. > > In general, I agree with you that a java-based testing framework is better > than the bash-based. It will > help a lot for the commons and utilities. > > Since I am trying to add a new bash-based Kubernetes HA test, I have some > quick questions. > * I am not sure where the java-based framework is. Do you mean > "flink-jepsen" module or sth else? > * Maybe it will be harder to run a cli command(e.g. flink run / > run-application) to submit a Flink job in the java-based framework. > * It will be harder to inject some operations. For example, kill the > JobManager in Kubernetes. Currently, I > am trying to use "kubectl exec" to do this. > > > Best, > Yang > > Robert Metzger <[email protected]> 于2020年11月17日周二 下午11:36写道： > > > Hi all, > > > > Since we are currently testing the 1.12 release, and potentially adding > > more automated e2e tests, I would like to bring up our end to end tests > for > > discussion. > > > > Some time ago, we introduced a Java-based testing framework, with the > idea > > of replacing the current bash-based end to end tests. > > Since the introduction of the java-based framework, more bash tests were > > actually added, making a future migration even harder. > > > > *For that reason, I would like to propose that we are stopping to add any > > new bash end to end tests to Flink. All new end to end tests must be > > written in Java and rely on the existing testing framework.* > > > > For the 1.13 release, I'm trying to find some time to revisit potential > > improvements for the existing java e2e framework (such as using Docker > > images everywhere), as well as a migration plan for the existing bash > > tests. We have a large number of bash e2e tests that are just > parameterized > > differently. If we would start migrating them to Java, we could move a > > larger proportion of tests over to the new Java framework, and tackle the > > more involved bash tests later (kerberized yarn, kubernetes, ...). > > > > Let me know what you think! > > > > Best, > > Robert > > > > > > PS: If you are wondering why I'm bringing this up now: I'm spending > quite a > > lot of time trying to figure out really hard to debug issues with our > bash > > testing infra. > > Also, it is very difficult to introduce something generic for all tests > > (such as a test-timeout, using docker as the preferred deployment method > > etc.) since the tests often don't share common tooling. > > Speaking about tooling: there are a lot of utilities everywhere, > sometimes > > duplicated, with different features / stability etc. > > I believe bash is not the right tool for a project this size (in terms of > > developers and lines of code) > > >
