Thanks Matthias, I have created sub-tasks for PyFlink related e2e tests.
Best, Xingbo > 2020年11月27日 下午10:53，Jark Wu <imj...@gmail.com> 写道： > > Thanks Matthias, > > I have created sub-tasks for the Table SQL related bash-based e2e tests. > > Best, > Jark > > On Fri, 27 Nov 2020 at 21:25, Matthias Pohl <matth...@ververica.com> wrote: > >> Thanks Robert for pushing this. +1 for creating java-based e2e tests. >> >> In the engine team, we decided to work towards the goal of migrating the >> bash-based e2e tests to Java/Docker. We plan to migrate the existing >> bash-based e2e tests located in the Engine team's component space >> step-by-step. I created an umbrella Jira issue [1] to collect and document >> the migration efforts. Feel free to do the same by creating subtasks under >> [1]. >> >> Best, >> Matthias >> >> [1] https://issues.apache.org/jira/browse/FLINK-20392 >> >> On Thu, Nov 19, 2020 at 8:02 AM Yun Tang <myas...@live.com> wrote: >> >>> +1 for java-based E2E test as bash scripts lack of the power to handle >>> more complicated cases. >>> >>> For the docker image improvement, I think we should be more cautious as >>> developers at China might suffer with the network issue, or at least we >>> should ensure or find some guides to speed up the image downloading. >>> >>> Best >>> Yun Tang >>> ________________________________ >>> From: Xingbo Huang <hxbks...@gmail.com> >>> Sent: Thursday, November 19, 2020 12:09 >>> To: dev <dev@flink.apache.org> >>> Subject: Re: [DISCUSS] Stop adding new bash-based e2e tests to Flink >>> >>> Big +1 for java-based e2e tests. Currently PyFlink related tests each >> take >>> ~15minutes in bash e2e tests because we are using a secured YARN cluster >>> which is the only convenient way of starting a YARN cluster in the bash >> e2e >>> tests. I think if we migrate these tests to the java-based testing >>> framework, we will start a Yarn Cluster more conveniently, which will >>> greatly reduce our testing time. >>> >>> Best, >>> Xingbo >>> >>> Rui Li <lirui.fu...@gmail.com> 于2020年11月19日周四 上午10:47写道： >>> >>>> Big +1 to java-based e2e tests. It'll be much easier to write/debug >> these >>>> tests. >>>> >>>> On Wed, Nov 18, 2020 at 9:44 PM Leonard Xu <xbjt...@gmail.com> wrote: >>>> >>>>> +1 to stop using bash scripts, >>>>> and I also have experienced the bash scripts that is really hard to >>>>> maintain and debug, thanks @Robert for the great work again. >>>>> >>>>> I think testcontainers is a nice candidate. >>>>> >>>>> Best, >>>>> Leonard >>>>> >>>>>> 在 2020年11月18日，19:46，Aljoscha Krettek <aljos...@apache.org> 写道： >>>>>> >>>>>> +1 >>>>>> >>>>>> And I want to second Arvid's mention of testcontainers [1]. >>>>>> >>>>>> [1] https://www.testcontainers.org/ >>>>>> >>>>>> On 18.11.20 10:43, Yang Wang wrote: >>>>>>> Thanks till and Jark for sharing the information. >>>>>>> I am also +1 for this proposal and glad to wire the new introduced >>> K8s >>>>> HA >>>>>>> e2e tests to java based framework. >>>>>>> Best, >>>>>>> Yang >>>>>>> Jark Wu <imj...@gmail.com> 于2020年11月18日周三 下午5:23写道： >>>>>>>> +1 to use the Java-based testing framework and +1 for using >> docker >>>>> images >>>>>>>> in the future. >>>>>>>> IIUC, the Java-based testing framework refers to the >>>>>>>> `flink-end-to-end-tests-common` module. >>>>>>>> The java-based framework helped us a lot when debugging the >>> unstable >>>>> e2e >>>>>>>> tests. >>>>>>>> >>>>>>>> Best, >>>>>>>> Jark >>>>>>>> >>>>>>>> On Wed, 18 Nov 2020 at 14:42, Yang Wang <danrtsey...@gmail.com> >>>> wrote: >>>>>>>> >>>>>>>>> Thanks for starting this discussion. >>>>>>>>> >>>>>>>>> In general, I agree with you that a java-based testing framework >>> is >>>>>>>> better >>>>>>>>> than the bash-based. It will >>>>>>>>> help a lot for the commons and utilities. >>>>>>>>> >>>>>>>>> Since I am trying to add a new bash-based Kubernetes HA test, I >>> have >>>>> some >>>>>>>>> quick questions. >>>>>>>>> * I am not sure where the java-based framework is. Do you mean >>>>>>>>> "flink-jepsen" module or sth else? >>>>>>>>> * Maybe it will be harder to run a cli command(e.g. flink run / >>>>>>>>> run-application) to submit a Flink job in the java-based >>> framework. >>>>>>>>> * It will be harder to inject some operations. For example, kill >>> the >>>>>>>>> JobManager in Kubernetes. Currently, I >>>>>>>>> am trying to use "kubectl exec" to do this. >>>>>>>>> >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Yang >>>>>>>>> >>>>>>>>> Robert Metzger <rmetz...@apache.org> 于2020年11月17日周二 下午11:36写道： >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Since we are currently testing the 1.12 release, and >> potentially >>>>> adding >>>>>>>>>> more automated e2e tests, I would like to bring up our end to >> end >>>>> tests >>>>>>>>> for >>>>>>>>>> discussion. >>>>>>>>>> >>>>>>>>>> Some time ago, we introduced a Java-based testing framework, >> with >>>> the >>>>>>>>> idea >>>>>>>>>> of replacing the current bash-based end to end tests. >>>>>>>>>> Since the introduction of the java-based framework, more bash >>> tests >>>>>>>> were >>>>>>>>>> actually added, making a future migration even harder. >>>>>>>>>> >>>>>>>>>> *For that reason, I would like to propose that we are stopping >> to >>>> add >>>>>>>> any >>>>>>>>>> new bash end to end tests to Flink. All new end to end tests >> must >>>> be >>>>>>>>>> written in Java and rely on the existing testing framework.* >>>>>>>>>> >>>>>>>>>> For the 1.13 release, I'm trying to find some time to revisit >>>>> potential >>>>>>>>>> improvements for the existing java e2e framework (such as using >>>>> Docker >>>>>>>>>> images everywhere), as well as a migration plan for the >> existing >>>> bash >>>>>>>>>> tests. We have a large number of bash e2e tests that are just >>>>>>>>> parameterized >>>>>>>>>> differently. If we would start migrating them to Java, we could >>>> move >>>>> a >>>>>>>>>> larger proportion of tests over to the new Java framework, and >>>> tackle >>>>>>>> the >>>>>>>>>> more involved bash tests later (kerberized yarn, kubernetes, >>> ...). >>>>>>>>>> >>>>>>>>>> Let me know what you think! >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Robert >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> PS: If you are wondering why I'm bringing this up now: I'm >>> spending >>>>>>>>> quite a >>>>>>>>>> lot of time trying to figure out really hard to debug issues >> with >>>> our >>>>>>>>> bash >>>>>>>>>> testing infra. >>>>>>>>>> Also, it is very difficult to introduce something generic for >> all >>>>> tests >>>>>>>>>> (such as a test-timeout, using docker as the preferred >> deployment >>>>>>>> method >>>>>>>>>> etc.) since the tests often don't share common tooling. >>>>>>>>>> Speaking about tooling: there are a lot of utilities >> everywhere, >>>>>>>>> sometimes >>>>>>>>>> duplicated, with different features / stability etc. >>>>>>>>>> I believe bash is not the right tool for a project this size >> (in >>>>> terms >>>>>>>> of >>>>>>>>>> developers and lines of code) >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>>> >>>> >>>> -- >>>> Best regards! >>>> Rui Li >>>> >>> >> >> >> -- >> >> Matthias Pohl | Engineer >> >> Follow us @VervericaData Ververica <https://www.ververica.com/> >> >> -- >> >> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >> Conference >> >> Stream Processing | Event Driven | Real Time >> >> -- >> >> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >> >> -- >> Ververica GmbH >> Registered at Amtsgericht Charlottenburg: HRB 158244 B >> Managing Directors: Yip Park Tung Jason, Jinwei (Kevin) Zhang, Karl Anton >> Wehner >>
