Hi Timo,
Thanks for opening this.
I agree that bash is not the best tool for all the reasons that you mention
plus:
1) it is difficult to write re-usable code
2) there are a lot of ways to express the same thing and difficult to build
“best practices” (as it should be
in a community project) resulting in a difficulty also to make it a real
community effort.
Now on the with what to replace bash, I agree that Java is the best option.
Flink is mainly written in Java
and most of its community is already familiar with it. In addition, there are a
lot of utilities already built
that can help reduce the boilerplate code that, for example, check logs.
The only alternative I can think of it Python, but given that Flink is a mainly
Java project.
I would vouch for Java.
Cheers,
Kostas
> On Mar 27, 2018, at 2:00 PM, Timo Walther <[email protected]> wrote:
>
> Hi everyone,
>
> after reviewing a bunch of end-to-end tests, I'm wondering if we should
> really continue implementing everything in bash scripts. Wouldn't it be nicer
> to implement them in Java code that just calls the interfaces of Flink (e.g.
> "./bin/flink run" or REST API)?
>
> Here are some thoughts why I think we should consider that:
>
> - Implement the tests in a more object oriented way: We could have classes
> such as Cluster, LogAnalyzer, WatchdogProcess, and other utility classes that
> can make writing tests easier. This also ensure maintainability in the future
> because all tools for proper Java coding style, comments etc. are already in
> place.
>
> - Exception handling and type safety: We could catch excpetion more easily
> (e.g. if a REST request fails or a process is not there)
>
> - Debuggability: We could run end-to-end tests in the IDE for debugging and
> set breakpoints etc.
>
>
> What do you think?
>
>
> Regards,
>
> Timo
>
