Thanks, Arvid. The guide was helpful in how to start working with Flink. I'm currently exploring SQL/Table API.
Will surely come back for queries on it. On Thu, Aug 13, 2020 at 1:25 PM Arvid Heise <ar...@ververica.com> wrote: > Hi, > > performance testing is quite vague. Usually you start by writing a small > first version of your pipeline and check how the well computation scales on > your data. Flink's web UI [1] already helps quite well for the first time. > Usually you'd also add some metric system and look for advanced metrics in > there [2]. > > Now, you might be satisfied with your current solution and just start to > extend it. Or you feel that it's not fast enough, scaling well enough. Then > you can tweak your pipeline and perform smaller performance tests on your > user code. Doing performance tests on the whole pipeline [3] would probably > be hard if you just started as you also need to understand internals of > Flink. > > I also recommend to use Table API / SQL instead of DataStream if your > application can be expressed well in relational operations. Table API > already applies a wide range of optimizations that are much harder to > implement manually in DataStream API. Table API will also bring your > noticeable performance improvements over time when you update to a newer > Flink version. > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.11/monitoring/back_pressure.html > [2] > https://ci.apache.org/projects/flink/flink-docs-release-1.11/monitoring/metrics.html > [3] > https://ci.apache.org/projects/flink/flink-docs-release-1.11/monitoring/application_profiling.html > > On Mon, Aug 10, 2020 at 1:06 PM narasimha <swamy.haj...@gmail.com> wrote: > >> Hi, >> >> I'm new to the streaming world, checking on Performance testing tools. >> Are there any recommended Performance testing tools for Flink? >> >> -- >> A.Narasimha Swamy >> > > > -- > > Arvid Heise | Senior Java Developer > > <https://www.ververica.com/> > > Follow us @VervericaData > > -- > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > Conference > > Stream Processing | Event Driven | Real Time > > -- > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > -- > Ververica GmbH > Registered at Amtsgericht Charlottenburg: HRB 158244 B > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji > (Toni) Cheng > -- A.Narasimha Swamy
