Thesharing opened a new pull request #7:
URL: https://github.com/apache/flink-benchmarks/pull/7
This pull request introduces several benchmarks to evaluate the performance
of the scheduler. For now it includes four procedures:
- Topology building
- Scheduling
- Task deploying
- Calculating the tasks to restart when failover
All the topologies of the benchmarks consist of two vertices: one source
vertex and one sink vertex. They are connected with all-to-all edges. The
parallelisms of all vertices are 8000. There are two scenarios:
- For the streaming job, the result partition type is `PIPELINED`, and the
execution mode is `STREAMING`.
- For the batch job, the result partition type is `BLOCKING` , and the
execution mode is `BATCH`.
It's worth noting that, for batch jobs, the sink tasks are scheduled and
deployed if and only if all the source tasks finish. So the scheduling and
deployment of source tasks and sink tasks have separated benchmarks.
Since the topology building, scheduling and deploying only executes once
during the job, we use the cold start time to evaluate these procedures. For
calculation of failover tasks, we first warm up the JVM and then evaluate the
procedure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
