DanielLeens opened a new issue, #11580: URL: https://github.com/apache/seatunnel/issues/11580
### Search Before Asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues) and found no similar feature requirement. ### Description SeaTunnel already has useful building blocks for correctness and integration validation, such as `FakeSource`, `Assert`, connector E2E tests, and engine E2E containers. However, there is no standard benchmark harness that can run reproducible performance workloads, collect metrics, and compare results across versions or engines. A dedicated benchmark framework would help contributors and users evaluate throughput, latency, checkpoint behavior, resource usage, and regression risk in a consistent way. ### Proposed Scope This issue proposes adding a benchmark harness for SeaTunnel with at least the following capabilities: 1. Define reusable benchmark workloads, for example `FakeSource -> LocalFile`, `FakeSource -> JDBC`, `JDBC -> JDBC`, and streaming workloads. 2. Support configurable job mode, engine, parallelism, row count, split count, payload size, checkpoint interval, and sink settings. 3. Collect benchmark evidence such as total records, elapsed time, throughput, failure status, and, where available, checkpoint and resource metrics. 4. Produce machine-readable result files, such as JSON or CSV, so benchmark results can be compared across commits or releases. 5. Reuse existing SeaTunnel test/container infrastructure where practical, instead of introducing a separate runtime model. ### Expected Deliverables - A new benchmark module, script set, or clearly documented benchmark entry point. - At least one runnable baseline benchmark workload for SeaTunnel Zeta. - Documentation explaining how to run the benchmark and how to interpret the output. - A result format that can be used for future regression comparison. ### Notes `FakeSource` is a good starting point for generating controlled data, but it may need benchmark-specific tuning or a separate performance source mode to avoid test-oriented throttling affecting throughput measurements. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
