DanielLeens commented on issue #11549: URL: https://github.com/apache/seatunnel/issues/11549#issuecomment-5081317375
Thanks for opening PR #11553 together with the discussion. That helps a lot, because this question is easier to judge with a concrete tree than as an abstract idea. I checked the current repository layout on July 26, 2026. SeaTunnel already has a dedicated `seatunnel-cli/` module, and the proposed benchmark is kept under `seatunnel-cli/benchmark/` as optional developer tooling rather than mixed into runtime paths. With that separation, the direction looks reasonable to me. The main review questions here are less about "can this exist at all" and more about long-term maintenance boundaries: 1. whether the task suite and scoring rules stay deterministic and reviewable 2. whether the benchmark remains optional and isolated from the normal release / runtime path 3. whether credentials, external services, and large result artifacts stay fully out of the repository state 4. whether the project is comfortable owning benchmark-task evolution over time Given that PR #11553 is already open, I think the best next step is to review it on those boundaries instead of keeping this only at the discussion level. If the benchmark stays optional, reproducible from a clean checkout, and clearly documented as non-product tooling, this seems worth evaluating in-tree. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
