aglinxinyuan commented on PR #7323: URL: https://github.com/apache/texera/pull/7323#issuecomment-5211735685
On the benchmark flag — I don't think there's a real regression here, and the raw CSV shows why. This PR adds exactly one file, `ClusterListenerSpec.scala`, under `src/test`. It ships no production bytecode at all, so there's no mechanism by which it could move engine throughput. That's the prior; the numbers agree with it: - **Throughput is within the bot's own noise band on all three configs** — -4.4%, -1.4%, -0.4% against the same-runner main baseline. The bot's summary column reports the *max* delta across metrics, so a latency tail is what promoted these rows to 🔴, not throughput. - **The `bs=10` tail is a single sample.** That config runs 20 batches, so p95 and p99 are the same observation — and the CSV confirms it, both exactly `40412.32`. A one-sample tail moving 15% on 200 total tuples isn't a signal. - **The 7d column is contaminated for this run.** At `bs=10` the PR reads 371 tuples/sec and the same-runner main baseline reads 388, against a 7d average of 767 — both roughly half. The runner was slow for the whole session, which inflates every 7d delta (`+160.1%` on that p95) without saying anything about the diff. So: same-runner throughput flat, the red cells are a single-sample tail on the smallest config, and the 7d deltas are a runner artifact shared by the baseline. Happy to re-run the benchmark job if a committer would rather see a second sample before merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
