GitHub user hubcio created a discussion: Benchmarks in non-Rust SDKs
Currently, iggy doesn't have benchmark for other SDKs. We only have `iggy-bench` Rust binary, which has tons of options. It would be great if we could bench other SDKs performance. We should decide between two approaches: - (A) separate native benchmark binaries per SDK (Go bench, Java bench, etc.), - (B) extend iggy-bench (Rust binary) to call other SDK functions via FFI. Trade-offs around consistency, performance measurement accuracy, and maintenance burden. Pros/cons: - Option A (separate binaries): idiomatic per language, no FFI overhead in measurements, but duplicated logic and harder to keep methodology consistent - Option B (iggy-bench calls SDKs via FFI): single tool, unified reporting and methodology, but FFI layer adds complexity and may skew benchmarks Either way, we want to share a common `report.json` schema so results are comparable across SDKs - this way we could upload these results to https://benchmarks.iggy.apache.org. Questions: - Does FFI overhead invalidate benchmark results, or is it negligible for I/O-bound workloads? - How much bench logic can realistically be shared vs needs per-SDK customization? Opening discussion. GitHub link: https://github.com/apache/iggy/discussions/2731 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
