Hi Pranav, I left some comments on your design. Your doc discusses a bunch of details about infrastructure such as testing frameworks, automation, and performance databases, but doesn't describe how it will fit in with our existing infrastructure (Load Tests, Nexmark, Jenkins, InfluxDB, Grafina). I would suspect we actually have most of the infrastructure already built?
What I didn't see (and expected to see) was details on how the tests would actually interact with IOs. Will there be a generic Schema IO test harness or do you plan to write one for each IO? Will you be comparing different data types (data stored as byte[] vs more complex structures)? What about different IO specific optimization (data sharding, pushdown)? Andrew On Mon, Sep 26, 2022 at 9:07 AM Pranav Bhandari <bhandari.prana...@gmail.com> wrote: > > Hello, > > Hope this email finds you well. I have attached a link to a doc which > discusses the design for a performance and cost benchmarking framework to be > used by Beam IOs and Google-provided dataflow templates. > > Please feel free to comment on the doc with any questions, concerns or ideas > you might have. > > Thank you, > Pranav Bhandari > > > https://docs.google.com/document/d/14GatBilwuR4jJGb-ZNpYeuB-KkVmDvEm/edit?usp=sharing&ouid=102139643796739130048&rtpof=true&sd=true
