It's night here so I appreciate the new darker palette. As for result readability I prefer the old interactive graph because it has classified points, trendline and bands.
old: https://conbench.arrow-dev.org/benchmark-results/06a3edbc2d1f7dc48000633dc612769e/ new:: https://conbench-v2.arrow-dev.org/benchmarks/history/06a3edbc2d1f7dc48000633dc612769e Any new feature that you'd point out for improved ergonomics? Rok On Sat, Jun 27, 2026 at 2:22 AM Wes McKinney <[email protected]> wrote: > TL;DR take a look at https://conbench-v2.arrow-dev.org. > > This was written by Codex to summarize the state of things: > > Hi all, > > I’ve pushed the Conbench v2 work for review in the following places: > > Conbench v2 app/server/docs: > https://github.com/conbench/conbench/tree/experimental-v2 > > Buildkite, Terraform, and CI adapter work: > https://github.com/wesm/arrow-benchmarks-ci/tree/v2-conbench-ci-report > > Python benchmark payload work: > https://github.com/wesm/benchmarks/tree/v2-conbench-submit > > R benchmark payload work: > https://github.com/wesm/arrowbench/tree/v2-conbench-payloads > > There is also a live read-only evaluator running here: > > https://conbench-v2.arrow-dev.org > > Temporary docs are published here: > > https://wesm.github.io/conbench-tmp/ > > The goal of this work is to provide a concrete migration path for > Conbench v2 while preserving the existing production database schema. > The evaluator is intended for review and experimentation only; it > should not modify the production database. > > Please take a look at the app, docs, and workflow changes. The main > things to review are whether the new UI is useful for Arrow > maintainers, whether the benchmark reporting path is understandable, > and whether the migration approach looks practical for the existing > Arrow benchmarking workflows. > > On Tue, Jun 23, 2026 at 5:34 PM Wes McKinney <[email protected]> wrote: > > > > If you can give me access to the servers and cloud resources in question > (ie the machines that currently run the benchmarks), I can implement the > necessary code adaptations and test things working end to end, and stand up > a parallel deployment of the application against RDS to enable better > evaluation of the UI ergonomics. Perhaps we can coordinate offline and come > back to the community with a report once the implementation is closer to > being able to “throw the switch”. > > > > On Tue, Jun 23, 2026 at 17:02 Rok Mihevc <[email protected]> wrote: > >> > >> Current architecture is not optimal or modern, but its maintenance cost > >> is well known and currently manageable. But I'm happy with changes that > >> move us to a more maintainable state and am willing to assist with the > >> transition. > >> > >> On a personal note, I'm hesitant to sign up for maintaining a deployment > >> of software that's not built yet. > >> So I'm just curious about who "owns" Arrow's conbench deployment > >> if I cannot commit to it. > >> > >> Rok > >> > >> On Tue, Jun 23, 2026 at 4:40 PM Wes McKinney <[email protected]> > wrote: > >> > >> > I think the objective is to create a more modern foundation while also > >> > improving performance. I think this looks like: > >> > > >> > * faster, easier to develop and deploy backup (use Go or Rust to > create > >> > static binaries: Go is good for backend services like this) > >> > * use modern web technologies versus generating pages with Jinja > templates > >> > * Use things like DuckDB and Parquet to scale result storage while > >> > improving performance > >> > * Add many more UI features to make the results most useful to > maintainers > >> > > >> > Like I said, I’m happy to fulfill feature requests and contribute > >> > development with agents, if it isn’t interesting I’m also fine to go > my own > >> > way. > >> > > >> > I think Buildkite is fine for job scheduling and management for now, I > >> > don’t think this system currently wants to own a task queue / durable > >> > execution state for workers, though it could grow this capability in > the > >> > future (workers would have to poll the server for jobs to take). > >> > > >> > On Tue, Jun 23, 2026 at 09:07 Antoine Pitrou <[email protected]> > wrote: > >> > > >> > > > >> > > Hi, > >> > > > >> > > I think the main question here is: what are we trying to do? > >> > > > >> > > Currently, the main operational issue with conbench is the slowness > of > >> > > the web UI, due to the large database size and that it's not > normalized > >> > > (some queries take much longer than they should). > >> > > > >> > > I can't speak about the maintenance / reliability aspects, though. > >> > > > >> > > Regards > >> > > > >> > > Antoine. > >> > > > >> > > > >> > > Le 04/06/2026 à 16:19, Wes McKinney a écrit : > >> > > > hi all, > >> > > > > >> > > > I saw that conbench.ursa.dev has been down and I had a need to > set up > >> > > > some continuous project benchmarks, and was interested in doing > >> > > > development on Conbench (well, having my agents do development on > >> > > > Conbench), and was interested in the following: > >> > > > > >> > > > 1) is there interest in migrating the historical Arrow conbench > data > >> > > > to a new server, has that been preserved somewhere? I'll probably > >> > > > rewrite the conbench backend in Go and give it a client CLI for > >> > > > submitting new data or querying old data. > >> > > > > >> > > > 2) are there other users of conbench (conbench/conbench) that > anyone > >> > > > is aware of? I'd be done doing in-situ development in that > repository > >> > > > or setting up a conbench-v2 project. > >> > > > > >> > > > No particular urgency but if anyone has opinions let me know! > >> > > > > >> > > > thanks, > >> > > > Wes > >> > > > >> > > > >> > >
