Hey Sam, Congrats on the release! You've made some good calls in there, and the connection-profile integration is clean. I'll put that on my task queue to have adbc_scanner support them. Thanks for the mention.
I'll be honest though, the email's framing annoys me and my mailing list reading robots a little. So I'm going to expound with their help for future human and robot readers of the mailing list. Calling adbc_scanner the "AI-developed community extension" next to your "hand-coded the core pieces," plus the bit about AI "absorbing and repackaging work," paints adbc_scanner as the cut-corners version which it isn't. And don't think that's fair to claim. Whether I used AI tooling or typed every line by hand says nothing about whether the code is good. What says that is whether it works and whether people rely on it. Some facts about adbc_scanner (https://query.farm/duckdb_extension_adbc_scanner) are: - It has been loaded into DuckDB sessions 59,000+ times in the last 30 days, 1,000+ in the last day - It is in production and commercial use - It is tested end-to-end in CI against seven drivers: SQLite, Postgres, MySQL, Flight SQL, DataFusion, Trino, and SQL Server I use AI tooling on purpose and I'm not quiet or shy about it. It is what makes development economically sensible in 2026. Judge the results, thats what the market at large does. On capabilities "appearing in community work," the dates speak for themselves. The first commit in the adbc_scanner repo is October 6, 2025, about two months before yours. Features landed at different times on both sides, as they do, and there's plenty in adbc_scanner the the adbc extension still doesn't do: - Filter and projection pushdown with per-driver SQL dialects - Real transactions with commit and rollback - Column stats handed to the DuckDB optimizer - The seven-driver CI above But honestly, I'm not sure what there is to argue about. Both projects are open source. The whole point is that anyone can read, learn from, and build on this work, in either direction. That's a good thing, not something to police. On where Query Farm (https://query.farm) comes down: my job is to do the best thing for my customers and for the wider DuckDB ecosystem, and those usually point the same way. Keeping adbc_scanner free, open, fast, and well-supported does both. adbc_scanner's license is MIT, it's in production, and Query Farm stands behind it, so it isn't going anywhere. It's also freely distributed and I don't track who's running it, so there's no list of users to ask to migrate even if I wanted one, and I don't. On the idea of supporting a single official ADBC extension specifically, Query Farm and I will pass, for a few reasons: - Speed and stability are what Query Farm's customers pay for. A bug report or a new driver need becomes a shipped, tested fix in days, on a release cadence Query Farm controls, not one gated by a foundation release cycle or another third-party with good intentions. - ASF process is deliberately slow and heavy. That's right for a neutral standard and wrong for a vendor that has to keep pace with its customers. - The licenses don't line up. MIT into an Apache 2.0 ASF repo isn't a clean swap. So Query Farm will keep shipping adbc_scanner in the open. Let both exist and let people use whichever serves them better. That kind of healthy overlap is good for the ecosystem, and I'm always happy to compare technical notes if something specific comes up. Best, Rusty -- Rusty Conover Founder Query.Farm - https://query.farm On Thu, Jun 25, 2026, at 2:35 PM, Sam Arch wrote: > Hi everyone, > > I'm a PhD student at Carnegie Mellon working on database research, > co-advised by Andy Pavlo and Jignesh Patel. I've worked on DuckDB in the > past through my research (https://github.com/duckdb/duckdb/pull/7528), and > I'm currently doing an internship at Columnar. > > My main internship project has been developing a DuckDB extension for ADBC. > The extension lets DuckDB users connect to Snowflake, Databricks, BigQuery, > PostgreSQL, MySQL, and any other system with an ADBC driver. > > The extension supports querying ADBC databases directly through a > `read_adbc` table function. It also supports using `ATTACH` to connect to > an ADBC database and then running `SELECT`, `INSERT`, `COPY`, and CTAS > statements as if the database were local to DuckDB. > > The repo is now public here: > https://github.com/columnar-tech/duckdb-adbc-client/ > > For those following recent work at the intersection of DuckDB and ADBC, you > may have seen that community member Rusty Conover previously published an > AI-developed `adbc_scanner` community extension for DuckDB. We took a > different approach with this extension, choosing to hand-code the core > pieces as part of the academic goals of my internship. The extension also > integrates with ADBC connection profiles, aims for broad database > compatibility, and includes automatic connection pooling, automatic > metadata caching, and memory-efficient `INSERT` and CTAS support through > streaming bulk ingest operations. > > Since we made the repo public, we have also seen some of these ideas and > capabilities begin to appear in related community work. That is allowed > under the Apache 2.0 license, and we want to be clear that we welcome > experimentation and reuse. At the same time, the speed with which > AI-assisted development can absorb and repackage work makes attribution, > coordination, and shared governance especially important. Columnar is also > a financial supporter of DuckLabs, and we care about keeping the > DuckDB/ADBC ecosystem collaborative and healthy. > > With that in mind, although we initially developed this extension under the > Columnar GitHub organization for convenience, we are interested in donating > it to the Arrow project and moving it to an ASF repo. There is some work to > do to assess the feasibility and details, including how DuckDB release > cycles would interact with ADBC release cycles. But we think it would be > valuable to have an official ADBC client extension for DuckDB maintained > under ASF governance, much like the official ADBC client libraries for > various languages. Our hope is that this could provide a neutral place for > third-party contributors, including Rusty and others in the DuckDB and ADBC > communities, to collaborate rather than maintaining separate DuckDB/ADBC > extensions with overlapping goals. > > I’d appreciate feedback from the Arrow community on whether this seems like > a good direction and what the right next steps would be. > > In the meantime, please take a look at the repo, try the extension using > the instructions in the README, and open issues for any bugs, compatibility > problems, or design feedback: > https://github.com/columnar-tech/duckdb-adbc-client/ > > - Sam -- Rusty Conover Query.Farm Founder https://query.farm
