It's cool to see an ADBC driver for Spark! Just to jump on this, we've also put out an ADBC driver for Spark (with a few orgs contributing), abstracting over Spark Connect/HiveServer2 Thrift/Apache Livy and testing against Spark 3.5/4 and Amazon EMR: https://github.com/adbc-drivers/spark
Maybe there's interest in working together on a canonical driver? [If the conversation turns to upstreaming, I think that would also be interesting!] -David On Mon, Jun 15, 2026, at 14:44, Hyukjin Kwon wrote: > I had some of my local work, and I decided to just release them as my > personal repositories, and thought it is great to share to see the interest > from the community, and even later we could hopefully discuss whether to > bring this into Apache repositories or not. > > Nothing is intended here. Just thought it's great to share my personal > projects. > > *1. Arrow ADBC with Spark Connect* > ADBC <https://arrow.apache.org/adbc/current/index.html> implementation which > will virtually support most of languages with Spark Connect (with Arrow ADBC) > > Source: https://github.com/HyukjinKwon/adbc-driver-spark > Docs: https://hyukjinkwon.github.io/adbc-driver-spark/ > Release: > - Go: https://pkg.go.dev/github.com/HyukjinKwon/adbc-driver-spark > - Python: https://pypi.org/project/adbc-driver-spark/ > - Other languages: https://github.com/HyukjinKwon/adbc-driver-spark/releases > > *2. Scala3 with Spark Connect* > > Scala 3 has been a long standing request. This opens up Scala 3 support. > > Source: https://github.com/HyukjinKwon/spark-connect-scala3 > Docs: https://hyukjinkwon.github.io/spark-connect-scala3/ > Release (Maven Central): > https://central.sonatype.com/artifact/com.github.hyukjinkwon/spark-connect-scala3-client_3 > > *3. Ruby with Spark Connect* > > Opens Ruby with Spark Connect. > > Source: https://github.com/HyukjinKwon/spark-connect-ruby > Docs: https://hyukjinkwon.github.io/spark-connect-ruby/ > Release (Gem): https://rubygems.org/gems/spark-connect > > *4. Wasm with Python Spark Connect* > > Runs a native Python Spark Connect within a browser. > > Source: https://github.com/HyukjinKwon/pyspark-client-wasm > Docs:https://hyukjinkwon.github.io/pyspark-client-wasm/ > Release (PyPI): https://pypi.org/project/pyspark-connect-web/
