Hi all, I'd like to open a discussion on publishing an xtable-java-runtime - a lightweight, Spark-free runtime for running XTable metadata sync - now that Delta Kernel support has landed on main.
Motivation Until now, any XTable sync involving Delta effectively required a SparkSession, since the Delta source/target went through delta-spark's DeltaLog APIs. The Hudi and Iceberg paths are already pure Java, so Delta was the one format anchoring us to Spark. Today the runtime options we ship (the xtable-utilities CLI and xtable-spark-runtime, #843) both carry Spark. With Delta Kernel PR's now merged, Delta can be read and written without Spark. That removes the last hard Spark dependency in the sync path, which means we can publish a genuinely Spark-free, pure-Java runtime jar covering Hudi, Iceberg, and Delta (via Kernel). Why it matters - non-Spark engines A Java-only runtime makes XTable embeddable anywhere on the JVM without dragging in Spark: - Flink jobs syncing metadata in-line - Trino / Presto plugins and connectors - Plain Java services, functions, or orchestration steps This drastically shrinks the footprint and the dependency-conflict surface versus a Spark bundle, and opens XTable to the large set of users who aren't on Spark. Context We discussed this during the community sync this morning and agreed it's worth pursuing - the notes capture that with Delta Kernel merged, an xtable-java-runtime jar can be published with Delta Kernel as source/target. Full notes: https://docs.google.com/document/d/1mSthtQBVDDzi9bLn9sWDsPaJLJHCDoK_MxDsSphhlos/edit?usp=sharing Questions for the list - Do we agree an xtable-java-runtime is worth publishing in the next release? - Any concerns about Delta Kernel feature parity for a Kernel-only runtime (e.g. deletion vectors, issue #713)? Looking forward to your thoughts. Thanks, Vinish
