Thanks Tim for driving this release. On Fri, Oct 2, 2026 at 15:58 Vinish Reddy <[email protected]> wrote:
> Hi all, > > Here is a status update on the 0.5.0-incubating release, following up on > the scope and timeline thread from August [1]. Tim is the release manager > for 0.5.0. > > The full list of tracked work is under the 0.5.0 label. > https://github.com/apache/incubator-xtable/issues?q=is%3Aopen+label%3A0.5.0 > > MOMENTUM SINCE 0.4.0 > > Since August 1, 31 PRs have merged on main from 9 contributors. Some > highlights: > > - The xtable-utilities bundled jar shrank from 1.1 GB to 423 MB (#897), and > the duplicate AWS SDK v2 modules were removed (#923). > - Delta nested field ids now carry into the converted schema (#912), and > Delta log listing handles absolute paths (#932). > - Incremental sync is skipped when a Delta source has no new commits > (#930), and the Delta snapshot is reused across an incremental backlog > (#861). > - Hudi partitions are batch-loaded before the parallel file listing (#950). > - Glue Catalog Federation works with Delta tables (#940). > - The parquet source now populates record_count (#929), and its limitations > are documented (#937). > - The build moved to ASF parent POM 39 and compiles on JDK 21+ (#881, > #900), with the Maven build cache persisted in CI (#882). > - The 0.4.0 release notes, blog post and Spark runtime quickstart are live > (#913, #915). > > Thanks to Sylwester Lachiewicz, Ranga Reddy, Arvin, Abdul Megoon, Akash > Reddy Jammula, Tim Brown, Vamsi Karnika and AadhiKat for their > contributions, and to everyone who reviewed and tested. > > STATUS BY THEME > > 1. Hudi table version 9 (Hudi 1.x) support (#834) > - #835 adds v9 to the Hudi target and source. It now builds on the released > Hudi 1.2.1, CI is green, and it is ready for review. > > 2. Iceberg pluggable table format managed by Hudi (#722), led by Ethan > - #894 implements the Iceberg versions of TableFormat and > HoodieTableMetadata. CI is green and review is in progress. > - #954 (Iceberg V3 deletion vectors and table services for MOR tables) and > #955 (a Spark plugin that routes Iceberg writes through Hudi) are early > drafts that build on #894. > > 3. Cross-format indexing (#887) > - #933 is RFC-4 for index support. > - #935 adds a Hudi-backed secondary index for Iceberg tables. It has been > validated on large tables on S3 and is close to ready. > - The xtable-spark-extensions work (Spark SQL to build, drop and check the > staleness of indexes) is pushed and stacks on #935. A PR will follow. > > 4. xtable-spark-runtime for Hudi 1.x (#836) > - The runtime shipped for 0.4.x on branch-0.4 (#843). > - #838 (thin drop-in bundle and resilient target discovery) and #839 > (XTableSyncListener for in-job sync) bring it to main. > - #931 adds Parquet source partitioning to the spark runtime (#927). > - #908 from Sylwester moves the build to Spark 3.5.9 and Delta 3.3.3, as > the first step toward the Spark 4 upgrade (#902). Reviews are welcome there > too. > > 5. Delta Kernel as the default Delta path (#886, #713) > - #903 adds Delta Kernel source and target coverage to > ITConversionController and is approved. > - #947 (local paths, #943) and #942 (commit tag mapping, #819) fix Kernel > gaps found along the way. > > 6. Correctness and robustness fixes. > - #951 falls back to parquet-footer stats when the Hudi metadata table > returns degenerate values. It is approved and ready to merge. > - #621 adds a custom Iceberg metadata cleaner. > - #956 resolves the parquet source file system from the base path, and #957 > keeps absolute paths for data files outside the table base path in Delta > targets. > - #938 adds fail-safe handling for Delta deletion vectors (#657). > - #952 bundles Iceberg's GlueCatalog and S3FileIO in xtable-utilities > (#944). > - #814 filters metadata directories out of Hudi partitions (#813). > - Open bugs without a PR yet are #958, #959, #941, #936, #901 and #736. > Help on any of these is welcome. > > 7. License and packaging (#701, #717, #865) > - #876 keeps exactly one LICENSE and NOTICE in every published jar, and > #895 makes the bundled-jar license metadata correct and checks it in CI. > - It would be good to land these in 0.5.0, since they address the footnotes > from the 0.4.0 vote. > > 8. AI agent tooling (#888) > - The issues for an agent-legible CLI, a datasetConfig JSON Schema and an > MCP surface are filed (#889, #890, #891), and #829 adds an xtable-converter > skill. These are open for contributors. > > 9. Docs > - #922 addresses the open review comments on the 0.4.0 Spark runtime docs > (#920), and #921 covers when Delta Kernel is selected. > > PROPOSED TIMELINE > > I propose Thursday, October 8 as the code cutoff for 0.5.0, with anything > not merged by then moving to 0.6.0. Tim, does that work for you? > > Reviews this week would make the biggest difference. In particular, it > would be good to get eyes on #835, #894, #933, #935, #838, #839, #876 and > #895. > If there is an item you think must be in 0.5.0 that is not on the label, or > if the date does not work for your area, please reply on this thread. > > [1] [DISCUSS] Scope and timeline for Apache XTable (incubating) 0.5.0 and > future releases > > Thanks, > Vinish >
