Hi folks, The biggest hurdle I see right now is the handling of the license files and automation around it. There doesn't seem to be much alignment there and I don't personally have much experience with it. Is there anyone with a clear idea of the path we should take? I know there have been a few threads that have died out on the subject.
To add some context on some of the stale branches, they were dependent on some features in the Hudi 1.X releases. Some of those releases introduced regressions in behavior that XTable relied on so we had to wait for another release cycle. I still think it may be valuable to have the Delta and Spark versions upgraded as part of their own PR as they are in #908 but it looks like that is now closed. Are we targeting a Spark and Delta upgrade in this release? -Tim On Mon, Oct 5, 2026 at 6:51 AM Vinoth Chandar <[email protected]> wrote: > Thanks Tim for driving this release. > > On Fri, Oct 2, 2026 at 15:58 Vinish Reddy <[email protected]> wrote: > > > Hi all, > > > > Here is a status update on the 0.5.0-incubating release, following up on > > the scope and timeline thread from August [1]. Tim is the release manager > > for 0.5.0. > > > > The full list of tracked work is under the 0.5.0 label. > > > https://github.com/apache/incubator-xtable/issues?q=is%3Aopen+label%3A0.5.0 > > > > MOMENTUM SINCE 0.4.0 > > > > Since August 1, 31 PRs have merged on main from 9 contributors. Some > > highlights: > > > > - The xtable-utilities bundled jar shrank from 1.1 GB to 423 MB (#897), > and > > the duplicate AWS SDK v2 modules were removed (#923). > > - Delta nested field ids now carry into the converted schema (#912), and > > Delta log listing handles absolute paths (#932). > > - Incremental sync is skipped when a Delta source has no new commits > > (#930), and the Delta snapshot is reused across an incremental backlog > > (#861). > > - Hudi partitions are batch-loaded before the parallel file listing > (#950). > > - Glue Catalog Federation works with Delta tables (#940). > > - The parquet source now populates record_count (#929), and its > limitations > > are documented (#937). > > - The build moved to ASF parent POM 39 and compiles on JDK 21+ (#881, > > #900), with the Maven build cache persisted in CI (#882). > > - The 0.4.0 release notes, blog post and Spark runtime quickstart are > live > > (#913, #915). > > > > Thanks to Sylwester Lachiewicz, Ranga Reddy, Arvin, Abdul Megoon, Akash > > Reddy Jammula, Tim Brown, Vamsi Karnika and AadhiKat for their > > contributions, and to everyone who reviewed and tested. > > > > STATUS BY THEME > > > > 1. Hudi table version 9 (Hudi 1.x) support (#834) > > - #835 adds v9 to the Hudi target and source. It now builds on the > released > > Hudi 1.2.1, CI is green, and it is ready for review. > > > > 2. Iceberg pluggable table format managed by Hudi (#722), led by Ethan > > - #894 implements the Iceberg versions of TableFormat and > > HoodieTableMetadata. CI is green and review is in progress. > > - #954 (Iceberg V3 deletion vectors and table services for MOR tables) > and > > #955 (a Spark plugin that routes Iceberg writes through Hudi) are early > > drafts that build on #894. > > > > 3. Cross-format indexing (#887) > > - #933 is RFC-4 for index support. > > - #935 adds a Hudi-backed secondary index for Iceberg tables. It has been > > validated on large tables on S3 and is close to ready. > > - The xtable-spark-extensions work (Spark SQL to build, drop and check > the > > staleness of indexes) is pushed and stacks on #935. A PR will follow. > > > > 4. xtable-spark-runtime for Hudi 1.x (#836) > > - The runtime shipped for 0.4.x on branch-0.4 (#843). > > - #838 (thin drop-in bundle and resilient target discovery) and #839 > > (XTableSyncListener for in-job sync) bring it to main. > > - #931 adds Parquet source partitioning to the spark runtime (#927). > > - #908 from Sylwester moves the build to Spark 3.5.9 and Delta 3.3.3, as > > the first step toward the Spark 4 upgrade (#902). Reviews are welcome > there > > too. > > > > 5. Delta Kernel as the default Delta path (#886, #713) > > - #903 adds Delta Kernel source and target coverage to > > ITConversionController and is approved. > > - #947 (local paths, #943) and #942 (commit tag mapping, #819) fix Kernel > > gaps found along the way. > > > > 6. Correctness and robustness fixes. > > - #951 falls back to parquet-footer stats when the Hudi metadata table > > returns degenerate values. It is approved and ready to merge. > > - #621 adds a custom Iceberg metadata cleaner. > > - #956 resolves the parquet source file system from the base path, and > #957 > > keeps absolute paths for data files outside the table base path in Delta > > targets. > > - #938 adds fail-safe handling for Delta deletion vectors (#657). > > - #952 bundles Iceberg's GlueCatalog and S3FileIO in xtable-utilities > > (#944). > > - #814 filters metadata directories out of Hudi partitions (#813). > > - Open bugs without a PR yet are #958, #959, #941, #936, #901 and #736. > > Help on any of these is welcome. > > > > 7. License and packaging (#701, #717, #865) > > - #876 keeps exactly one LICENSE and NOTICE in every published jar, and > > #895 makes the bundled-jar license metadata correct and checks it in CI. > > - It would be good to land these in 0.5.0, since they address the > footnotes > > from the 0.4.0 vote. > > > > 8. AI agent tooling (#888) > > - The issues for an agent-legible CLI, a datasetConfig JSON Schema and an > > MCP surface are filed (#889, #890, #891), and #829 adds an > xtable-converter > > skill. These are open for contributors. > > > > 9. Docs > > - #922 addresses the open review comments on the 0.4.0 Spark runtime docs > > (#920), and #921 covers when Delta Kernel is selected. > > > > PROPOSED TIMELINE > > > > I propose Thursday, October 8 as the code cutoff for 0.5.0, with anything > > not merged by then moving to 0.6.0. Tim, does that work for you? > > > > Reviews this week would make the biggest difference. In particular, it > > would be good to get eyes on #835, #894, #933, #935, #838, #839, #876 and > > #895. > > If there is an item you think must be in 0.5.0 that is not on the label, > or > > if the date does not work for your area, please reply on this thread. > > > > [1] [DISCUSS] Scope and timeline for Apache XTable (incubating) 0.5.0 and > > future releases > > > > Thanks, > > Vinish > > >
