rangareddy opened a new pull request, #850: URL: https://github.com/apache/incubator-xtable/pull/850
## What is the purpose of the pull request The demo notebook (`demo/notebook/demo.ipynb`) no longer runs on current `main`. This PR fixes all the issues reported in #727 plus three more found while getting it running end-to-end. Closes #727 ## Changes **`demo/notebook/demo.ipynb`** 1. **Jar coordinate**: `xtable-core-0.2.0-SNAPSHOT.jar` → `xtable-core_2.12-0.2.0-SNAPSHOT.jar` (artifact gained the Scala binary version suffix). 2. **API drift**: ported all sync cells from the removed `PerTableConfigImpl` (deleted in #480) to the current `SourceTable`/`TargetTable`/`ConversionConfig` builders, and `ConversionSourceProvider#init` to its single-`Configuration` signature. Explicit empty `Properties` are passed on the builders because `HudiSourceConfig.fromProperties` throws an NPE when `additionalProperties` is null. 3. **Dependency versions**: Hudi `0.14.0` → `1.2.0` (matches #772) and `iceberg-hive-runtime` `1.4.2` → `1.7.2` (the version pinned in the root pom). Added `delta-kernel-api`/`delta-kernel-defaults` `4.0.0` — they are `provided`-scope dependencies of xtable-core, and without them the `ConversionTarget` ServiceLoader fails on `DeltaKernelConversionTarget` before any sync can run. 4. **Removed the `hoodie.write.schema` override**: under Hudi 1.2's lazy record serialization the rows are encoded with the nullable dataframe-derived schema but were decoded with the non-null override schema, so every record failed with `AvroRuntimeException: Malformed data. Length is negative`. With the override removed one consistent schema is used. **`demo/docker-compose.yaml` + `demo/jupyter/setup-jdk11.sh` (new)** Hudi 1.x classes are Java 11 bytecode, but the `almondsh/almond` image only ships Java 8 (there is no JDK 11 tag), which fails with `UnsupportedClassVersionError`. The new `before-notebook.d` startup hook installs OpenJDK 11 via mamba on first container start and points the Scala kernels at it. ## Verification Executed every cell of the notebook end-to-end against a freshly recreated jupyter container (so the JDK 11 hook provisioning is exercised from scratch) with pristine seed data: - Hudi → Iceberg sync + incremental re-sync after appending round-2 CSV - Trino `INSERT` into the Iceberg table, then the new row read back **as Delta** via Spark - Delta → Hudi sync and read-back as Hudi - Cross-format Delta join of the two tables - Delta → Iceberg sync Final state via Trino: `iceberg.demo.hudi_dimCustomer` = 5001 rows, `iceberg.demo.delta_dimGeography` = 674 rows. Note for reviewers: on first container start the hook downloads OpenJDK 11 (~200 MB, one-time); subsequent starts are instant. ## Verify this pull request This pull request is a trivial rework / code cleanup without any test coverage. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
