vinishjail97 opened a new issue, #968: URL: https://github.com/apache/incubator-xtable/issues/968
### Feature Request / Improvement `RunSync` (`xtable-utilities`) and `XTableSyncService` (`xtable-spark-runtime`, #964) both build the `SourceTable`, the `TargetTable` list and an incremental `ConversionConfig`, then pick a `ConversionSourceProvider` and call `ConversionController.sync`. Raised in https://github.com/apache/incubator-xtable/pull/964#discussion_r4238792218. Before the two can share this code, three differences need to be settled: - **Target base path.** `RunSync` sets the target base path to `tableBasePath`. `ITConversionController`, `ConversionService` and `XTableSyncService` use the source data path. The two differ only for an Iceberg source with a separate `data/` directory, where the Hudi target derives partition paths relative to its base path. Before e162b3bd, the Hudi and Delta targets also used `tableDataPath`, which is the layout in #404 and the Fabric guide. - **Source provider lookup.** `RunSync` reflects over `xtable-conversion-defaults.yaml` in `xtable-utilities`, which has no Delta Kernel or Parquet entry and needs `jackson-dataformat-yaml`. `XTableSyncService` uses a switch. - **Results.** `RunSync` only logs a failed sync, while the spark-runtime needs the `SyncResult` map to count failures. Proposal: move a small builder (source, targets, `ConversionConfig`) and a source provider lookup by format into `xtable-core`, use it from both callers, and align `RunSync` on the data path for targets. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
