vinishjail97 opened a new issue, #934: URL: https://github.com/apache/incubator-xtable/issues/934
## Feature request The Hudi conversion target registers parquet files written by other formats (Iceberg, Delta, Paimon) in a Hudi table and maintains the files and column stats partitions of the Hudi metadata table. It cannot build the Hudi **record index** or a **secondary index** for those files today: - the registered files carry no `_hoodie_record_key` column, so Hudi has no record key to index, and - the Hudi target only runs on the Java engine, while building those indexes reads every registered file and is better distributed on Spark. apache/hudi#19869 makes both indexes work for such files by keying each row with `<file path relative to the table>_<row position>`. With that, XTable can: 1. let the Hudi target enable the record index and a secondary index on a configured column, optionally on the Spark engine, and 2. expose a lookup that resolves the values of the indexed column to the data file, row position and partition that hold them, so engines can locate the rows touched by a merge or delete on an Iceberg table without joining against the whole table. Depends on #834 (Hudi table version 9 support) since the secondary index needs table version 8 or later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
