rangareddy commented on issue #17052: URL: https://github.com/apache/hudi/issues/17052#issuecomment-5391451187
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-9515). **Findings: still open.** The constraint is visible in the code as a runtime type assertion rather than a signature: `HoodieBackedTableMetadata.java:522` asserts `secondaryKeys instanceof HoodieListData` with the message "readSecondaryIndex only support HoodieListData at the moment", with a second implementation-driven branch at `:550`. So the API advertises `HoodieData` but rejects the distributed implementation. **This is one of four related tickets**, all the same underlying problem - the `HoodieData` abstraction is not honoured end to end, so callers cannot pass a distributed implementation where the API says they can: - #17052 (HUDI-9515) - index lookup requires the set to fit on one machine; needs an RDD-based implementation - #17063 (HUDI-9542) - `HoodieListData` / `HoodieListPairData` need a notion of partitions - #17064 (HUDI-9544) - the RLI metric does not fire for RDD because of lazy evaluation - #17101 (HUDI-9613) - `HoodieBackedTableMetadata` asserts `instanceof HoodieListData` at runtime Fixing them independently risks four partial answers. Recommend one design covering what `HoodieData` guarantees - partitioning, laziness, and which implementations each API accepts - with these as its consequences. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
