rangareddy commented on issue #17294: URL: https://github.com/apache/hudi/issues/17294#issuecomment-5365816393
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8397). **Findings: this has been done. Closing as fixed.** The ask was to move the non-engine-specific part of `SparkHoodieBackedTableMetadataWriter.getFunctionalIndexRecords` into common code, leaving only the engine-specific transformation overridable in Spark. **Before** (`a001ea87da7b^`): `SparkHoodieBackedTableMetadataWriter.java:273` defined `getExpressionIndexRecords` - the renamed `getFunctionalIndexRecords` - and assembled the records inline at `:268-282`. It was one of four per-engine overrides of an abstract method declared on `HoodieBackedTableMetadataWriter:636` (Spark, Spark-table-version-six, Flink and Java each had their own). **Now**: that method is gone. `SparkHoodieBackedTableMetadataWriter` contains only engine plumbing - `convertHoodieDataToEngineSpecificData` (`:140`), `convertEngineSpecificDataToHoodieData` (`:145`), `streamWriteToMetadataTable` (`:150`), `upsertAndCommit`, `bulkCommit` and so on. The orchestration lives in the engine-agnostic `ExpressionIndexer` in `hudi-client-common` (`hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/index/expression/ExpressionIndexer.java`), and the engine-specific step is now an SPI method, `EngineIndexerSupport.generateExpressionIndexRecords` (`EngineIndexerSupport.java:67`), implemented by `SparkIndexerSupport` with `UnsupportedEngineIndexerSupport` covering Flink and Java. Landed via PR #18348 (`a001ea87da7b`) and PR #18372 (`2baa29b14d37`), both merged 2026-07-08. One caveat for the record: the logic moved to `ExpressionIndexer` / `IndexerFactory` rather than literally onto `HoodieBackedTableMetadataWriter` as the description suggested. The stated goal - engine-agnostic core with an overridable engine-specific step - is met. **Not to be confused with #17399 (HUDI-9343)**, which asks for the next step: replacing this SPI with a proper engine-based expression *evaluator* so record construction also moves into common code. That remains open. Closing as fixed. Please reopen if the intended scope here was broader. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
