rangareddy commented on issue #17294: URL: https://github.com/apache/hudi/issues/17294#issuecomment-5366038326
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8397). **Findings: the refactor named in the title has landed. Keeping this open because whether that satisfies the description in full is a scope call, not a code question.** **What landed.** `SparkHoodieBackedTableMetadataWriter` no longer contains any expression/functional index record method. It holds only engine plumbing - `convertHoodieDataToEngineSpecificData` (`:140`), `convertEngineSpecificDataToHoodieData` (`:145`), `streamWriteToMetadataTable` (`:150`), `upsertAndCommit`, `bulkCommit`, `deletePartitions`. Before the refactor (`a001ea87da7b^`) that same class defined `getExpressionIndexRecords` at `:273` and assembled the records inline at `:268-282`, as one of four per-engine overrides of an abstract method on `HoodieBackedTableMetadataWriter:636` - Spark, Spark-table-version-six, Flink and Java each had their own. Now the orchestration lives in the engine-agnostic `ExpressionIndexer` in `hudi-client-common` (`hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/index/expression/ExpressionIndexer.java`), and the engine-specific step is an SPI method, `EngineIndexerSupport.generateExpressionIndexRecords` (`EngineIndexerSupport.java:67`), implemented by `SparkIndexerSupport` with `UnsupportedEngineIndexerSupport` covering Flink and Java. Delivered by PR #18348 (`a001ea87da7b`) and PR #18372 (`2baa29b14d37`), both merged 2026-07-08. **Why this is staying open.** The description asks for "the rest of the logic" to move to common code, with only the engine-specific transformation overridden in Spark. The SPI that resulted still has each engine return finished `HoodieRecord`s rather than only evaluating the expression - so record construction is still engine-side. Removing exactly that shim, in favour of an engine-based expression evaluator, is what #17399 (HUDI-9343) tracks, and it is still open. So under a strict reading of this description the move is partial, and the natural resolution is either to close this in favour of #17399 or to keep it as the umbrella for that remaining step. That is a call for whoever owns HUDI-9096 rather than something the code decides. Leaving this open pending that decision. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
