rangareddy commented on issue #17310: URL: https://github.com/apache/hudi/issues/17310#issuecomment-5365820571
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8845). **Findings: confirmed, the API has not been added.** The complete public surface of `hudi-common/src/main/java/org/apache/hudi/common/util/FileFormatUtils.java` on `master`: | method | line | | --- | --- | | `readRowKeys` | `:150` | | `readAvroRecords` (two overloads) | `:212`, `:222` | | `readFooter` | `:233` | | `getRowCount` | `:242` | | `filterRowKeys` | `:253` | | `fetchRecordKeysWithPositions` | `:262`, `:311` | | `getHoodieKeyIterator` | `:275`, `:287` | | `readSchema` | `:327` | | `readColumnStatsFromMetadata` | `:338` | | `writeMetaFile` | `:356` | | `serializeRecordsToLogBlock` | `:372` | None of these takes a set of columns and returns an iterator of records. The closest are the key-oriented readers, which project only the record-key fields, and `readAvroRecords`, which reads everything. So the motivating use case in the description - updating the secondary index when a file slice has only a base file - still has to read full records where a projection of the indexed column plus the record key would do. That is the read-amplification this ticket exists to remove. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
