vinishjail97 opened a new issue, #19886: URL: https://github.com/apache/hudi/issues/19886
### Describe the problem #19869 adds record index (RLI) and secondary index (SI) support for parquet files written outside Hudi and registered through replace commits without a known operation type, the way tables converted from other formats are. Such a table has no record key, so every row is keyed by the file path relative to the table and the row position. Three code paths that a replace commit with `INSERT_OVERWRITE`, `INSERT_OVERWRITE_TABLE` or `DELETE_PARTITION` takes were left out of that PR and still assume Hudi-written files: 1. `HoodieAvroParquetReader.getRecordKeyIterator` reads `_hoodie_record_key` with no null guard, so a key-less file under one of these operation types fails with a `NullPointerException` instead of the "Record key is missing" message that `ParquetUtils.filterParquetRowKeys` now raises. `BaseRecordIndexer.getRecordIndexAdditionalUpserts` reaches it through `HoodieTableMetadataUtil.readRecordKeysFromBaseFiles`. 2. `HoodieTableMetadataUtil.readRecordKeysFromBaseFiles` builds the data file path from `baseFile.getFileName()`, which drops the directory prefix of an external file registered below a subdirectory of its partition (for example `bucket-0/file.parquet`). It has the same shape as the bug fixed in `SecondaryIndexRecordGenerationUtils.readSecondaryKeysFromFileSlices` by #19869, which now uses `HoodieBaseFile::getStoragePath`. 3. `DELETE_PARTITION` shares the same reader and path construction. The repository's own external-file registration test, `TestExternalPathHandling`, commits with `INSERT_OVERWRITE`, so the in-repo flow takes these branches. XTable sets `UNKNOWN`, which #19869 covers. ### Proposed change - Give `HoodieAvroParquetReader.getRecordKeyIterator` the same positional-key fallback as `ParquetUtils`, gated on `HoodieTableConfig.hasRecordKey()`. - Build the data file path in `HoodieTableMetadataUtil.readRecordKeysFromBaseFiles` from `HoodieBaseFile.getStoragePath()`. - Cover `INSERT_OVERWRITE`, `INSERT_OVERWRITE_TABLE` and `DELETE_PARTITION` on a key-less table with a prefixed file in `TestExternalFileRecordAndSecondaryIndex`. ### Context Review thread: https://github.com/apache/hudi/pull/19869#discussion_r3965129565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
