vinothchandar commented on a change in pull request #4067:
URL: https://github.com/apache/hudi/pull/4067#discussion_r773279641
##########
File path:
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java
##########
@@ -257,16 +275,20 @@ public Option getRecordByKey(String key, Schema
readerSchema) throws IOException
}
if (value != null) {
- R record = (R)HoodieAvroUtils.bytesToAvro(value, getSchema(),
readerSchema);
+ R record = (R) HoodieAvroUtils.bytesToAvro(value, getSchema(),
readerSchema);
+ materializeRecordIfNeeded(this.keyField,
ByteBuffer.wrap(key.getBytes()), record);
return Option.of(record);
}
return Option.empty();
}
- private R getRecordFromCell(Cell c, Schema writerSchema, Schema
readerSchema) throws IOException {
- byte[] value = Arrays.copyOfRange(c.getValueArray(), c.getValueOffset(),
c.getValueOffset() + c.getValueLength());
- return (R)HoodieAvroUtils.bytesToAvro(value, writerSchema, readerSchema);
+ private Pair<String, R> getRecordFromCell(Cell c, Schema writerSchema,
Schema readerSchema) throws IOException {
Review comment:
@prashantwason @manojpec Tying the thread here. A much smaller PR would
be to just fill the value of the cell's key into the `keyField` (if present) or
else `_hoodie_record_key` (for compatibility). keyField is something we'd know
even at the tableMetaClient level. So this need not touch the writeconfig as
long as its backwards compatible with all the HFile content out there.
I think I am okay to just enable this to be true always and make it work.
remove the need for the config and the plumbing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
