manojpec commented on a change in pull request #4449:
URL: https://github.com/apache/hudi/pull/4449#discussion_r792079895
##########
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java
##########
@@ -508,6 +520,255 @@ public void
testMetadataTableWithPendingCompaction(boolean simulateFailedCompact
}
}
+ /**
+ * Test arguments - Table type, populate meta fields, exclude key from
payload.
+ */
+ public static List<Arguments> testMetadataRecordKeyExcludeFromPayloadArgs() {
Review comment:
Sure, trimmed the test, reduced the run time and moved it to
TestHoodieBackedTableMetadata.
##########
File path:
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java
##########
@@ -257,16 +265,60 @@ public Option getRecordByKey(String key, Schema
readerSchema) throws IOException
}
if (value != null) {
- R record = (R)HoodieAvroUtils.bytesToAvro(value, getSchema(),
readerSchema);
+ R record = deserialize(key.getBytes(), value, getSchema(), readerSchema,
keySchemaField);
return Option.of(record);
}
return Option.empty();
}
- private R getRecordFromCell(Cell c, Schema writerSchema, Schema
readerSchema) throws IOException {
- byte[] value = Arrays.copyOfRange(c.getValueArray(), c.getValueOffset(),
c.getValueOffset() + c.getValueLength());
- return (R)HoodieAvroUtils.bytesToAvro(value, writerSchema, readerSchema);
+ /**
+ * Get the record from HBase cell.
+ *
+ * @param cell - HBase Cell
Review comment:
removed this.
##########
File path:
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java
##########
@@ -196,6 +198,8 @@ public BloomFilter readBloomFilter() {
@Override
public Iterator getRecordIterator(Schema readerSchema) throws IOException {
final HFileScanner scanner = reader.getScanner(false, false);
+ final Option<Schema.Field> keySchemaField =
Option.ofNullable(readerSchema.getField(KEY_FIELD_NAME));
+ ValidationUtils.checkState(keySchemaField != null);
Review comment:
added the assertion message.
##########
File path:
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java
##########
@@ -196,6 +198,8 @@ public BloomFilter readBloomFilter() {
@Override
public Iterator getRecordIterator(Schema readerSchema) throws IOException {
final HFileScanner scanner = reader.getScanner(false, false);
+ final Option<Schema.Field> keySchemaField =
Option.ofNullable(readerSchema.getField(KEY_FIELD_NAME));
Review comment:
fixed.
##########
File path:
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java
##########
@@ -151,15 +153,15 @@ public BloomFilter readBloomFilter() {
}
public List<Pair<String, R>> readAllRecords(Schema writerSchema, Schema
readerSchema) throws IOException {
+ final Option<Schema.Field> keySchemaField =
Option.ofNullable(readerSchema.getField(KEY_FIELD_NAME));
Review comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
