yihua commented on code in PR #9892:
URL: https://github.com/apache/hudi/pull/9892#discussion_r1371413479
##########
hudi-common/src/main/java/org/apache/hudi/common/model/DefaultHoodieRecordPayload.java:
##########
@@ -86,30 +86,26 @@ public Option<IndexedRecord> getInsertValue(Schema schema,
Properties properties
GenericRecord incomingRecord = HoodieAvroUtils.bytesToAvro(recordBytes,
schema);
eventTime = updateEventTime(incomingRecord, properties);
- return isDeleteRecord(incomingRecord, properties) ? Option.empty() :
Option.of(incomingRecord);
+ return isDeleted(schema, properties) ? Option.empty() :
Option.of(incomingRecord);
}
- /**
- * @param genericRecord instance of {@link GenericRecord} of interest.
- * @param properties payload related properties
- * @returns {@code true} if record represents a delete record. {@code false}
otherwise.
- */
- protected boolean isDeleteRecord(GenericRecord genericRecord, Properties
properties) {
- final String deleteKey = properties.getProperty(DELETE_KEY);
+ @Override
+ protected boolean isDeleteRecord(GenericRecord record, Properties props) {
+ final String deleteKey = props.getProperty(DELETE_KEY);
if (StringUtils.isNullOrEmpty(deleteKey)) {
- return isDeleteRecord(genericRecord);
+ return super.isDeleteRecord(record, props);
Review Comment:
The intention is that `isDeleted(Schema schema, Properties props)` should
not deserialize the bytes for perf reason so we cannot implement the custom
delete marker there for `DefaultHoodieRecordPayload`. The way now in this fix
is to check the delete key and marker in the constructor before the Avro record
is serialized; that’s why props needs to be passed in to the constructor.
##########
hudi-common/src/main/java/org/apache/hudi/common/model/DefaultHoodieRecordPayload.java:
##########
@@ -86,30 +86,26 @@ public Option<IndexedRecord> getInsertValue(Schema schema,
Properties properties
GenericRecord incomingRecord = HoodieAvroUtils.bytesToAvro(recordBytes,
schema);
eventTime = updateEventTime(incomingRecord, properties);
- return isDeleteRecord(incomingRecord, properties) ? Option.empty() :
Option.of(incomingRecord);
+ return isDeleted(schema, properties) ? Option.empty() :
Option.of(incomingRecord);
}
- /**
- * @param genericRecord instance of {@link GenericRecord} of interest.
- * @param properties payload related properties
- * @returns {@code true} if record represents a delete record. {@code false}
otherwise.
- */
- protected boolean isDeleteRecord(GenericRecord genericRecord, Properties
properties) {
- final String deleteKey = properties.getProperty(DELETE_KEY);
+ @Override
+ protected boolean isDeleteRecord(GenericRecord record, Properties props) {
+ final String deleteKey = props.getProperty(DELETE_KEY);
if (StringUtils.isNullOrEmpty(deleteKey)) {
- return isDeleteRecord(genericRecord);
+ return super.isDeleteRecord(record, props);
Review Comment:
The intention is that `isDeleted(Schema schema, Properties props)` should
not deserialize the bytes for perf reason so we cannot implement the custom
delete marker there for `DefaultHoodieRecordPayload`. The way now in this fix
is to check the delete key and marker in the constructor before the Avro record
is serialized; that’s why props needs to be passed in to the constructor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
