danny0405 commented on PR #10398:
URL: https://github.com/apache/hudi/pull/10398#issuecomment-1903283157
Thanks for raising this fix, I think it is a good chance we fix the event
time sequence comparison of delete records with payloads, I can see 2 mistaks
in our code that uses processing time sequence for deletes:
1. `OverwriteWithLatestAvroPayload#preCombine`:
```java
public OverwriteWithLatestAvroPayload
preCombine(OverwriteWithLatestAvroPayload oldValue) {
if (oldValue.recordBytes.length == 0) {
// use natural order for delete record
return this;
}
...
}
```
2. `DefaultHoodieRecordPayload#combineAndGetUpdateValue`
```java
public Option<IndexedRecord> combineAndGetUpdateValue(IndexedRecord
currentValue, Schema schema, Properties properties) throws IOException {
if (recordBytes.length == 0) {
return Option.empty();
}
...
}
```
In any case, the `orderingVal` should be set up correctly and we should
utilize it as much as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
