wombatu-kun commented on code in PR #20366:
URL: https://github.com/apache/hudi/pull/20366#discussion_r4293868678
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/FileGroupReaderBasedInlineLogAppendHandle.java:
##########
@@ -103,7 +104,7 @@ public void doAppend() {
// instead of using config.enableOptimizedLogBlocksScan(), we set to
true as log compaction blocks only supported in scanV2
.build()) {
recordItr = new
CloseableMappingIterator<>(fileGroupReader.getLogRecordsOnly(), record -> {
- HoodieRecord<T> hoodieRecord =
readerContext.getRecordContext().constructHoodieRecord(record);
+ HoodieRecord<T> hoodieRecord =
readerContext.getRecordContext().constructHoodieRecord(withJavaOrderingValue(record));
Review Comment:
Done b050060b85ae: the inline handle no longer converts. The
`HoodieEmptyRecord` branch of the Spark `constructHoodieRecord` now does, and
`HoodieEmptyRecord.getOrderingValueAsJava()` returns that Java value while
`getOrderingValue()` keeps the engine one, the same split `HoodieSparkRecord`
has. Storing only the Java value breaks v10, whose native delete writer puts
`getOrderingValue()` into a Spark row: `SPARK;decimal(10,2);10` then fails with
`BigDecimal cannot be cast to Decimal` and log compaction drops the delete.
This also covers deletes that win precombine:
`testPrecombinedDeleteWithEngineOrdering` from #20341 passes on this branch
without the `isEmpty()` guard. The native handle keeps a conversion only for
the Avro record type, where Spark-read deletes reach the Avro delete writer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
