n3nash commented on a change in pull request #1119: Fix: HoodieCommitMetadata
only show first commit insert rows.
URL: https://github.com/apache/incubator-hudi/pull/1119#discussion_r361273870
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieCommitMetadata.java
##########
@@ -175,7 +175,9 @@ public long fetchTotalInsertRecordsWritten() {
long totalInsertRecordsWritten = 0;
for (List<HoodieWriteStat> stats : partitionToWriteStats.values()) {
for (HoodieWriteStat stat : stats) {
- if (stat.getPrevCommit() != null &&
stat.getPrevCommit().equalsIgnoreCase("null")) {
Review comment:
@cdmikechen I think the problem is here : The FileSizing of Hoodie takes
care of padding extra records to the same file till the file grows to a certain
size. This logic lies in HoodieMergeHandle.java. Since the merge handle will
always have a prevCommit, for the records that are inserted as part of the
HoodieMergeHandle (records padded to an existing file), those will not pass the
condition of stat.getPrevCommit().equalsIgnoreCase("null"). So, I think
removing the check `stat.getPrevCommit().equalsIgnoreCase("null")` and then
adding the num-inserts should be the right thing to do. Let me know if this
makes sense to you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
