danny0405 commented on code in PR #9013:
URL: https://github.com/apache/hudi/pull/9013#discussion_r1236784837
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/UpsertPartitioner.java:
##########
@@ -379,6 +374,9 @@ protected static long averageBytesPerRecord(HoodieTimeline
commitTimeline, Hoodi
Iterator<HoodieInstant> instants =
commitTimeline.getReverseOrderedInstants().iterator();
while (instants.hasNext()) {
HoodieInstant instant = instants.next();
+ if (instant.getAction().equals(REPLACE_COMMIT_ACTION)) {
+ continue;
Review Comment:
Would prefer `(CollectionUtils.createSet(COMMIT_ACTION,
DELTA_COMMIT_ACTION)`, one thing needs to note is the avro file uses the Java
reflection to calcute the in-memory size of a record, while Parquet file size
is the actual file size so it is more accurate, not sure whether we should
include the log if there are parquets, that means, the parquet file size should
have higher priority than log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
