nsivabalan commented on code in PR #13305:
URL: https://github.com/apache/hudi/pull/13305#discussion_r2115101556
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/WriteStatus.java:
##########
@@ -61,6 +61,8 @@ public class WriteStatus implements Serializable {
private final List<Pair<HoodieRecordDelegate, Throwable>> failedRecords =
new ArrayList<>();
+ // true if this WriteStatus refers to a write happening in metadata table.
+ private boolean isMetadataTable;
Review Comment:
actually, we need to know whether a writestatus is coming from a data table
or metadata table.
there are chances that data table write status may not have any stats to add
only. for eg, a log file added to data table containing just delete block may
not have col stats or Secondary index stats for instance.
So, we need a way to differentiate these and categorize them accordingly.
eventually, we will route data table's write status to writeclient.commit()
pertaining to data table. and metadata table's write status pertaining to
metadata table to writeClient.commit() pertaining to metadata table.
Not sure, why adding an additional isMetadataTable() is not acceptable in
this case.
for instance, we do use HoodieBaseFile to represent a base file as well as a
bootstrap file.
I am w/ you on this `we can add optional component named IndexStats`. and
only set it for data table.
but we need a deterministic way to differentiate data table write status
from metadata table write status.
let me know if you have any other good suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
