nsivabalan commented on code in PR #19205:
URL: https://github.com/apache/hudi/pull/19205#discussion_r3654507874
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -1772,8 +1773,40 @@ public int getSmallFileGroupCandidatesLimit() {
return getInt(MERGE_SMALL_FILE_GROUP_CANDIDATES_LIMIT);
}
+ /**
+ * @return true when every meta column is populated.
+ *
+ * <p>Derived from {@link #getMetaFieldsMode()} so that call sites still
written against the
+ * deprecated {@code hoodie.populate.meta.fields} boolean observe the same
answer as the enum:
+ * only {@link MetaFieldsMode#ALL} populates every meta column.
+ */
public boolean populateMetaFields() {
- return getBooleanOrDefault(HoodieTableConfig.POPULATE_META_FIELDS);
+ return getMetaFieldsMode().toLegacyPopulateMetaFields();
+ }
+
+ /**
+ * @return the {@link MetaFieldsMode} resolved from the write config.
+ * {@code hoodie.meta.fields.mode} is the source of truth; configs written
before that property
+ * existed fall back to {@link MetaFieldsMode#ALL} or {@link
MetaFieldsMode#NONE} based on the
+ * deprecated {@code hoodie.populate.meta.fields} boolean.
+ */
+ public MetaFieldsMode getMetaFieldsMode() {
+ return
MetaFieldsMode.resolve(getStringOrDefault(HoodieTableConfig.META_FIELDS_MODE),
Review Comment:
Fixed in
[`c266b82`](https://github.com/apache/hudi/pull/19205/commits/c266b8286b4e).
`validateAgainstTableProperties` now compares the full `MetaFieldsMode`
instead of the legacy booleans:
```java
MetaFieldsMode tableMetaFieldsMode = tableConfig.getMetaFieldsMode();
MetaFieldsMode writeMetaFieldsMode = writeConfig.getMetaFieldsMode();
if (tableMetaFieldsMode != writeMetaFieldsMode) { throw ... }
```
Worth noting your scenario is a **narrowing**, not a widening — `NONE`
against a persisted `COMMIT_TIME_ONLY`. I'd separately added a widening-only
guard (`isWiderThan`) in the stacked PR, and it would *not* have caught this:
`NONE` is not wider than `COMMIT_TIME_ONLY`. Only a full mismatch check does.
Good catch.
`TestBaseHoodieWriteClient` gains a regression test for exactly that
combination — persisted `COMMIT_TIME_ONLY`, writer with `populate=false` and no
mode, both legacy booleans `false`. I confirmed it fails against the previous
boolean-only guard and passes with the enum comparison, so it isn't vacuous. A
second test covers the matching cases (default `ALL` writer, and a selective
writer against a table recorded with the same mode) to make sure the stricter
check doesn't reject legitimate flows — the metadata table, which forces
`NONE`, was the one I most wanted to confirm, and
`TestSparkRDDMetadataWriteClient` passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
