hudi-agent commented on code in PR #19205:
URL: https://github.com/apache/hudi/pull/19205#discussion_r3720599434
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java:
##########
@@ -1294,6 +1296,21 @@ private Pair<HoodieWriteConfig, HoodieSchema>
getHoodieClientConfigAndWriterSche
if (metaClient != null) {
HoodieTableConfig tableConfig = metaClient.getTableConfig();
+ // Inherit the table's meta-fields mode when this run does not state
one, mirroring how the Spark
+ // datasource folds table props into the write params
(HoodieSparkSqlWriter#mergeParamsAndGetHoodieConfig).
+ //
+ // Meta-field population is physical, so it belongs to the table, not to
the run. StreamSync builds
+ // its write config from `props` alone, and the mode is persisted only
in `initializeEmptyTable` --
+ // which runs solely when the base path does not exist. So a restart
against an existing table that
+ // passes only the legacy boolean (or nothing at all) resolved to NONE
and wrote base files with a
+ // null _hoodie_commit_time, while hoodie.properties still advertised
COMMIT_TIME_ONLY. Incremental
+ // queries were then admitted and silently dropped every one of those
rows.
+ //
+ // Only fills the gap: an explicitly stated mode is left alone so a
genuine conflict is still caught
+ // downstream by BaseHoodieWriteClient#validateAgainstTableProperties.
+ if (!props.containsKey(HoodieTableConfig.META_FIELDS_MODE.key())) {
Review Comment:
🤖 This inherits the table's mode whenever `META_FIELDS_MODE` is absent from
the run props, but it doesn't consider `POPULATE_META_FIELDS`. So a restart
that explicitly passes `hoodie.populate.meta.fields=false` against an ALL table
now silently resolves to ALL instead of narrowing to NONE — the narrowing that
`validateAgainstTableProperties` still documents as long-standing behavior
(HUDI-2161). Is overriding an explicitly-set legacy boolean intended? The
comment says an explicitly-stated mode is left alone, but that only holds for
`META_FIELDS_MODE`, not the deprecated boolean.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
