hudi-agent commented on code in PR #19509:
URL: https://github.com/apache/hudi/pull/19509#discussion_r3901119617
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/BulkInsertPartitioner.java:
##########
@@ -71,4 +74,22 @@ default String getFileIdPfx(int partitionId) {
default Option<WriteHandleFactory> getWriteHandleFactory(int partitionId) {
return Option.empty();
}
+
+ /**
+ * Whether the records being written carry a partition path, derived from
the write config alone.
+ * <p>
+ * A partitioner named through
+ * {@code HoodieWriteConfig.BULKINSERT_USER_DEFINED_PARTITIONER_CLASS_NAME}
is instantiated by
+ * reflection with only the write config, so an implementation that
otherwise takes the flag from
+ * the {@link HoodieTable} has nothing else to derive it from. The write
side partition path field
+ * governs whether records end up with a non-empty partition path, which is
what those
+ * implementations branch on.
+ *
+ * @param config Write config.
+ * @return {@code true} if a partition path field is configured; {@code
false} otherwise.
+ */
+ static boolean isTablePartitioned(HoodieWriteConfig config) {
+ return !StringUtils.isNullOrEmpty(
Review Comment:
🤖 When these partitioners are built by the factory, `isTablePartitioned`
comes from `table.isPartitioned()`, which resolves to
`HoodieTableConfig.isTablePartitioned()` (i.e. `hoodie.table.partition.fields`
/ PARTITION_FIELDS). Here the reflection path instead derives it from
`KeyGeneratorOptions.PARTITIONPATH_FIELD_NAME`
(`hoodie.datasource.write.partitionpath.field`). Those two keys can diverge —
e.g. a partitioned table whose write config doesn't carry the write-side
partitionpath field would get `false` here and silently fall back to
`coalesce()` instead of `partitionBy(partitionPath)` in
PartitionPathRepartition[AndSort]Partitioner. Is it worth reconciling to the
same source the factory uses (or documenting the intended precedence) so a
user-defined use of these partitioners repartitions the same way the built-in
sort mode does?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
