suryaprasanna commented on code in PR #17787:
URL: https://github.com/apache/hudi/pull/17787#discussion_r2739013263
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/catalyst/catalog/HoodieCatalogTable.scala:
##########
@@ -310,6 +316,17 @@ class HoodieCatalogTable(val spark: SparkSession, var
table: CatalogTable) exten
extraConfig(URL_ENCODE_PARTITIONING.key) =
String.valueOf(isUrlEncodeEnabled(allPartitionPaths, table))
}
+ if
(originTableConfig.contains(HoodieTableConfig.SLASH_SEPARATED_DATE_PARTITIONING.key))
{
+ extraConfig(HoodieTableConfig.SLASH_SEPARATED_DATE_PARTITIONING.key) =
+
originTableConfig(HoodieTableConfig.SLASH_SEPARATED_DATE_PARTITIONING.key)
+ } else {
+ extraConfig(HoodieTableConfig.SLASH_SEPARATED_DATE_PARTITIONING.key) =
+ String.valueOf(isHierarchicalDatePartitioning(allPartitionPaths,
table))
Review Comment:
Yeah, we can drop it.
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/SparkHoodieTableFileIndex.scala:
##########
@@ -410,11 +410,13 @@ class SparkHoodieTableFileIndex(spark: SparkSession,
val (staticPartitionColumnNames, staticPartitionColumnValues) =
staticPartitionColumnNameValuePairs.unzip
val hiveStylePartitioning =
metaClient.getTableConfig.getHiveStylePartitioningEnable.toBoolean
+ val hierarchicalDatePartitioning =
metaClient.getTableConfig.getSlashSeparatedDatePartitioning.toBoolean
Review Comment:
Yes, made the changes accordingly.
##########
hudi-common/src/main/java/org/apache/hudi/keygen/constant/KeyGeneratorOptions.java:
##########
@@ -34,6 +34,12 @@
description = "")
public class KeyGeneratorOptions extends HoodieConfig {
+ public static final ConfigProperty<String> SLASH_SEPARATED_DATE_PARTITIONING
= ConfigProperty
+ .key("hoodie.datasource.write.slash.separated.date.partitionpath")
+ .defaultValue("false")
+ .markAdvanced()
+ .withDocumentation("Should we url encode the partition path value,
before creating the folder structure.");
Review Comment:
Adding from version as 1.12.0 is failing the job. Since that version is not
yet released. What do you think we should do?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
