voonhous commented on code in PR #8687:
URL: https://github.com/apache/hudi/pull/8687#discussion_r1190816528
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/SparkHoodieTableFileIndex.scala:
##########
@@ -319,6 +319,8 @@ class SparkHoodieTableFileIndex(spark: SparkSession,
* @return relative partition path and a flag to indicate if the path is
complete (i.e., not a prefix)
*/
private def
composeRelativePartitionPath(staticPartitionColumnNameValuePairs: Seq[(String,
(String, Any))]): String = {
+ // NOTE: Do not blindly use default value if tableConfig does not have the
hiveStylePartitioningEnable config
+ checkState(metaClient.getTableConfig.getHiveStylePartitioningEnableOrNull
!= null)
Review Comment:
We are still investigating this issue on why hoodie.properties does not have
the `hoodie.datasource.write.hive_style_partitioning` config key.
However, we are seeing that some tables do not have this configuration.
Given that there's an unsafe usage and unsafe assumption, i figured that it
would be better to add a check here to prevent worse-case scenario from
happening, i.e. wrong partitioning format is assumed and a partition is then
created.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
