Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23010 The root cause is, `DynamicPartitionDataWriter` treats null and empty string as different partition values, and creates new files. However, null and empty string are converted to `__HIVE_DEFAULT_PARTITION__` at the end. I think we should deal with invalid partition values ahead, so that we don't need to worry about them during writing.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org