Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23010
The root cause is, `DynamicPartitionDataWriter` treats null and empty
string as different partition values, and creates new files. However, null and
empty string are converted to `__HIVE_DEFAULT_PARTITION__` at the end.
I think we should deal with invalid partition values ahead, so that we
don't need to worry about them during writing.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
