Github user eatoncys commented on the issue:
https://github.com/apache/spark/pull/23010
But we may forget to filter null values when we write sql. The following
function protects this situation and writes the value of null partitions as
__HIVE_DEFAULT_PARTITION__
def getPartitionPathString(col: String, value: String): String = {
val partitionString = if (value == null || value.isEmpty) {
DEFAULT_PARTITION_NAME
} else {
escapePathName(value)
}
escapePathName(col) + "=" + partitionString
}
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
