cloud-fan commented on issue #23010: [SPARK-26012][SQL]Null and '' values should not cause dynamic partition failure of string types URL: https://github.com/apache/spark/pull/23010#issuecomment-481273293 I just realized that we may create `InsertIntoHadoopFsRelationCommand` on the fly, e.g. in `OptimizedCreateHiveTableAsSelectCommand`. In this case an anlyze rule can not help. Another idea is in `FileFormatWriter.write`, we add a project above the input `SparkPlan` to add the `Empty2Null` expression. Sorry for the back and forth.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
