ulysses-you commented on code in PR #37831:
URL: https://github.com/apache/spark/pull/37831#discussion_r966786913
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala:
##########
@@ -477,7 +480,7 @@ case class DataSource(
// will be adjusted within InsertIntoHadoopFsRelation.
InsertIntoHadoopFsRelationCommand(
outputPath = outputPath,
- staticPartitions = Map.empty,
+ staticPartitions = staticPartitions,
Review Comment:
I think the main benefit is we can save a local sort. yes, I agree, the
current implementation is over complex. The reason is I guess some downstream
projects or extensions may depend on the static partitions(e.g. add repartition
for dynamic partition writes), so I merge the infered static partitions into
the original. I'm fine to simplify the code to just remove a sort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
