Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19001#discussion_r183950638
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala
---
@@ -156,40 +144,14 @@ object FileFormatWriter extends Logging {
statsTrackers = statsTrackers
)
- // We should first sort by partition columns, then bucket id, and
finally sorting columns.
- val requiredOrdering = partitionColumns ++ bucketIdExpression ++
sortColumns
--- End diff --
Can we send an individual PR to do this? i.e. do the sorting via
`requiredOrdering` instead of doing it manually.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
