[GitHub] spark issue #23163: [SPARK-26164][SQL] Allow FileFormatWriter to write multi...

2018-12-03 Thread c21
Github user c21 commented on the issue: https://github.com/apache/spark/pull/23163 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23163: [SPARK-26164][SQL] Allow FileFormatWriter to write multi...

2018-12-02 Thread c21
Github user c21 commented on the issue: https://github.com/apache/spark/pull/23163 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23163: [SPARK-26164][SQL] Allow FileFormatWriter to write multi...

2018-12-01 Thread c21
Github user c21 commented on the issue: https://github.com/apache/spark/pull/23163 cc @cloud-fan and @gatorsmile: I think this pr is ready for review. Could you guys take a look when you have time? Thanks! The test failure (fails due to an unknown error code, -9) seems

[GitHub] spark issue #23163: [SPARK-26164][SQL] Allow FileFormatWriter to write multi...

2018-11-28 Thread c21
Github user c21 commented on the issue: https://github.com/apache/spark/pull/23163 @gatorsmile: > Any perf number? From my employer company workload, we see >20% reserved CPU time (executor wall clock time) reduction, and >20% disk spill size reduction, after rol

[GitHub] spark issue #23163: [SPARK-26164][SQL] Allow FileFormatWriter to write multi...

2018-11-28 Thread c21
Github user c21 commented on the issue: https://github.com/apache/spark/pull/23163 cc people who have most context for review - @cloud-fan, @tejasapatil and @sameeragarwal. Thanks! --- - To unsubscribe, e-mail

[GitHub] spark pull request #23163: [SPARK-26164][SQL] Allow FileFormatWriter to writ...

2018-11-28 Thread c21
GitHub user c21 opened a pull request: https://github.com/apache/spark/pull/23163 [SPARK-26164][SQL] Allow FileFormatWriter to write multiple partitions/buckets without sort ## What changes were proposed in this pull request? Currently spark always requires a local sort