[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21859 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21859 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94931/ Test PASSed. ---

[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21859 **[Test build #94931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94931/testReport)** for PR 21859 at commit

[GitHub] spark pull request #22135: [SPARK-25093][SQL] Avoid recompiling regexp for c...

2018-08-18 Thread igreenfield
Github user igreenfield commented on a diff in the pull request: https://github.com/apache/spark/pull/22135#discussion_r211091975 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala --- @@ -91,10 +94,7 @@ object

[GitHub] spark issue #22140: [SPARK-25072][PySpark] Forbid extra value for custom Row

2018-08-18 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22140 cc @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #94932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94932/testReport)** for PR 22141 at commit

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r211089742 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala --- @@ -366,6 +366,43 @@ class AggregateBenchmark

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21931 Minor comments. LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r211089705 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala --- @@ -366,6 +366,43 @@ class AggregateBenchmark

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r211089695 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1437,6 +1437,16 @@ object SQLConf { .intConf

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r211089716 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala --- @@ -366,6 +366,43 @@ class AggregateBenchmark

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-18 Thread cclauss
Github user cclauss commented on the issue: https://github.com/apache/spark/pull/20838 This is not working at all... I am wasting way too much time. 5+ months and 80+ comments for 12 lines of code is I do not have the skills to solve the following undefined name 'long' in

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21931 LGTM, cc @cloud-fan @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21859 **[Test build #94931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94931/testReport)** for PR 21859 at commit

[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21859 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94930/ Test FAILed. ---

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #94930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94930/testReport)** for PR 22141 at commit

[GitHub] spark pull request #22131: [SPARK-25141][SQL][TEST] Modify tests for higher-...

2018-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22131 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22131: [SPARK-25141][SQL][TEST] Modify tests for higher-order f...

2018-08-18 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22131 Thanks! I'd use this one. merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread dmateusp
Github user dmateusp commented on the issue: https://github.com/apache/spark/pull/22141 I reproduced the issue with the following code (was a bit surprised with the behavior) The tables: ```scala scala> spark.sql("SELECT * FROM users").show +---+---+ |

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21899 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21899 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94929/ Test PASSed. ---

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21899 **[Test build #94929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94929/testReport)** for PR 21899 at commit

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #94930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94930/testReport)** for PR 22141 at commit

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22141: [SPARK-25154] Support NOT IN sub-queries inside n...

2018-08-18 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22141 [SPARK-25154] Support NOT IN sub-queries inside nested OR conditions. ## What changes were proposed in this pull request? Currently NOT IN subqueries (predicated null aware subquery) are

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-08-18 Thread witten
Github user witten commented on the issue: https://github.com/apache/spark/pull/21669 I see that this branch currently has merge conflicts, but any idea on when this might land? This is the last feature we're waiting for in order to switch from the abandoned [apache-spark-on-k8s

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94923/ Test FAILed. ---

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94923/testReport)** for PR 22112 at commit

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20838 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94927/ Test FAILed. ---

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20838 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22135 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94926/ Test PASSed. ---

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22135 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20838 **[Test build #94927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94927/testReport)** for PR 20838 at commit

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22135 **[Test build #94926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94926/testReport)** for PR 22135 at commit

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22078 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22078 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94925/ Test PASSed. ---

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22078 **[Test build #94925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94925/testReport)** for PR 22078 at commit

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22078 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94924/ Test PASSed. ---

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22078 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22078 **[Test build #94924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94924/testReport)** for PR 22078 at commit

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21899 **[Test build #94929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94929/testReport)** for PR 21899 at commit

[GitHub] spark issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during br...

2018-08-18 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21899 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22123: [SPARK-25134][SQL] Csv column pruning with checki...

2018-08-18 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22123#discussion_r211081732 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1603,6 +1603,25 @@ class CSVSuite extends

[GitHub] spark issue #22123: [SPARK-25134][SQL] Csv column pruning with checking of h...

2018-08-18 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22123 May I ask you check the `multiLine` mode additionally since we use different methods of uniVocity parser. When `multiLine` is disabled, the `parseLine` method is used but in the `multiLine` mode:

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94928/ Test PASSed. ---

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22121 **[Test build #94928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94928/testReport)** for PR 22121 at commit

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21909 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22121 **[Test build #94928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94928/testReport)** for PR 22121 at commit

[GitHub] spark issue #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22121 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22121: [SPARK-25133][SQL][Doc]Avro data source guide

2018-08-18 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211081684 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,260 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- +

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21909 LGTM. Thanks for being patient to address all the comments! Merged to master. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21909 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21909 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94922/ Test PASSed. ---

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21909 **[Test build #94922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94922/testReport)** for PR 21909 at commit

[GitHub] spark issue #22123: [SPARK-25134][SQL] Csv column pruning with checking of h...

2018-08-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22123 cc @MaxGekk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21087: [SPARK-23997][SQL] Configurable maximum number of...

2018-08-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21087#discussion_r211080067 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -164,9 +165,12 @@ case class BucketSpec(

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21860 cc @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20838 **[Test build #94927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94927/testReport)** for PR 20838 at commit

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22135 **[Test build #94926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94926/testReport)** for PR 22135 at commit

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22135 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22135 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22124 Comes from: [[SPARK-22834][SQL] Make insertion commands have real children to fix UI issues](https://github.com/apache/spark/pull/20020). ---

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22078 **[Test build #94925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94925/testReport)** for PR 22078 at commit

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22078 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22078 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22124 > But it is inconsistent now. Can you point out in the codebase where the inconsistency comes from? --- - To

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22112 I've removed the concept of "order sensitive partitioner" and came up with a better abstraction. Please take a look at the updated PR descrption, thanks! ---

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22078 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22078 **[Test build #94924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94924/testReport)** for PR 22078 at commit

[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22078 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22112 @tgravescs The `FileCommitProtocol` is an internal API, and our current implementation does store task-level data temporary in a staging directory (See `HadoopMapReduceCommitProtocol`). That

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #94923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94923/testReport)** for PR 22112 at commit

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94921/ Test PASSed. ---

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22124 **[Test build #94921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94921/testReport)** for PR 22124 at commit

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21909 **[Test build #94922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94922/testReport)** for PR 21909 at commit

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-18 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r211075385 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -223,7 +224,8 @@ object

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-18 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r211075384 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1492,6 +1492,15 @@ object SQLConf { "This usually

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-18 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22112 I can't envision how that would work? You can't change how output committers work. You would have to not store anything until all pass or store it temporarily, both in my opinion are not good.

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-18 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22135 thanks for the comment @kiszk , I am doing it! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22124 The root project should be consistent with the schema of the target table. But now it is inconsistent. **Before this PR**:

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22124 **[Test build #94921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94921/testReport)** for PR 22124 at commit

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-18 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/21860 cc @cloud-fan @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21819: [SPARK-24863][SS] Report Kafka offset lag as a cu...

2018-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21819 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22098: [SPARK-24886][INFRA] Fix the testing script to in...

2018-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22098 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21819: [SPARK-24863][SS] Report Kafka offset lag as a custom me...

2018-08-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21819 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-08-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22098 Let me just push this in. Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22132: [SPARK-25142][PYSPARK] Add error messages when Py...

2018-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22132 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

  1   2   >