[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21911 **[Test build #94108 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94108/testReport)** for PR 21911 at commit [`739aeb4`](https://github.com/apache/spark/commit/73

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21981 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21981 **[Test build #94105 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94105/testReport)** for PR 21981 at commit [`c571279`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21981 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94105/ Test PASSed. ---

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21952 The regression happens at writing. Looks like when benchmarking writing time, we don't use `df.count`? --- - To unsubscribe, e-ma

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21965 **[Test build #94107 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94107/testReport)** for PR 21965 at commit [`ace19dd`](https://github.com/apache/spark/commit/ac

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21965 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21965 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1737/

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21952 I noticed that the benchmark uses `df.count`, is it possible that column pruning has some issues in master? --- - To unsubscri

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-02 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21965 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21948 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21948 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1736/

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21948 **[Test build #94106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94106/testReport)** for PR 21948 at commit [`86817c7`](https://github.com/apache/spark/commit/86

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21948 @rdblue I have documented the object reuse behavior and ask data source to handle it, please take a look, thanks! --- - To uns

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21898 > Also, if there shouldn't exist two active attempts at the same time for a barrier stage, maybe we should store attemptId as a state variable. Basically, if we see a new attempt ID, we should a

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/21981 BTW, @HyukjinKwon, do you know who's still reviewing the ML PRs? I have a few old PRs and I really want to know which're considered meaningful. --- -

[GitHub] spark issue #21979: [SPARK-25009][CORE]Standalone Cluster mode application s...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21979 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21979: [SPARK-25009][CORE]Standalone Cluster mode application s...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21979 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94082/ Test PASSed. ---

[GitHub] spark issue #21895: [SPARK-24948][SHS] Delegate check access permissions to ...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21895 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21895: [SPARK-24948][SHS] Delegate check access permissions to ...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21895 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94084/ Test PASSed. ---

[GitHub] spark issue #21979: [SPARK-25009][CORE]Standalone Cluster mode application s...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21979 **[Test build #94082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94082/testReport)** for PR 21979 at commit [`e753ff8`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21895: [SPARK-24948][SHS] Delegate check access permissions to ...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21895 **[Test build #94084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94084/testReport)** for PR 21895 at commit [`c620fff`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21952 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94104/ Test PASSed. ---

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21952 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21898 Also, if there shouldn't exist two active attempts at the same time for a barrier stage, maybe we should store attemptId as a state variable. Basically, if we see a new attempt ID, we should abort th

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21952 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21952 **[Test build #94104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94104/testReport)** for PR 21952 at commit [`ec17d58`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/21981 Thanks for the review @HyukjinKwon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21898 Here is what I mean: ~~~scala case class ContextBarrierId(stageId: Int, stageAttemptId: Int) class ContextBarrierState(val numTasks: Int) { private var epoch: Int = 0

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21981 **[Test build #94105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94105/testReport)** for PR 21981 at commit [`c571279`](https://github.com/apache/spark/commit/c5

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21981 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1735/

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21981 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21952 Ah, finally I can reproduce this. It needs to allocate the array feature with length 16000. I was reducing it to 1600 and it largely relieve the regression. `com.databricks.spark.avro` is faster only

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/21981 Ah, this triggers the doc check. Updating. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21965 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21965 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94088/ Test FAILed. ---

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21965 **[Test build #94088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94088/testReport)** for PR 21965 at commit [`ace19dd`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #21963: [SPARK-24997[SQL] Enable support of MINUS ALL

2018-08-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21963 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21952 we can keep investigating the perf regression, this patch itself LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.a

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21952 **[Test build #94104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94104/testReport)** for PR 21952 at commit [`ec17d58`](https://github.com/apache/spark/commit/ec

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21981 Looks we should fix the doc in PrefixSpan too. LGTM if the check pass. --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21965 cc @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21952 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94081/ Test PASSed. ---

[GitHub] spark issue #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21952 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1734/

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21898 **[Test build #94081 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94081/testReport)** for PR 21898 at commit [`33d4827`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toStr...

2018-08-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21964 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21752: [SPARK-24788][SQL] fixed UnresolvedException when...

2018-08-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21752 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/21941 Thanks a lot @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21586: [SPARK-24586][SQL] Upcast should not allow casting from ...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21586 **[Test build #94103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94103/testReport)** for PR 21586 at commit [`c89d12e`](https://github.com/apache/spark/commit/c8

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207442439 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,199 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207444103 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207443099 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207444701 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -39,6 +44,17 @@ class BarrierTaskContext( extends TaskContextImpl(stageI

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207444041 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207442661 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r20759 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21952#discussion_r207444698 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -151,11 +155,12 @@ class AvroSerializer(rootCatalystType: Dat

[GitHub] spark pull request #21974: [SPARK-25002][SQL] Avro: revise the output record...

2018-08-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21974 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-02 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/21860 cc @kiszk,Can you review it again if you have some time? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21952#discussion_r207444381 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -151,11 +155,12 @@ class AvroSerializer(rootCatalystType: DataTy

[GitHub] spark issue #21895: [SPARK-24948][SHS] Delegate check access permissions to ...

2018-08-02 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21895 Ping @mridulm , would you please also take a review, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21974: [SPARK-25002][SQL] Avro: revise the output record namesp...

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21974 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21952#discussion_r207444037 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -100,17 +100,20 @@ class AvroSerializer(rootCatalystType: DataTy

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21981 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1733/

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21981 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21981 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94102/ Test FAILed. ---

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21981 **[Test build #94102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94102/testReport)** for PR 21981 at commit [`6af3644`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21981 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21952#discussion_r207443421 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -151,11 +155,12 @@ class AvroSerializer(rootCatalystType: Dat

[GitHub] spark issue #21953: [SPARK-24992][Core] spark should randomize yarn local di...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21953 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21953: [SPARK-24992][Core] spark should randomize yarn local di...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21953 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94085/ Test FAILed. ---

[GitHub] spark pull request #21952: [SPARK-24993] [SQL] Make Avro Fast Again

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21952#discussion_r207443196 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -100,17 +100,20 @@ class AvroSerializer(rootCatalystType: Dat

[GitHub] spark issue #21953: [SPARK-24992][Core] spark should randomize yarn local di...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21953 **[Test build #94085 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94085/testReport)** for PR 21953 at commit [`a8c1654`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21981 **[Test build #94102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94102/testReport)** for PR 21981 at commit [`6af3644`](https://github.com/apache/spark/commit/6a

[GitHub] spark pull request #21981: [SAPRK-25011][ML]add prefix to __all__ in fpm.py

2018-08-02 Thread hhbyyh
GitHub user hhbyyh opened a pull request: https://github.com/apache/spark/pull/21981 [SAPRK-25011][ML]add prefix to __all__ in fpm.py ## What changes were proposed in this pull request? jira: https://issues.apache.org/jira/browse/SPARK-25011 add prefix to __all__ in

[GitHub] spark pull request #21980: [SPARK-25010][SQL] Rand/Randn should produce diff...

2018-08-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21980#discussion_r207442765 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/IncrementalExecution.scala --- @@ -75,14 +75,15 @@ class IncrementalExecution(

[GitHub] spark pull request #21941: [SPARK-24966][SQL] Implement precedence rules for...

2018-08-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21941 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21980: [SPARK-25010][SQL] Rand/Randn should produce diff...

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21980#discussion_r207441687 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/IncrementalExecution.scala --- @@ -75,14 +75,15 @@ class IncrementalExecution(

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21941 LGTM Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional com

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21966 **[Test build #94099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94099/testReport)** for PR 21966 at commit [`16b9949`](https://github.com/apache/spark/commit/16

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21933 **[Test build #94100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94100/testReport)** for PR 21933 at commit [`0251bd5`](https://github.com/apache/spark/commit/02

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21889 **[Test build #94101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94101/testReport)** for PR 21889 at commit [`37e0a97`](https://github.com/apache/spark/commit/37

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21966 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1732/

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21966 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-02 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/21889 > Anybody else able to reproduce this failure? It succeeded on my developer machine. It worked for me, too. Let's see what a retest does. --- -

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21889 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21977 cc @BryanCutler and @icexelloss too since we recently discussed about memory issue. --- - To unsubscribe, e-mail: reviews-un

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21933 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21933 **[Test build #94098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94098/testReport)** for PR 21933 at commit [`0251bd5`](https://github.com/apache/spark/commit/02

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21933 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94080/ Test FAILed. ---

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21933 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21933 **[Test build #94080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94080/testReport)** for PR 21933 at commit [`0251bd5`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21721 **[Test build #94097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94097/testReport)** for PR 21721 at commit [`1775c2a`](https://github.com/apache/spark/commit/17

[GitHub] spark issue #21975: [WIP][SPARK-25001][BUILD] Fix miscellaneous build warnin...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21975 **[Test build #94096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94096/testReport)** for PR 21975 at commit [`2354e10`](https://github.com/apache/spark/commit/23

[GitHub] spark issue #21975: [WIP][SPARK-25001][BUILD] Fix miscellaneous build warnin...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21975 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21975: [WIP][SPARK-25001][BUILD] Fix miscellaneous build warnin...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21975 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1731/

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21721 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21721 Yup, looks the resent Kafka upgrade has an issue. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addi

  1   2   3   4   5   6   7   8   9   10   >