[GitHub] spark pull request #21036: [SPARK-23958][CORE] HadoopRdd filters empty files...

2018-04-11 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/21036#discussion_r180652894 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -55,7 +56,8 @@ private[spark] class HadoopPartition(rddId: Int, override va

[GitHub] spark issue #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpark as a...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21007 **[Test build #89173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89173/testReport)** for PR 21007 at commit [`deacb17`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21034 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89175/ Test FAILed. ---

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #89172 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89172/testReport)** for PR 20345 at commit [`94d9171`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20871 **[Test build #89171 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89171/testReport)** for PR 20871 at commit [`a7665fd`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21034 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21037 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89176/ Test FAILed. ---

[GitHub] spark issue #21031: [SPARK-23923][SQL] Add cardinality function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21031 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20874 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89168/ Test FAILed. ---

[GitHub] spark issue #21031: [SPARK-23923][SQL] Add cardinality function

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21031 **[Test build #89169 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89169/testReport)** for PR 21031 at commit [`d405d8a`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpark as a...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21007 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89173/ Test FAILed. ---

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21037 **[Test build #89176 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89176/testReport)** for PR 21037 at commit [`535e511`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpark as a...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21007 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20874 **[Test build #89168 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89168/testReport)** for PR 20874 at commit [`9ef19df`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21034 **[Test build #89175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89175/testReport)** for PR 21034 at commit [`3a76d87`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20874 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21021 **[Test build #89170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89170/testReport)** for PR 21021 at commit [`0203920`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20871 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89171/ Test FAILed. ---

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89172/ Test FAILed. ---

[GitHub] spark issue #21031: [SPARK-23923][SQL] Add cardinality function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21031 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89169/ Test FAILed. ---

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20871 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89170/ Test FAILed. ---

[GitHub] spark pull request #21036: [SPARK-23958][CORE] HadoopRdd filters empty files...

2018-04-11 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/21036#discussion_r180655799 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -55,7 +56,8 @@ private[spark] class HadoopPartition(rddId: Int, override va

[GitHub] spark issue #21036: [SPARK-23958][CORE] HadoopRdd filters empty files to avo...

2018-04-11 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/21036 Thanks, I will try to add test cases. @felixcheung --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21038: [SPARK-22968][DStream] Fix Kafka connector partit...

2018-04-11 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/21038 [SPARK-22968][DStream] Fix Kafka connector partition revoked issue ## What changes were proposed in this pull request? Kafka partitions can be revoked when new consumers joined in the con

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2018-04-11 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15770 @wangmiao1981 If you're busy I can help take over this. -:) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.o

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka connector partition rev...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21038 **[Test build #89177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89177/testReport)** for PR 21038 at commit [`f317dec`](https://github.com/apache/spark/commit/f3

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #89178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89178/testReport)** for PR 20345 at commit [`94d9171`](https://github.com/apache/spark/commit/94

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka connector partition rev...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21038 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka connector partition rev...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2197/ Tes

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2198/ Tes

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21038 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89177/ Test PASSed. ---

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21038 **[Test build #89177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89177/testReport)** for PR 21038 at commit [`f317dec`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-04-11 Thread tiboun
Github user tiboun commented on the issue: https://github.com/apache/spark/pull/21014 Ah yes, correct Vanzin, I didn't think about that use case. So I will apply shellEscape for both in this PR. --- - To unsubscribe

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-11 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21021 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-11 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180664561 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -274,3 +279,7 @@ abstract class BinaryNode exte

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-11 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180664594 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala --- @@ -22,10 +22,11 @@ import org.apache.spark.sql.{execution, Row}

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21021 **[Test build #89179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89179/testReport)** for PR 21021 at commit [`0203920`](https://github.com/apache/spark/commit/02

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-11 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180664857 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -733,6 +735,17 @@ object EliminateSorts extends Rule[

[GitHub] spark issue #21009: [SPARK-23905][SQL] Add UDF weekday

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21009 **[Test build #89181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89181/testReport)** for PR 21009 at commit [`79beb00`](https://github.com/apache/spark/commit/79

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21014 **[Test build #89180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89180/testReport)** for PR 21014 at commit [`4732c4b`](https://github.com/apache/spark/commit/47

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2199/ Tes

[GitHub] spark issue #21019: [SPARK-23948] Trigger mapstage's job listener in submitM...

2018-04-11 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/21019 @squito @cloud-fan How do you think this change ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For a

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21014 **[Test build #89180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89180/testReport)** for PR 21014 at commit [`4732c4b`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21014 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89180/ Test PASSed. ---

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21014 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpark as a...

2018-04-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21007 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpark as a...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21007 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2200/ Tes

[GitHub] spark issue #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpark as a...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21007 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpark as a...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21007 **[Test build #89182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89182/testReport)** for PR 21007 at commit [`deacb17`](https://github.com/apache/spark/commit/de

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread Hackeruncle
Github user Hackeruncle commented on the issue: https://github.com/apache/spark/pull/21038 @jerryshao Thank you very much for this issue. I go to compile and test. --- - To unsubscribe, e-mail: reviews-unsubscr..

[GitHub] spark pull request #20983: [SPARK-23747][Structured Streaming] Add EpochCoor...

2018-04-11 Thread efimpoberezkin
Github user efimpoberezkin commented on a diff in the pull request: https://github.com/apache/spark/pull/20983#discussion_r180679855 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/EpochCoordinatorSuite.scala --- @@ -0,0 +1,225 @@ +/* + * Licens

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89178/ Test FAILed. ---

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #89178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89178/testReport)** for PR 20345 at commit [`94d9171`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #20981: [SPARK-23873][SQL] Use accessors in interpreted L...

2018-04-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20981#discussion_r180684792 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/InternalRow.scala --- @@ -119,4 +119,28 @@ object InternalRow { case v: MapData

[GitHub] spark pull request #20981: [SPARK-23873][SQL] Use accessors in interpreted L...

2018-04-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20981#discussion_r180685351 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/InternalRow.scala --- @@ -119,4 +119,28 @@ object InternalRow { case v: MapD

[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...

2018-04-11 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21034#discussion_r180686345 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -212,6 +213,96 @@ case class SortArray(b

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21034 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21034 **[Test build #89183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89183/testReport)** for PR 21034 at commit [`3a76d87`](https://github.com/apache/spark/commit/3a

[GitHub] spark pull request #20888: [SPARK-23775][TEST] DataFrameRangeSuite should wa...

2018-04-11 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20888#discussion_r180691534 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameRangeSuite.scala --- @@ -164,10 +164,13 @@ class DataFrameRangeSuite extends QueryTest

[GitHub] spark issue #20888: [SPARK-23775][TEST] DataFrameRangeSuite should wait for ...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20888 **[Test build #89184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89184/testReport)** for PR 20888 at commit [`4e7be70`](https://github.com/apache/spark/commit/4e

[GitHub] spark issue #20888: [SPARK-23775][TEST] Make DataFrameRangeSuite not flaky

2018-04-11 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20888 Updated the title and the description to reflect the changes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apach

[GitHub] spark pull request #21039: [SPARK-23960][SQL][MINOR] Mark HashAggregateExec....

2018-04-11 Thread rednaxelafx
GitHub user rednaxelafx opened a pull request: https://github.com/apache/spark/pull/21039 [SPARK-23960][SQL][MINOR] Mark HashAggregateExec.bufVars as transient ## What changes were proposed in this pull request? Mark `HashAggregateExec.bufVars` as transient to avoid it from

[GitHub] spark issue #21039: [SPARK-23960][SQL][MINOR] Mark HashAggregateExec.bufVars...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21039 **[Test build #89185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89185/testReport)** for PR 21039 at commit [`0b5a075`](https://github.com/apache/spark/commit/0b

[GitHub] spark issue #21039: [SPARK-23960][SQL][MINOR] Mark HashAggregateExec.bufVars...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21039 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2201/ Tes

[GitHub] spark issue #21039: [SPARK-23960][SQL][MINOR] Mark HashAggregateExec.bufVars...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21039 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21039: [SPARK-23960][SQL][MINOR] Mark HashAggregateExec.bufVars...

2018-04-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/21039 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.ap

[GitHub] spark pull request #21029: [WIP][SPARK-23952] remove type parameter in DataR...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21029#discussion_r180705219 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExec.scala --- @@ -95,21 +77,29 @@ case class DataSourceV

[GitHub] spark pull request #21029: [WIP][SPARK-23952] remove type parameter in DataR...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21029#discussion_r180705366 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala --- @@ -143,7 +144,7 @@ case class MemoryStream[A : Encoder](id:

[GitHub] spark pull request #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpa...

2018-04-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21007#discussion_r180705768 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TestQueryExecutionListener.scala --- @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #20981: [SPARK-23873][SQL] Use accessors in interpreted LambdaVa...

2018-04-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20981 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.ap

[GitHub] spark issue #20981: [SPARK-23873][SQL] Use accessors in interpreted LambdaVa...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20981 **[Test build #89186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89186/testReport)** for PR 20981 at commit [`5711ea1`](https://github.com/apache/spark/commit/57

[GitHub] spark issue #20981: [SPARK-23873][SQL] Use accessors in interpreted LambdaVa...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20981 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2202/ Tes

[GitHub] spark issue #20981: [SPARK-23873][SQL] Use accessors in interpreted LambdaVa...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20981 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpa...

2018-04-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21007#discussion_r180710305 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TestQueryExecutionListener.scala --- @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark issue #21019: [SPARK-23948] Trigger mapstage's job listener in submitM...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21019 cc @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #20931: [SPARK-23815][Core]Spark writer dynamic partition overwr...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20931 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #20931: [SPARK-23815][Core]Spark writer dynamic partition overwr...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20931 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark issue #20931: [SPARK-23815][Core]Spark writer dynamic partition overwr...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20931 **[Test build #89187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89187/testReport)** for PR 20931 at commit [`08b9601`](https://github.com/apache/spark/commit/08

[GitHub] spark pull request #19868: [SPARK-22676] Avoid iterating all partition paths...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19868#discussion_r180713699 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala --- @@ -176,12 +176,13 @@ class HadoopTableReader( val ma

[GitHub] spark pull request #20535: [SPARK-23341][SQL] define some standard options f...

2018-04-11 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r180714763 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/DataSourceOptions.java --- @@ -17,16 +17,61 @@ package org.apache.spark.s

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21034 **[Test build #89183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89183/testReport)** for PR 21034 at commit [`3a76d87`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180714835 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -274,3 +279,7 @@ abstract class BinaryNode ext

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21034 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21034 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89183/ Test FAILed. ---

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180715118 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -733,6 +735,17 @@ object EliminateSorts extends Rule

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20871 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21037 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20874 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #21031: [SPARK-23923][SQL] Add cardinality function

2018-04-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21031 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180716049 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala --- @@ -169,4 +169,6 @@ case class InMemoryRelation(

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180716173 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala --- @@ -22,10 +22,11 @@ import org.apache.spark.sql.{execution, Row}

  1   2   3   4   5   6   >