[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #85203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85203/testReport)** for PR 19884 at commit [`715f83d`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r158112470 --- Diff: python/pyspark/sql/tests.py --- @@ -3356,6 +3356,7 @@ def test_schema_conversion_roundtrip(self): self.assertEquals(self.schema, s

[GitHub] spark pull request #20037: [SPARK-22849] ivy.retrieve pattern should also co...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20037 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML expo...

2017-12-20 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r158113236 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -994,6 +998,38 @@ class LinearRegressionSuite L

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20037 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158113948 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -104,6 +103,60 @@ object SparkHiveExample {

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #85204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85204/testReport)** for PR 19876 at commit [`6e9cdc3`](https://github.com/apache/spark/commit/6e

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20016 @srowen I think, we can merge this now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/19993 To make it available for other classes, we need to support checking for both `fit` and `transform`, that means we also need a sample input Dataset, so we may have to add the explicit test in each of

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #85204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85204/testReport)** for PR 19876 at commit [`6e9cdc3`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85204/ Test FAILed. ---

[GitHub] spark pull request #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML expo...

2017-12-20 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r158114844 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #85205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85205/testReport)** for PR 19884 at commit [`faa9f09`](https://github.com/apache/spark/commit/fa

[GitHub] spark issue #20035: [SPARK-22848][SQL] Eliminate mutable state from Stack

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20035 **[Test build #85198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85198/testReport)** for PR 20035 at commit [`f0163e7`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20035: [SPARK-22848][SQL] Eliminate mutable state from Stack

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20035 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20035: [SPARK-22848][SQL] Eliminate mutable state from Stack

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20035 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85198/ Test FAILed. ---

[GitHub] spark pull request #20002: [SPARK-22465][Core][WIP] Add a safety-check to RD...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/20002#discussion_r158119432 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -67,6 +71,16 @@ object Partitioner { } } } + + /*

[GitHub] spark pull request #20002: [SPARK-22465][Core][WIP] Add a safety-check to RD...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/20002#discussion_r158108582 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -57,7 +60,8 @@ object Partitioner { def defaultPartitioner(rdd: RDD[_], others

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20037 **[Test build #85197 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85197/testReport)** for PR 20037 at commit [`331ba33`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20037 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85197/ Test FAILed. ---

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158124569 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -165,13 +165,23 @@ public ManagedB

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158123441 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -812,10 +812,10 @@ private[spark] object MapOutputTracker extends Logging {

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158124746 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleIndexInformation.java --- @@ -59,9 +59,9 @@ public int getSize() {

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158121392 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -165,13 +165,23 @@ public ManagedB

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158120856 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -203,22 +203,23 @@ private ShuffleM

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158124202 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -203,22 +203,23 @@ private ShuffleM

[GitHub] spark pull request #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML expo...

2017-12-20 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r158126871 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #20039: [SPARK-22850][core] Ensure queued events are deli...

2017-12-20 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/20039 [SPARK-22850][core] Ensure queued events are delivered to all event queues. The code in LiveListenerBus was queueing events before start in the queues themselves; so in situations like the follow

[GitHub] spark issue #20039: [SPARK-22850][core] Ensure queued events are delivered t...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20039 **[Test build #85206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85206/testReport)** for PR 20039 at commit [`80b900a`](https://github.com/apache/spark/commit/80

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158129065 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -217,7 +21

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #85207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85207/testReport)** for PR 19876 at commit [`b8844c7`](https://github.com/apache/spark/commit/b8

[GitHub] spark issue #20029: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20029 This indeed is the primary change as it's open vs master. https://github.com/apache/spark/pull/19989 had some concerns about whether this affects correctness though? --- --

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158131312 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -217,7 +217

[GitHub] spark pull request #19992: [SPARK-22805][CORE] Use StorageLevel aliases in e...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19992#discussion_r158131498 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -2022,12 +1947,7 @@ private[spark] object JsonProtocolSuite extends Asser

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20002 **[Test build #85200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85200/testReport)** for PR 20002 at commit [`ca6aa08`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20002 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20032 **[Test build #85208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85208/testReport)** for PR 20032 at commit [`4adb04b`](https://github.com/apache/spark/commit/4a

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20002 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85200/ Test FAILed. ---

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20016 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20016 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158133877 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -102,8 +101,63 @@ object SparkHiveExample { // |

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158133606 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -102,8 +101,63 @@ object SparkHiveExample { // |

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158134032 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -102,8 +101,63 @@ object SparkHiveExample { // |

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20032 **[Test build #85208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85208/testReport)** for PR 20032 at commit [`4adb04b`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20032 Within the allocator's control loop, it's all asynchronous requests being made for executor pods from the k8s API, so, each loop doesn't take very long. If a user were to set a very low value for the

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20032 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20032 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85208/ Test PASSed. ---

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20032 > (And what happens if set to 0/-ve ?) We have a check preventing that in the option itself. The value should be strictly greater than 0 ms. --- ---

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20023 In am generally in favor of following the SQL standard. How about we do this. Let's make the standard behavior the default, and add a flag to revert to the old behavior. This allows us to ease us

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19993 **[Test build #85209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85209/testReport)** for PR 19993 at commit [`9872bfd`](https://github.com/apache/spark/commit/98

[GitHub] spark pull request #19156: [SPARK-19634][SQL][ML][FOLLOW-UP] Improve interfa...

2017-12-20 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19156#discussion_r158138431 --- Diff: mllib/src/test/scala/org/apache/spark/ml/stat/SummarizerSuite.scala --- @@ -35,237 +34,252 @@ class SummarizerSuite extends SparkFunSuite with

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19993 thanks @hhbyyh, I updated the PR according to your suggestion and previous comments. --- - To unsubscribe, e-mail: reviews-unsu

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19940 **[Test build #85202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85202/testReport)** for PR 19940 at commit [`c650196`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19940 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19940 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85202/ Test FAILed. ---

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158116796 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala --- @@ -0,0 +1,195 @@ +/* + * Licensed

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158135254 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/* +

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158138974 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/* +

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158134853 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/* +

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158136868 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/* +

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158139103 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/* +

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158116511 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala --- @@ -0,0 +1,195 @@ +/* + * Licensed

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158121770 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + * Licen

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158135301 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/* +

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158122336 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + * Licen

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158120930 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + * Licen

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158120028 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + * Licen

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158119082 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -266,6 +266,21 @@ class HDFSMetadataLog[T <: AnyRef

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158121081 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + * Licen

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158117033 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -303,7 +299,7 @@ abstract class StreamExecution(

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158137370 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/* +

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20038 **[Test build #85201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85201/testReport)** for PR 20038 at commit [`52641db`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20038 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20038 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85201/ Test FAILed. ---

[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19994 **[Test build #85210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85210/testReport)** for PR 19994 at commit [`e02d60f`](https://github.com/apache/spark/commit/e0

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #85207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85207/testReport)** for PR 19876 at commit [`b8844c7`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85207/ Test PASSed. ---

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 thanks for looking at this @hvanhovell. The reasons why I didn't introduce a configuration variable for this behavior are: 1. As far as I know, currently there is no way to read reliably a

[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19994 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85210/ Test PASSed. ---

[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19994 **[Test build #85210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85210/testReport)** for PR 19994 at commit [`e02d60f`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158147697 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -86,7 +86,7

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158148895 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -86,7 +86,7

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158149295 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -86,7 +86,7

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19993 **[Test build #85209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85209/testReport)** for PR 19993 at commit [`9872bfd`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19993 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85209/ Test PASSed. ---

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19993 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20038 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark issue #20033: [SPARK-22847] [CORE] Remove redundant code in AppStatusL...

2017-12-20 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20033 Pyspark failures are unrelated. Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addition

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20038 **[Test build #85211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85211/testReport)** for PR 20038 at commit [`52641db`](https://github.com/apache/spark/commit/52

[GitHub] spark pull request #20033: [SPARK-22847] [CORE] Remove redundant code in App...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20033 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19940 **[Test build #85212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85212/testReport)** for PR 19940 at commit [`6143b22`](https://github.com/apache/spark/commit/61

[GitHub] spark pull request #20040: [SPARK-22852][BUILD] Exclude -Xlint:unchecked fro...

2017-12-20 Thread easel
GitHub user easel opened a pull request: https://github.com/apache/spark/pull/20040 [SPARK-22852][BUILD] Exclude -Xlint:unchecked from sbt javadoc flags ## What changes were proposed in this pull request? Moves the -Xlint:unchecked flag in the sbt build configuration from Co

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r158154036 --- Diff: sql/core/src/test/resources/sql-tests/inputs/decimals.sql --- @@ -0,0 +1,16 @@ +-- tests for decimals handling in operations +-- Spar

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20023 I don't fully agree :)... 1. You can use `SQLConf.get` for this. Or you can wire up the rules using the `SessionStateBuilder`. 2. I am reluctant to change this for a minor version. I

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r158154378 --- Diff: sql/core/src/test/resources/sql-tests/inputs/decimals.sql --- @@ -0,0 +1,16 @@ +-- tests for decimals handling in operations +-- Spar

<    1   2   3   4   5   6   7   >