[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2017-12-20 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r158170154 --- Diff: mllib/src/test/scala/org/apache/spark/ml/param/ParamsSuite.scala --- @@ -430,4 +433,49 @@ object ParamsSuite extends SparkFunSuite {

[GitHub] spark issue #19946: [SPARK-22648] [Scheduler] Spark on Kubernetes - Document...

2017-12-20 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/19946 Makes sense, will be sure to categorize appropriately moving forward. Thanks! On Dec 20, 2017 4:16 PM, "Marcelo Vanzin" wrote: BTW can I ask you guys

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20032 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19946: [SPARK-22648] [Scheduler] Spark on Kubernetes - Document...

2017-12-20 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19946 BTW can I ask you guys to start using "K8S" in the PR titles? These are not really scheduler changes. --- - To unsubscribe,

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20032 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19984 **[Test build #85218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85218/testReport)** for PR 19984 at commit

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158168099 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainerSuite.scala --- @@ -0,0 +1,91 @@ +/*

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167741 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/BasicInitContainerConfigurationStepSuite.scala

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167643 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/DriverMountSecretsStepSuite.scala --- @@ -0,0 +1,47 @@

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158166901 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala --- @@ -0,0 +1,119 @@ +/* +

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158168059 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/InitContainerMountSecretsStepSuite.scala ---

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167023 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala --- @@ -0,0 +1,119 @@ +/* +

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158165498 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DependencyResolutionStep.scala --- @@ -31,21 +32,23 @@

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167807 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/BasicInitContainerConfigurationStepSuite.scala

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167975 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/InitContainerConfigOrchestratorSuite.scala

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167791 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/BasicInitContainerConfigurationStepSuite.scala

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167537 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/DriverInitContainerBootstrapStepSuite.scala --- @@ -0,0

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167923 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/DriverMountSecretsStepSuite.scala --- @@ -0,0 +1,47 @@

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158165025 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/MountSecretsBootstrap.scala --- @@ -0,0 +1,63 @@ +/* + *

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158166916 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala --- @@ -0,0 +1,119 @@ +/* +

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158168037 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/InitContainerConfigOrchestratorSuite.scala

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158168278 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainerSuite.scala --- @@ -0,0 +1,91 @@ +/*

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158167623 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/DriverMountSecretsStepSuite.scala --- @@ -0,0 +1,47 @@

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158164898 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -14,13 +14,49 @@ * See the

[GitHub] spark pull request #19946: [SPARK-22648] [Scheduler] Spark on Kubernetes - D...

2017-12-20 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19946#discussion_r158168783 --- Diff: docs/running-on-kubernetes.md --- @@ -0,0 +1,573 @@ +--- +layout: global +title: Running Spark on Kubernetes +--- +* This

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20023 I thought adding more cases into `decimals.sql`. :) But, for now, never mind about that~ --- - To unsubscribe, e-mail:

[GitHub] spark issue #20039: [SPARK-22850][core] Ensure queued events are delivered t...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20039 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85206/ Test FAILed. ---

[GitHub] spark issue #20039: [SPARK-22850][core] Ensure queued events are delivered t...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20039 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20039: [SPARK-22850][core] Ensure queued events are delivered t...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20039 **[Test build #85206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85206/testReport)** for PR 20039 at commit

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-20 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 @kiszk Yea, I have now checked through the codegen. I didn't find the places that can cause that issue (`a + 1` as the codegen output value) yet. I may submit another PR to let us easily identify

[GitHub] spark issue #20021: [SPARK-22668][SQL] Ensure no global variables in argumen...

2017-12-20 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20021 Ok. I'm fine with only checking it in tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20008: [SPARK-22822][TEST] Basic tests for WindowFrameCoercion ...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20008 **[Test build #85217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85217/testReport)** for PR 20008 at commit

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 Thank you for your review @dongjoon-hyun. I think what we can do is add more test to the whitelist in `HiveCompatibilitySuite`, updating them according to HIVE-15331. Were you thinking to this or

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19984 **[Test build #85216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85216/testReport)** for PR 19984 at commit

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158159270 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* +

[GitHub] spark issue #20008: [SPARK-22822][TEST] Basic tests for WindowFrameCoercion ...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20008 **[Test build #85215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85215/testReport)** for PR 20008 at commit

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158158855 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2017-12-20 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r158158375 --- Diff: mllib/src/test/scala/org/apache/spark/ml/param/ParamsSuite.scala --- @@ -430,4 +433,49 @@ object ParamsSuite extends SparkFunSuite {

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20032 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20032 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85214/ Test PASSed. ---

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20032 **[Test build #85214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85214/testReport)** for PR 20032 at commit

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2017-12-20 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r158157137 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -1526,15 +1526,15 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 @hvanhovell, as far as 1 is regarded, I was referring to [this comment](https://github.com/apache/spark/pull/19449#pullrequestreview-67789784) and [this

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158156114 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* +

[GitHub] spark issue #20040: [SPARK-22852][BUILD] Exclude -Xlint:unchecked from sbt j...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20040 **[Test build #85213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85213/testReport)** for PR 20040 at commit

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20032 **[Test build #85214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85214/testReport)** for PR 20032 at commit

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20023 Thank you for pining me, @mgaido91 . The approach of PR looks good to me. BTW, do we need to borrow more Hive 2.2 test cases? ---

[GitHub] spark issue #20040: [SPARK-22852][BUILD] Exclude -Xlint:unchecked from sbt j...

2017-12-20 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20040 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85205/ Test FAILed. ---

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #85205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85205/testReport)** for PR 19884 at commit

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r158155039 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -1526,15 +1526,15 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20032 Fixed, thanks for the reviews. @mridulm, the questions about performance are super helpful for us also to keep track of how we're doing wrt other cluster managers and wrt user expectations. Thanks!

[GitHub] spark pull request #19992: [SPARK-22805][CORE] Use StorageLevel aliases in e...

2017-12-20 Thread superbobry
Github user superbobry commented on a diff in the pull request: https://github.com/apache/spark/pull/19992#discussion_r158154921 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -2022,12 +1947,7 @@ private[spark] object JsonProtocolSuite extends

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2017-12-20 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r158154050 --- Diff: mllib/src/test/scala/org/apache/spark/ml/param/ParamsSuite.scala --- @@ -430,4 +433,49 @@ object ParamsSuite extends SparkFunSuite {

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158154833 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -86,7

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2017-12-20 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r158153277 --- Diff: mllib/src/test/scala/org/apache/spark/ml/param/ParamsSuite.scala --- @@ -430,4 +433,49 @@ object ParamsSuite extends SparkFunSuite {

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2017-12-20 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r158153048 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -249,6 +250,31 @@ object ParamValidators { def

[GitHub] spark issue #20040: [SPARK-22852][BUILD] Exclude -Xlint:unchecked from sbt j...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20040 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r158154378 --- Diff: sql/core/src/test/resources/sql-tests/inputs/decimals.sql --- @@ -0,0 +1,16 @@ +-- tests for decimals handling in operations +--

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20023 I don't fully agree :)... 1. You can use `SQLConf.get` for this. Or you can wire up the rules using the `SessionStateBuilder`. 2. I am reluctant to change this for a minor version. I

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r158154036 --- Diff: sql/core/src/test/resources/sql-tests/inputs/decimals.sql --- @@ -0,0 +1,16 @@ +-- tests for decimals handling in operations +--

[GitHub] spark pull request #20040: [SPARK-22852][BUILD] Exclude -Xlint:unchecked fro...

2017-12-20 Thread easel
GitHub user easel opened a pull request: https://github.com/apache/spark/pull/20040 [SPARK-22852][BUILD] Exclude -Xlint:unchecked from sbt javadoc flags ## What changes were proposed in this pull request? Moves the -Xlint:unchecked flag in the sbt build configuration from

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19940 **[Test build #85212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85212/testReport)** for PR 19940 at commit

[GitHub] spark pull request #20033: [SPARK-22847] [CORE] Remove redundant code in App...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20033 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20038 **[Test build #85211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85211/testReport)** for PR 20038 at commit

[GitHub] spark issue #20033: [SPARK-22847] [CORE] Remove redundant code in AppStatusL...

2017-12-20 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20033 Pyspark failures are unrelated. Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20038 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19993 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19993 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85209/ Test PASSed. ---

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19993 **[Test build #85209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85209/testReport)** for PR 19993 at commit

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158149295 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -86,7

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158148895 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -86,7

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158147697 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -86,7

[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19994 **[Test build #85210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85210/testReport)** for PR 19994 at commit

[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85210/ Test PASSed. ---

[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19994 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 thanks for looking at this @hvanhovell. The reasons why I didn't introduce a configuration variable for this behavior are: 1. As far as I know, currently there is no way to read reliably

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85207/ Test PASSed. ---

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #85207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85207/testReport)** for PR 19876 at commit

[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19994 **[Test build #85210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85210/testReport)** for PR 19994 at commit

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20038 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85201/ Test FAILed. ---

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20038 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20038 **[Test build #85201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85201/testReport)** for PR 20038 at commit

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158117033 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -303,7 +299,7 @@ abstract class StreamExecution(

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158137370 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/*

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158120930 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + *

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158120028 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + *

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158119082 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -266,6 +266,21 @@ class HDFSMetadataLog[T <:

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158121081 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + *

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158135301 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/*

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158122336 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + *

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158136868 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/*

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158139103 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/*

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158116511 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala --- @@ -0,0 +1,195 @@ +/* + *

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158121770 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -0,0 +1,343 @@ +/* + *

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158138974 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/*

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158134853 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/*

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158135254 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDDIter.scala --- @@ -0,0 +1,205 @@ +/*

<    1   2   3   4   5   6   7   >