[GitHub] spark pull request #18118: [SPARK-20199][ML] : Provided featureSubsetStrateg...

2017-11-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/18118#discussion_r148564380 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/GBTClassifierSuite.scala --- @@ -354,6 +356,41 @@ class GBTClassifierSuite extends

[GitHub] spark pull request #19623: [SPARK-22078][SQL] clarify exception behaviors fo...

2017-11-02 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/19623#discussion_r148595625 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -50,28 +53,34 @@ /** *

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18118 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18118 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83348/ Test FAILed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83335/ Test FAILed. ---

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-02 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19631 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19644: [SPARK-22306][SQL] alter table schema should not erase t...

2017-11-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19644 `checkFieldNames` is just a small cleanup, I think we can keep it in master. --- - To unsubscribe, e-mail:

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18118 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83350/ Test FAILed. ---

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18118 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19623: [SPARK-22078][SQL] clarify exception behaviors for all d...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19623 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-11-02 Thread pralabhkumar
Github user pralabhkumar commented on the issue: https://github.com/apache/spark/pull/18118 @sethah Have done the changes as suggested ,but build is failing because of this error Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?

[GitHub] spark pull request #17644: [SPARK-17729] [SQL] Enable creating hive bucketed...

2017-11-02 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/17644#discussion_r148566192 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogSuite.scala --- @@ -247,7 +247,7 @@ abstract class

[GitHub] spark pull request #19623: [SPARK-22078][SQL] clarify exception behaviors fo...

2017-11-02 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/19623#discussion_r148596100 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -50,28 +53,34 @@ /** *

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19631 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19631 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83349/ Test FAILed. ---

[GitHub] spark pull request #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior ...

2017-11-02 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19607#discussion_r148601935 --- Diff: python/pyspark/serializers.py --- @@ -274,12 +278,13 @@ def load_stream(self, stream): """ Deserialize

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19433 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83353/ Test FAILed. ---

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18118 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83354/ Test FAILed. ---

[GitHub] spark issue #19635: [SPARK-22413][SQL] Type coercion for IN is not coherent ...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19635 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19631 **[Test build #83352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83352/testReport)** for PR 19631 at commit

[GitHub] spark issue #19635: [SPARK-22413][SQL] Type coercion for IN is not coherent ...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19635 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83351/ Test FAILed. ---

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18118 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19433 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19644: [SPARK-22306][SQL] alter table schema should not ...

2017-11-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19644#discussion_r148575508 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -857,19 +857,23 @@ object DDLUtils { } }

[GitHub] spark issue #19552: [SPARK-22329][SQL] Use NEVER_INFER for `spark.sql.hive.c...

2017-11-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19552 I close this since #19622 is merged. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19636: [SPARK-22416][SQL] Move OrcOptions from `sql/hive` to `s...

2017-11-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19636 Hi, @cloud-fan and @gatorsmile . Could you review this PR? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19427: [SPARK-22294][Deploy] Reset spark.driver.bindAddress whe...

2017-11-02 Thread ssaavedra
Github user ssaavedra commented on the issue: https://github.com/apache/spark/pull/19427 Is anyone considering this patch? Should I advertise it anywhere else? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior ...

2017-11-02 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19607#discussion_r148598592 --- Diff: python/pyspark/serializers.py --- @@ -274,12 +278,13 @@ def load_stream(self, stream): """ Deserialize

[GitHub] spark pull request #19552: [SPARK-22329][SQL] Use NEVER_INFER for `spark.sql...

2017-11-02 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/19552 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19644: [SPARK-22306][SQL] alter table schema should not ...

2017-11-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19644#discussion_r148575793 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -857,19 +857,23 @@ object DDLUtils { } }

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83335/testReport)** for PR 19439 at commit

[GitHub] spark issue #19623: [SPARK-22078][SQL] clarify exception behaviors for all d...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83346/ Test PASSed. ---

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-02 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19607 >Aside: Could someone start preparing a patch that uses Arrow 0.8.x (where a lot of issues that surfaced throughout this process have been fixed)? @wesm I could start on this maybe in

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-11-02 Thread smurching
Github user smurching commented on the issue: https://github.com/apache/spark/pull/19433 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-02 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19631 Sigh. retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19636: [SPARK-22416][SQL] Move OrcOptions from `sql/hive` to `s...

2017-11-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19636 LGTM, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19642: [SPARK-22410][SQL] Remove unnecessary output from BatchE...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19642 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83347/ Test FAILed. ---

[GitHub] spark issue #19642: [SPARK-22410][SQL] Remove unnecessary output from BatchE...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19642 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior ...

2017-11-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19607#discussion_r148576712 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -948,6 +948,14 @@ object SQLConf { .intConf

[GitHub] spark pull request #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior ...

2017-11-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19607#discussion_r148576678 --- Diff: python/pyspark/sql/types.py --- @@ -1629,35 +1629,121 @@ def to_arrow_type(dt): return arrow_type -def

[GitHub] spark pull request #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior ...

2017-11-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19607#discussion_r148576656 --- Diff: python/pyspark/sql/types.py --- @@ -1629,35 +1629,121 @@ def to_arrow_type(dt): return arrow_type -def

[GitHub] spark pull request #19644: [SPARK-22306][SQL] alter table schema should not ...

2017-11-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19644#discussion_r148578288 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -306,10 +305,10 @@ class

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148578359 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -0,0 +1,227 @@

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148300912 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148044235 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala --- @@ -0,0 +1,103 @@

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148566806 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -0,0 +1,227 @@

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148569836 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -0,0 +1,227 @@

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148040015 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerBackendUtils.scala --- @@ -0,0 +1,47 @@ +/* + * Licensed to the

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148300842 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148042590 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/ConfigurationUtils.scala --- @@ -0,0 +1,58 @@ +/* + *

[GitHub] spark issue #19644: [SPARK-22306][SQL] alter table schema should not erase t...

2017-11-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19644 +1, LGTM. Thank you, @cloud-fan . BTW, although this is a forward-port, `checkFieldNames` part is new. I'm wondering if you are going to backport this into branch-2.2 again. ---

[GitHub] spark issue #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with the im...

2017-11-02 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18538 @mgaido91 Don't worry, I'll post a follow-up PR for discussion in a few days. Thanks. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19623: [SPARK-22078][SQL] clarify exception behaviors fo...

2017-11-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19623#discussion_r148599528 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -50,28 +53,34 @@ /** *

[GitHub] spark issue #19623: [SPARK-22078][SQL] clarify exception behaviors for all d...

2017-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19623 **[Test build #83346 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83346/testReport)** for PR 19623 at commit

[GitHub] spark pull request #19623: [SPARK-22078][SQL] clarify exception behaviors fo...

2017-11-02 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/19623#discussion_r148605519 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -50,28 +53,34 @@ /** * Creates a

[GitHub] spark issue #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMetrics

2017-11-02 Thread tengpeng
Github user tengpeng commented on the issue: https://github.com/apache/spark/pull/19638 @srowen I have fixed scaladocs and since issues. I will pay special attention to this issue next time. --- - To unsubscribe,

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-02 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r148611431 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #19637: [SPARK-22243][DStream]spark.yarn.jars should reload from...

2017-11-02 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19637 @srowen we have already discussed this in #19469. I'm going to merge into master since the change is the same as #19469. --- -

[GitHub] spark issue #19469: [SPARK-22243][DStreams]spark.yarn.jars reload from confi...

2017-11-02 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19469 > sure, @zsxwing please beaware of apache-spark-on-k8s#516 and #19427 Yeah. I'm aware of them. I will review #19427. ---

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148618367 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/RegressionEvaluator.scala --- @@ -49,8 +49,8 @@ final class RegressionEvaluator

[GitHub] spark pull request #19427: [SPARK-22294][Deploy] Reset spark.driver.bindAddr...

2017-11-02 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19427#discussion_r148621269 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -62,6 +63,7 @@ class Checkpoint(ssc: StreamingContext, val

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148623948 --- Diff: mllib/src/test/scala/org/apache/spark/ml/evaluation/RegressionEvaluatorSuite.scala --- @@ -73,6 +73,11 @@ class RegressionEvaluatorSuite

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83344/ Test PASSed. ---

[GitHub] spark pull request #19644: [SPARK-22306][SQL] alter table schema should not ...

2017-11-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19644#discussion_r148613092 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -306,10 +305,10 @@ class

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148612789 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/RegressionEvaluator.scala --- @@ -49,8 +49,8 @@ final class RegressionEvaluator

[GitHub] spark pull request #19637: [SPARK-22243][DStream]spark.yarn.jars should relo...

2017-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19637 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMetrics

2017-11-02 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19638 Hm, right, this fails because it's adding a method to a public trait. Although it's unlikely anyone implements this, they might. Hm, not sure what we do here that can preserve source and binary

[GitHub] spark issue #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMetrics

2017-11-02 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/19638 Do we need to expose it publicly in the trait? Could we make it `private[mllib]`? Since it will only be used to compute this metric. Alternatively, you can just compute `n` from the size of

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread tengpeng
Github user tengpeng commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148626297 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/RegressionEvaluator.scala --- @@ -49,8 +49,8 @@ final class RegressionEvaluator

[GitHub] spark pull request #19636: [SPARK-22416][SQL] Move OrcOptions from `sql/hive...

2017-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19636 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMetrics

2017-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19638 **[Test build #3977 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3977/testReport)** for PR 19638 at commit

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148634321 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/RegressionEvaluator.scala --- @@ -49,8 +49,8 @@ final class RegressionEvaluator

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148634479 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -722,6 +722,17 @@ class LinearRegressionSummary

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-02 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/19468 @foxish @mridulm Heads up - since the last review iteration, I wrote an extra test in `KubernetesClusterSchedulerBackend` that exposed a bug where if executors never register with the driver but

[GitHub] spark issue #19644: [SPARK-22306][SQL] alter table schema should not erase t...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19644 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMetrics

2017-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19638 **[Test build #3977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3977/testReport)** for PR 19638 at commit

[GitHub] spark pull request #19632: Added description to python spark Pi example

2017-11-02 Thread animenon
Github user animenon commented on a diff in the pull request: https://github.com/apache/spark/pull/19632#discussion_r148620630 --- Diff: examples/src/main/python/pi.py --- @@ -27,12 +27,16 @@ if __name__ == "__main__": """ Usage: pi [partitions] +

[GitHub] spark issue #19644: [SPARK-22306][SQL] alter table schema should not erase t...

2017-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19644 **[Test build #83345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83345/testReport)** for PR 19644 at commit

[GitHub] spark issue #19644: [SPARK-22306][SQL] alter table schema should not erase t...

2017-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83345/ Test PASSed. ---

[GitHub] spark issue #19636: [SPARK-22416][SQL] Move OrcOptions from `sql/hive` to `s...

2017-11-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19636 Thank you so much, @cloud-fan , @HyukjinKwon , and @jiangxb1987 ! --- - To unsubscribe, e-mail:

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83344/testReport)** for PR 19439 at commit

[GitHub] spark issue #19469: [SPARK-22243][DStreams]spark.yarn.jars reload from confi...

2017-11-02 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19469 FYI, looks like @ChenjunZou opened a new PR #19637 rather than reopening this. Since the content is the same, I just merged #19637. ---

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148619618 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/MultivariateOnlineSummarizer.scala --- @@ -230,6 +230,13 @@ class MultivariateOnlineSummarizer

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148619202 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/MultivariateOnlineSummarizer.scala --- @@ -230,6 +230,13 @@ class MultivariateOnlineSummarizer

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148585371 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/RegressionEvaluator.scala --- @@ -49,8 +49,8 @@ final class RegressionEvaluator

[GitHub] spark pull request #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMe...

2017-11-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19638#discussion_r148619734 --- Diff: mllib/src/test/scala/org/apache/spark/ml/evaluation/RegressionEvaluatorSuite.scala --- @@ -73,6 +73,11 @@ class RegressionEvaluatorSuite

[GitHub] spark pull request #19632: Added description to python spark Pi example

2017-11-02 Thread animenon
Github user animenon commented on a diff in the pull request: https://github.com/apache/spark/pull/19632#discussion_r148628286 --- Diff: examples/src/main/python/pi.py --- @@ -27,12 +27,16 @@ if __name__ == "__main__": """ Usage: pi [partitions] +

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148437575 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/AggregateFieldExtractionPushdown.scala --- @@ -0,0 +1,77 @@ +/* + *

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148436498 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/AggregateFieldExtractionPushdown.scala --- @@ -0,0 +1,77 @@ +/* + *

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148433577 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/ProjectionOverSchema.scala --- @@ -0,0 +1,61 @@ +/* + * Licensed to

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148446149 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/JoinFieldExtractionPushdown.scala --- @@ -0,0 +1,66 @@ +/* + *

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148446166 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/AggregateFieldExtractionPushdown.scala --- @@ -0,0 +1,77 @@ +/* + *

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19601 @cloud-fan could you please review this PR? In my prototype, I succeeded to support a current nested array for table cache by changing only `UnsafeColumnVector.java`. For ease of review,

[GitHub] spark issue #19553: [SPARK-22330][CORE] Linear containsKey operation for ser...

2017-11-02 Thread Whoosh
Github user Whoosh commented on the issue: https://github.com/apache/spark/pull/19553 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148446396 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/JoinFieldExtractionPushdown.scala --- @@ -0,0 +1,66 @@ +/* + *

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148437069 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/JoinFieldExtractionPushdown.scala --- @@ -0,0 +1,66 @@ +/* + *

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148434529 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/ProjectionOverSchema.scala --- @@ -0,0 +1,61 @@ +/* + * Licensed to

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-11-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r148446321 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/FieldExtractionPushdown.scala --- @@ -0,0 +1,53 @@ +/* + * Licensed

  1   2   3   4   5   >