[GitHub] spark issue #20375: [SPARK-23199][SQL]improved Removes repetition from group...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20375 **[Test build #86688 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86688/testReport)** for PR 20375 at commit

[GitHub] spark issue #20386: [WIP][SPARK-23202][SQL] Break down DataSourceV2Writer.co...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20386: [WIP][SPARK-23202][SQL] Break down DataSourceV2Writer.co...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20386 **[Test build #86690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86690/testReport)** for PR 20386 at commit

[GitHub] spark issue #20386: [WIP][SPARK-23202][SQL] Break down DataSourceV2Writer.co...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20375: [SPARK-23199][SQL]improved Removes repetition from group...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20375 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86688/ Test FAILed. ---

[GitHub] spark issue #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20395 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86691/ Test FAILed. ---

[GitHub] spark issue #20386: [WIP][SPARK-23202][SQL] Break down DataSourceV2Writer.co...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86689/ Test FAILed. ---

[GitHub] spark issue #20386: [WIP][SPARK-23202][SQL] Break down DataSourceV2Writer.co...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86690/ Test FAILed. ---

[GitHub] spark issue #20386: [WIP][SPARK-23202][SQL] Break down DataSourceV2Writer.co...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20386 **[Test build #86689 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86689/testReport)** for PR 20386 at commit

[GitHub] spark issue #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20395 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20395 **[Test build #86691 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86691/testReport)** for PR 20395 at commit

[GitHub] spark issue #20375: [SPARK-23199][SQL]improved Removes repetition from group...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20375 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20369: [SPARK-23196] Unify continuous and microbatch V2 sinks

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20369 **[Test build #86687 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86687/testReport)** for PR 20369 at commit

[GitHub] spark issue #20369: [SPARK-23196] Unify continuous and microbatch V2 sinks

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20369 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86687/ Test FAILed. ---

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to match 0.4.2

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 > the problem is with the standard pickling too, right? Yup. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20383: [SPARK-23200] Reset Kubernetes-specific config on Checkp...

2018-01-26 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20383 We are going to cut another RC? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164052169 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164053213 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164052606 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164052969 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164053364 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164055326 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164055523 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #20403: [MINOR][PYTHON] Minor doc correction for 'spark.s...

2018-01-26 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20403 [MINOR][PYTHON] Minor doc correction for 'spark.sql.execution.arrow.enabled' ## What changes were proposed in this pull request? This PR proposes to fix the description for

[GitHub] spark issue #20383: [SPARK-23200] Reset Kubernetes-specific config on Checkp...

2018-01-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20383 This is not a big issue unless when we run Spark Streaming with checkpoint enabled. I'm not sure for now it is OK to add to 2.3.0 release (as this is not a block issue). @felixcheung up

[GitHub] spark issue #20403: [MINOR][PYTHON] Minor doc correction for 'spark.sql.exec...

2018-01-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20403 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164062967 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #20396: [SPARK-23217][ML] Add cosine distance measure to Cluster...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20396 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/276/

[GitHub] spark issue #20303: [SPARK-23128][SQL] A new approach to do adaptive executi...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20303 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20396: [SPARK-23217][ML] Add cosine distance measure to Cluster...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20396 **[Test build #86696 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86696/testReport)** for PR 20396 at commit

[GitHub] spark issue #20303: [SPARK-23128][SQL] A new approach to do adaptive executi...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20303 **[Test build #86697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86697/testReport)** for PR 20303 at commit

[GitHub] spark issue #20303: [SPARK-23128][SQL] A new approach to do adaptive executi...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20303 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/277/

[GitHub] spark pull request #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20395#discussion_r164065557 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -182,57 +187,54 @@ } /** - * Returns the

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20303: [SPARK-23128][SQL] A new approach to do adaptive ...

2018-01-26 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20303#discussion_r164070918 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/QueryStage.scala --- @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2018-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19993 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20396: [SPARK-23217][ML] Add cosine distance measure to Cluster...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20396 **[Test build #86696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86696/testReport)** for PR 20396 at commit

[GitHub] spark issue #20396: [SPARK-23217][ML] Add cosine distance measure to Cluster...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20396 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20396: [SPARK-23217][ML] Add cosine distance measure to Cluster...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20396 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86696/ Test PASSed. ---

[GitHub] spark issue #20403: [MINOR][PYTHON] Minor doc correction for 'spark.sql.exec...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86693/ Test FAILed. ---

[GitHub] spark issue #20403: [MINOR][PYTHON] Minor doc correction for 'spark.sql.exec...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20383: [SPARK-23200] Reset Kubernetes-specific config on...

2018-01-26 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20383#discussion_r164084503 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -53,6 +53,21 @@ class Checkpoint(ssc: StreamingContext, val

[GitHub] spark issue #20403: [MINOR][PYTHON] Minor doc correction for 'spark.sql.exec...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/281/

[GitHub] spark pull request #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20395#discussion_r164086846 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -182,57 +187,54 @@ } /** - * Returns

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164054702 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #20375: [SPARK-23199][SQL]improved Removes repetition from group...

2018-01-26 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20375 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20396: [SPARK-23217][ML] Add cosine distance measure to ...

2018-01-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20396#discussion_r164059420 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -421,13 +460,220 @@ private[evaluation] object

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164062504 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #20396: [SPARK-23217][ML] Add cosine distance measure to Cluster...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20396 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/279/

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164063514 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #86699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86699/testReport)** for PR 13599 at commit

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 @zjffdu @HyukjinKwon please help to review. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2018-01-26 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/19993 Merged to master / branch-2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2018-01-26 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/19993 Thanks @mgaido91 and @jkbradley for working on this and others for review --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19892: [SPARK-22797][PySpark] Bucketizer support multi-c...

2018-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19892 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20369: [SPARK-23196] Unify continuous and microbatch V2 sinks

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20369 **[Test build #86692 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86692/testReport)** for PR 20369 at commit

[GitHub] spark issue #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), unbounded...

2018-01-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20400 cc @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkS...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20404#discussion_r164088909 --- Diff: python/pyspark/sql/session.py --- @@ -225,6 +225,7 @@ def __init__(self, sparkContext, jsparkSession=None): if

[GitHub] spark issue #20375: [SPARK-23199][SQL]improved Removes repetition from group...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20375 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86694/ Test FAILed. ---

[GitHub] spark issue #20375: [SPARK-23199][SQL]improved Removes repetition from group...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20375 **[Test build #86694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86694/testReport)** for PR 20375 at commit

[GitHub] spark issue #20405: [SPARK-23229][SQL] Dataset.hint should use planWithBarri...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20405 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20405: [SPARK-23229][SQL] Dataset.hint should use planWithBarri...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20405 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/282/

[GitHub] spark pull request #20399: [SPARK-23209][core] Allow credential manager to w...

2018-01-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20399#discussion_r164055956 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala --- @@ -75,6 +75,17 @@ private[spark] class

[GitHub] spark pull request #20396: [SPARK-23217][ML] Add cosine distance measure to ...

2018-01-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20396#discussion_r164058701 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -111,6 +129,53 @@ object ClusteringEvaluator }

[GitHub] spark issue #20386: [WIP][SPARK-23202][SQL] Break down DataSourceV2Writer.co...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20386 **[Test build #86698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86698/testReport)** for PR 20386 at commit

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164063467 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20395#discussion_r164066651 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ArrowColumnVector.java --- @@ -450,13 +437,12 @@ final boolean isNullAt(int rowId) {

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164064918 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164065094 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164066958 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkS...

2018-01-26 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/20404 [SPARK-23228][PYSPARK] Add Python Created jsparkSession to JVM's defaultSession ## What changes were proposed in this pull request? In the current PySpark code, Python created

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/280/

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20404 **[Test build #86700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86700/testReport)** for PR 20404 at commit

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20404 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20403: [MINOR][PYTHON] Minor doc correction for 'spark.sql.exec...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20403 **[Test build #86693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86693/testReport)** for PR 20403 at commit

[GitHub] spark pull request #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20395#discussion_r164086245 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java --- @@ -602,7 +603,17 @@ public final int

[GitHub] spark pull request #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20395#discussion_r164088592 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java --- @@ -602,7 +603,17 @@ public final int

[GitHub] spark pull request #19285: [SPARK-22068][CORE]Reduce the duplicate code betw...

2018-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19285 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20303: [SPARK-23128][SQL] A new approach to do adaptive ...

2018-01-26 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20303#discussion_r164089610 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/QueryStage.scala --- @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20405: [SPARK-23229][SQL] Dataset.hint should use planWi...

2018-01-26 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/20405 [SPARK-23229][SQL] Dataset.hint should use planWithBarrier logical plan ## What changes were proposed in this pull request? Every time `Dataset.hint` is used it triggers execution

[GitHub] spark pull request #20396: [SPARK-23217][ML] Add cosine distance measure to ...

2018-01-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20396#discussion_r164059036 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -421,13 +460,220 @@ private[evaluation] object

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164059019 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20395 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/275/

[GitHub] spark issue #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20395 **[Test build #86695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86695/testReport)** for PR 20395 at commit

[GitHub] spark issue #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20395 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-01-26 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 cc @gatorsmile @hvanhovell @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20395#discussion_r164067505 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java --- @@ -602,7 +603,17 @@ public final int

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86699/ Test FAILed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #86699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86699/testReport)** for PR 13599 at commit

[GitHub] spark pull request #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pys...

2018-01-26 Thread zjffdu
Github user zjffdu commented on a diff in the pull request: https://github.com/apache/spark/pull/13599#discussion_r164068172 --- Diff: core/src/main/scala/org/apache/spark/api/python/VirtualEnvFactory.scala --- @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #20383: [SPARK-23200] Reset Kubernetes-specific config on Checkp...

2018-01-26 Thread ssaavedra
Github user ssaavedra commented on the issue: https://github.com/apache/spark/pull/20383 However, Spark Streaming should always be used with checkpoint enabled if you are using at least `updateStateByKey` or `reduceByKeyAndWindow` and you don't want to lose data, or miscalculate

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2018-01-26 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/19892 Merged to master / branch-2.3. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164078885 --- Diff: python/pyspark/sql/functions.py --- @@ -809,6 +809,36 @@ def ntile(n): return Column(sc._jvm.functions.ntile(int(n)))

[GitHub] spark issue #20403: [MINOR][PYTHON] Minor doc correction for 'spark.sql.exec...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20403 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20383: [SPARK-23200] Reset Kubernetes-specific config on Checkp...

2018-01-26 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20383 I'm ok with backporting it once the non-existent config is removed and we're confident we're covering all the requisite config. Also would make sense to have a test under

[GitHub] spark issue #20369: [SPARK-23196] Unify continuous and microbatch V2 sinks

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20369 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20369: [SPARK-23196] Unify continuous and microbatch V2 sinks

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20369 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86692/ Test FAILed. ---

  1   2   3   4   5   >