[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75553873 --- Diff: yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnShuffleEncryptionSuite.scala --- @@ -0,0 +1,355 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request #14384: [Spark-16443][SparkR] Alternating Least Squares (...

2016-08-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14384 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75553741 --- Diff: yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnShuffleEncryptionSuite.scala --- @@ -0,0 +1,355 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75553689 --- Diff: dev/deps/spark-deps-hadoop-2.7 --- @@ -112,6 +113,7 @@ jets3t-0.9.3.jar jetty-6.1.26.jar jetty-util-6.1.26.jar jline-2.12.1.jar +jn

[GitHub] spark issue #14384: [Spark-16443][SparkR] Alternating Least Squares (ALS) wr...

2016-08-19 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/14384 LGTM. Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75553471 --- Diff: yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnShuffleEncryptionSuite.scala --- @@ -0,0 +1,355 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r75553470 --- Diff: dev/py-validate.sh --- @@ -0,0 +1,110 @@ +#!/usr/bin/env bash --- End diff -- Where is this file used? --- If your project is set

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r75553298 --- Diff: python/pyspark/cloudpickle.py --- @@ -42,17 +42,17 @@ """ --- End diff -- This file is copied, please ignore it --- If your proje

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r75553255 --- Diff: python/pyspark/heapq3.py --- @@ -408,10 +408,12 @@ __all__ = ['heappush', 'heappop', 'heapify', 'heapreplace', 'merge', 'nlarges

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r75553180 --- Diff: python/pyspark/ml/param/shared.py --- @@ -25,7 +25,8 @@ class HasMaxIter(Params): Mixin for param maxIter: max number of iterations (>= 0).

[GitHub] spark issue #14718: [SPARK-16711] YarnShuffleService doesn't re-init properl...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14718 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64094/ Test PASSed. ---

[GitHub] spark issue #14718: [SPARK-16711] YarnShuffleService doesn't re-init properl...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14718 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14705: [SPARK-16508][SparkR] Fix CRAN undocumented/duplicated a...

2016-08-19 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/14705 Agree with @felixcheung . Let do the `setGeneric(, ...)` and do not use `...` in function definition if it is not required. Note that having every param documented is not a strict requirement of CRAN

[GitHub] spark issue #14718: [SPARK-16711] YarnShuffleService doesn't re-init properl...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14718 **[Test build #64094 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64094/consoleFull)** for PR 14718 at commit [`2643d56`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75551862 --- Diff: core/src/main/scala/org/apache/spark/serializer/SerializerManager.scala --- @@ -103,6 +107,20 @@ private[spark] class SerializerManager(defaultSeria

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75551679 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1962,6 +1962,26 @@ private[spark] object Utils extends Logging { path

[GitHub] spark issue #14475: [SPARK-16862] Configurable buffer size in `UnsafeSorterS...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14475 **[Test build #64104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64104/consoleFull)** for PR 14475 at commit [`fcc1d4d`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14721 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64095/ Test PASSed. ---

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64103/consoleFull)** for PR 14723 at commit [`520a17f`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14721 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14721 **[Test build #64095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64095/consoleFull)** for PR 14721 at commit [`da07ad9`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-19 Thread clockfly
GitHub user clockfly opened a pull request: https://github.com/apache/spark/pull/14723 [SQL][WIP][Test] Supports object-based aggregation function which can store arbitrary objects in aggregation buffer. ## What changes were proposed in this pull request This PR allows user

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75551246 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1962,6 +1962,26 @@ private[spark] object Utils extends Logging { path

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75551184 --- Diff: core/src/main/scala/org/apache/spark/serializer/SerializerManager.scala --- @@ -103,6 +107,20 @@ private[spark] class SerializerManager(defaultSeria

[GitHub] spark issue #14674: [SPARK-17002][CORE]: Document that spark.ssl.protocol. i...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14674 **[Test build #64102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64102/consoleFull)** for PR 14674 at commit [`9017807`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75550687 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark issue #14674: [SPARK-17002][CORE]: Document that spark.ssl.protocol. i...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14674 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64078/ Test FAILed. ---

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75550523 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/MetastoreRelation.scala --- @@ -141,7 +142,16 @@ private[hive] case class MetastoreRelation(

[GitHub] spark issue #14674: [SPARK-17002][CORE]: Document that spark.ssl.protocol. i...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14674 **[Test build #64101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64101/consoleFull)** for PR 14674 at commit [`9017807`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14674: [SPARK-17002][CORE]: Document that spark.ssl.protocol. i...

2016-08-19 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/14674 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75549841 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1962,6 +1962,26 @@ private[spark] object Utils extends Logging { path }

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75549695 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1962,6 +1962,26 @@ private[spark] object Utils extends Logging { path }

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75549736 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoStreamUtils.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #14674: [SPARK-17002][CORE]: Document that spark.ssl.protocol. i...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14674 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14674: [SPARK-17002][CORE]: Document that spark.ssl.protocol. i...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14674 **[Test build #64078 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64078/consoleFull)** for PR 14674 at commit [`9017807`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75549270 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -168,6 +170,57 @@ class StatisticsSuite extends QueryTest with T

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread srinathshankar
Github user srinathshankar commented on the issue: https://github.com/apache/spark/pull/14721 Fixed up error messages in ExpressionParserSuite --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75549255 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1962,6 +1962,26 @@ private[spark] object Utils extends Logging { path }

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14721 **[Test build #64100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64100/consoleFull)** for PR 14721 at commit [`89a73e3`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #14721: [SPARK-17158][SQL] Change error message for out o...

2016-08-19 Thread srinathshankar
Github user srinathshankar commented on a diff in the pull request: https://github.com/apache/spark/pull/14721#discussion_r75548962 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1278,10 +1278,17 @@ class AstBuilder extends SqlB

[GitHub] spark pull request #14721: [SPARK-17158][SQL] Change error message for out o...

2016-08-19 Thread srinathshankar
Github user srinathshankar commented on a diff in the pull request: https://github.com/apache/spark/pull/14721#discussion_r75548986 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1278,10 +1278,17 @@ class AstBuilder extends SqlB

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75548733 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoStreamUtils.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75548671 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoStreamUtils.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #14666: [SPARK-16578][SparkR] Enable SparkR to connect to a remo...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14666 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14666: [SPARK-16578][SparkR] Enable SparkR to connect to a remo...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14666 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64091/ Test PASSed. ---

[GitHub] spark issue #14666: [SPARK-16578][SparkR] Enable SparkR to connect to a remo...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14666 **[Test build #64091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64091/consoleFull)** for PR 14666 at commit [`c1adc95`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75548282 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1962,6 +1962,26 @@ private[spark] object Utils extends Logging { path }

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75548186 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -168,6 +170,57 @@ class StatisticsSuite extends QueryTest with T

[GitHub] spark pull request #14637: [WIP] [SPARK-16967] move mesos to module

2016-08-19 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/14637#discussion_r75547904 --- Diff: mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterManager.scala --- @@ -0,0 +1,61 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75547546 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherArguments.scala --- @@ -102,7 +115,9 @@ private[mesos] class MesosClusterD

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75547461 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1962,6 +1962,26 @@ private[spark] object Utils extends Logging { path }

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-08-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r75547368 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherArguments.scala --- @@ -73,6 +82,10 @@ private[mesos] class MesosClusterDi

[GitHub] spark issue #10212: [SPARK-12221] add cpu time to metrics

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10212 **[Test build #64099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64099/consoleFull)** for PR 10212 at commit [`5a9ea8e`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #14717: [WIP][SPARK-17090][ML]Make tree aggregation level...

2016-08-19 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14717#discussion_r75546637 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1331,8 +1343,8 @@ private class LogisticCostFun(

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75546604 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoStreamUtils.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #14722: [SPARK-13286] [SQL] add the next expression of SQLExcept...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14722 **[Test build #64098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64098/consoleFull)** for PR 14722 at commit [`51fbf25`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #14722: [SPARK-13286] [SQL] add the next expression of SQ...

2016-08-19 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/14722 [SPARK-13286] [SQL] add the next expression of SQLException as cause ## What changes were proposed in this pull request? Some JDBC driver (for example PostgreSQL) does not use the underlying

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75546361 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark issue #14717: [WIP][SPARK-17090][ML]Make tree aggregation level in lin...

2016-08-19 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14717 If `SharedParamsCodeGen.scala` is changed, you need to manually run ```scala build/sbt "mllib/runMain org.apache.spark.ml.param.shared.SharedParamsCodeGen" ``` to generate a

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75545710 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark issue #14517: [SPARK-16931][PYTHON] PySpark APIS for bucketBy and sort...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14517 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64097/ Test FAILed. ---

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64087/ Test PASSed. ---

[GitHub] spark issue #14517: [SPARK-16931][PYTHON] PySpark APIS for bucketBy and sort...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14517 **[Test build #64097 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64097/consoleFull)** for PR 14517 at commit [`78a7b63`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14517: [SPARK-16931][PYTHON] PySpark APIS for bucketBy and sort...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14517 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14721 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14721 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64089/ Test PASSed. ---

[GitHub] spark issue #14517: [SPARK-16931][PYTHON] PySpark APIS for bucketBy and sort...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14517 **[Test build #64097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64097/consoleFull)** for PR 14517 at commit [`78a7b63`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14721 **[Test build #64089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64089/consoleFull)** for PR 14721 at commit [`cca27be`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14467 **[Test build #64087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64087/consoleFull)** for PR 14467 at commit [`04a1d37`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14719: [SPARK-17154][SQL] Wrong result can be returned or Analy...

2016-08-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14719 See a related PR by @cloud-fan : https://github.com/apache/spark/pull/11632 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75545302 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75545187 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark issue #10212: [SPARK-12221] add cpu time to metrics

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10212 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64096/ Test FAILed. ---

[GitHub] spark issue #10212: [SPARK-12221] add cpu time to metrics

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10212 **[Test build #64096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64096/consoleFull)** for PR 10212 at commit [`bf8d4f8`](https://github.com/apache/spark/commit/

[GitHub] spark issue #10212: [SPARK-12221] add cpu time to metrics

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10212 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75544361 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -108,4 +126,8 @@ case class AnalyzeTableCommand

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14721 **[Test build #3229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3229/consoleFull)** for PR 14721 at commit [`cca27be`](https://github.com/apache/spark/commit

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14712 If the goal also includes the interoperability with Hive, the test cases should also verify whether the table property `COLUMN_STATS_ACCURATE` is `true` or not. This should be implicitly updated

[GitHub] spark issue #14721: [SPARK-17158][SQL] Change error message for out of range...

2016-08-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14721 This looks pretty good. Could you also add a few tests to ExpressionParserSuite? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request #14721: [SPARK-17158][SQL] Change error message for out o...

2016-08-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14721#discussion_r75542559 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1278,10 +1278,17 @@ class AstBuilder extends SqlBaseB

[GitHub] spark issue #10212: [SPARK-12221] add cpu time to metrics

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10212 **[Test build #64096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64096/consoleFull)** for PR 10212 at commit [`bf8d4f8`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14649: [SPARK-17059][SQL] Allow FileFormat to specify partition...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14649 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14649: [SPARK-17059][SQL] Allow FileFormat to specify partition...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14649 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64086/ Test PASSed. ---

[GitHub] spark issue #14673: [SPARK-15083] [Web UI] History Server can OOM due to unl...

2016-08-19 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/14673 secondary sort in this case is whatever order the collection is stored in, since `LinkedHashMap` keeps an order (essentially by `tasked`, but really by insertion order) `tasked` becomes the defact

[GitHub] spark issue #14649: [SPARK-17059][SQL] Allow FileFormat to specify partition...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14649 **[Test build #64086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64086/consoleFull)** for PR 14649 at commit [`7a95989`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r75540560 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -88,14 +90,30 @@ case class AnalyzeTableCommand

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13650 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64093/ Test PASSed. ---

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13650 **[Test build #64093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64093/consoleFull)** for PR 13650 at commit [`505e3c2`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14721: [SPARK-17158][SQL] Change error message for out o...

2016-08-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14721#discussion_r75538597 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1278,10 +1278,17 @@ class AstBuilder extends SqlBaseB

[GitHub] spark pull request #14607: [SPARK-17063] [SQL] Improve performance of MSCK R...

2016-08-19 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/14607#discussion_r75538523 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -310,6 +310,13 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #14607: [SPARK-17063] [SQL] Improve performance of MSCK R...

2016-08-19 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/14607#discussion_r75538376 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -518,21 +550,87 @@ case class AlterTableRecoverPartitionsComm

[GitHub] spark pull request #14607: [SPARK-17063] [SQL] Improve performance of MSCK R...

2016-08-19 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/14607#discussion_r75538170 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -518,21 +550,87 @@ case class AlterTableRecoverPartitionsComm

[GitHub] spark pull request #14607: [SPARK-17063] [SQL] Improve performance of MSCK R...

2016-08-19 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/14607#discussion_r75537591 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -518,21 +550,87 @@ case class AlterTableRecoverPartitionsComm

[GitHub] spark pull request #14607: [SPARK-17063] [SQL] Improve performance of MSCK R...

2016-08-19 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/14607#discussion_r75537354 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -443,6 +446,31 @@ case class AlterTableDropPartitionCommand(

[GitHub] spark pull request #14607: [SPARK-17063] [SQL] Improve performance of MSCK R...

2016-08-19 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/14607#discussion_r75537096 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -443,6 +446,31 @@ case class AlterTableDropPartitionCommand(

[GitHub] spark issue #14384: [Spark-16443][SparkR] Alternating Least Squares (ALS) wr...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14384 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14384: [Spark-16443][SparkR] Alternating Least Squares (ALS) wr...

2016-08-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14384 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64090/ Test PASSed. ---

[GitHub] spark issue #14384: [Spark-16443][SparkR] Alternating Least Squares (ALS) wr...

2016-08-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14384 **[Test build #64090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64090/consoleFull)** for PR 14384 at commit [`a653256`](https://github.com/apache/spark/commit/

<    1   2   3   4   5   6   7   >