[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15523 **[Test build #67697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67697/consoleFull)** for PR 15523 at commit [`49daace`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15660: [SPARK-18133][Examples] [ML] [Python ML Pipeline Example...

2016-10-28 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15660 @jagadeesanas2 I'd like to merge it into branch-2.0 also, but there is conflict. Could you please prepare a separate PR for branch-2.0? Thanks. --- If your project is set up for it, you can repl

[GitHub] spark pull request #15660: [SPARK-18133][Examples] [ML] [Python ML Pipeline ...

2016-10-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15660 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15660: [SPARK-18133][Examples] [ML] [Python ML Pipeline Example...

2016-10-28 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15660 LGTM2, merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15513 Thanks @gatorsmile. Just FYI, I would like to note the rule I used for argument types (just to avoid extra efforts when you review). As we all know, I did not mention implicit casting as

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85491680 --- Diff: dev/run-tests.py --- @@ -583,6 +589,7 @@ def main(): modules_with_python_tests = [m for m in test_modules if m.python_test_goals]

[GitHub] spark issue #15667: [SPARK-18107][SQL] Insert overwrite statement runs much ...

2016-10-28 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15667 I checked the failed test. Special characters in the partition path cause the failure, e..g., alter table ppr_test add partition (ds = '12:4'); alter table ppr_test add partition

[GitHub] spark pull request #15665: [SPARK-18146] [SQL] Avoid using Union to chain to...

2016-10-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15665#discussion_r85490391 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -254,7 +254,8 @@ final class DataFrameWriter[T] private[sql](ds: Dataset[T]

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15670 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #15667: [SPARK-18107][SQL] Insert overwrite statement runs much ...

2016-10-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15667 cc @ericl --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #15670: [SPARK-18161] [Python] Allow use of pickle to ser...

2016-10-28 Thread singularperturbation
GitHub user singularperturbation opened a pull request: https://github.com/apache/spark/pull/15670 [SPARK-18161] [Python] Allow use of pickle to serialize > 4 GB objects where available ## What changes were proposed in this pull request? Since pickle protocol 4 (available

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileProvider

2016-10-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15634 FileIndex sounds good to me. I wouldn't call it "Metastore" though, since that is a Hive specific term. I'd call it Catalog. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #15024: [SPARK-17470][SQL] unify path for data source tab...

2016-10-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15024#discussion_r85488796 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -415,16 +422,34 @@ private[spark] class HiveExternalCatalog(co

[GitHub] spark pull request #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect N...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15523#discussion_r85488370 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -1617,6 +1617,72 @@ class DataFrameSuite extends QueryTest with SharedSQLC

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15669 **[Test build #67699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67699/consoleFull)** for PR 15669 at commit [`67a5ccf`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85486513 --- Diff: dev/pip-sanity-check.py --- @@ -0,0 +1,36 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor lice

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85486184 --- Diff: dev/make-distribution.sh --- @@ -201,6 +201,12 @@ fi # Copy data files cp -r "$SPARK_HOME/data" "$DISTDIR" +# Make pip package

[GitHub] spark issue #14640: [SPARK-17055] [MLLIB] add groupKFold to CrossValidator

2016-10-28 Thread rdelassus
Github user rdelassus commented on the issue: https://github.com/apache/spark/pull/14640 There is an infinite number of ways to make folds. Until now we had the mlutils kfold. You want to add the groupedKfold. But I don't think we should add one by one every folding method that can be

[GitHub] spark pull request #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile do...

2016-10-28 Thread zjffdu
GitHub user zjffdu opened a pull request: https://github.com/apache/spark/pull/15669 [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't work in yarn-cluster mode ## What changes were proposed in this pull request? Will fallback to container working directory if the abso

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15668 **[Test build #67698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67698/consoleFull)** for PR 15668 at commit [`7029e89`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-10-28 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15668 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85484770 --- Diff: dev/create-release/release-build.sh --- @@ -187,10 +208,10 @@ if [[ "$1" == "package" ]]; then # We increment the Zinc port each time to avoid

[GitHub] spark pull request #15636: [SPARK-18109][ML] Add instrumentation to GMM

2016-10-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15636 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15655: [SPARK-18010][CORE] Reduce work performed for building u...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15655 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15655: [SPARK-18010][CORE] Reduce work performed for building u...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15655 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67695/ Test FAILed. ---

[GitHub] spark issue #15655: [SPARK-18010][CORE] Reduce work performed for building u...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15655 **[Test build #67695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67695/consoleFull)** for PR 15655 at commit [`1d9476a`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15636: [SPARK-18109][ML] Add instrumentation to GMM

2016-10-28 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15636 LGTM, merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15513 Will review it tomorrow. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15523 **[Test build #67697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67697/consoleFull)** for PR 15523 at commit [`49daace`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15660: [SPARK-18133][Examples] [ML] [Python ML Pipeline Example...

2016-10-28 Thread vincentpham1991
Github user vincentpham1991 commented on the issue: https://github.com/apache/spark/pull/15660 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] spark pull request #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect N...

2016-10-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15523#discussion_r85483392 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -1617,6 +1617,72 @@ class DataFrameSuite extends QueryTest with Shared

[GitHub] spark issue #11374: [SPARK-12042] Python API for mllib.stat.test.StreamingTe...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11374 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #11374: [SPARK-12042] Python API for mllib.stat.test.StreamingTe...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11374 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67696/ Test PASSed. ---

[GitHub] spark issue #11374: [SPARK-12042] Python API for mllib.stat.test.StreamingTe...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11374 **[Test build #67696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67696/consoleFull)** for PR 11374 at commit [`615fbbb`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67690/ Test FAILed. ---

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67690/consoleFull)** for PR 15626 at commit [`1390711`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67694/ Test PASSed. ---

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #67694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67694/consoleFull)** for PR 15354 at commit [`4d69ab2`](https://github.com/apache/spark/commit/

<    1   2   3   4   5