[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15538 **[Test build #67707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67707/consoleFull)** for PR 15538 at commit

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67707/ Test PASSed. ---

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15538 **[Test build #3377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3377/consoleFull)** for PR 15538 at commit

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15538 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15627: [SPARK-18099][YARN] Fail if same files added to d...

2016-10-28 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15627#discussion_r85552081 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -598,8 +598,12 @@ private[spark] class Client( ).foreach { case

[GitHub] spark pull request #15671: [SPARK-14567][ML]Add instrumentation logs to MLli...

2016-10-28 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15671#discussion_r85551637 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -96,11 +96,13 @@ class DecisionTreeClassifier

[GitHub] spark issue #15654: [SPARK-16881][MESOS] Migrate Mesos configs to use Config...

2016-10-28 Thread techaddict
Github user techaddict commented on the issue: https://github.com/apache/spark/pull/15654 cc: @mgummelt @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15024 **[Test build #67708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67708/consoleFull)** for PR 15024 at commit

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-10-28 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15024 I have updated this PR according to these rules: 1. outside of `ExternalCatalog`, data source and hive tables should both use `locationUri` to store table location. 2. inside

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85538940 --- Diff: python/setup.py --- @@ -0,0 +1,170 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85538754 --- Diff: python/setup.py --- @@ -0,0 +1,170 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85538196 --- Diff: python/setup.py --- @@ -0,0 +1,170 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85536498 --- Diff: dev/create-release/release-build.sh --- @@ -187,10 +208,10 @@ if [[ "$1" == "package" ]]; then # We increment the Zinc port each time to

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85536240 --- Diff: dev/make-distribution.sh --- @@ -201,6 +201,12 @@ fi # Copy data files cp -r "$SPARK_HOME/data" "$DISTDIR" +# Make pip package

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85533965 --- Diff: python/setup.py --- @@ -0,0 +1,170 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #15655: [SPARK-18010][CORE] Reduce work performed for building u...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15655 **[Test build #3378 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3378/consoleFull)** for PR 15655 at commit

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-28 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15553 `mvn compile` takes about 5 minutes on my laptop, and `mvn compile test-compile` takes about 9 minutes. I don't use either of these during development, because incremental compiles are tens of

[GitHub] spark issue #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15603 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67705/ Test FAILed. ---

[GitHub] spark issue #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15603 **[Test build #67705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67705/consoleFull)** for PR 15603 at commit

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85531031 --- Diff: python/setup.py --- @@ -0,0 +1,170 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15538 **[Test build #67707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67707/consoleFull)** for PR 15538 at commit

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15538 **[Test build #3377 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3377/consoleFull)** for PR 15538 at commit

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-28 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15538 Jenkins add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-10-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14788 I will be back after testing/looking into other databases soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85526799 --- Diff: dev/make-distribution.sh --- @@ -201,6 +201,12 @@ fi # Copy data files cp -r "$SPARK_HOME/data" "$DISTDIR" +# Make pip package

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85525949 --- Diff: python/setup.py --- @@ -0,0 +1,170 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67706/ Test PASSed. ---

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67706 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67706/consoleFull)** for PR 15671 at commit

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85524567 --- Diff: python/setup.py --- @@ -0,0 +1,170 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15672 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15672 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67702/ Test FAILed. ---

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15672 **[Test build #67702 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67702/consoleFull)** for PR 15672 at commit

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-10-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14788 https://github.com/apache/spark/pull/14788#issuecomment-242906410 here is my observation. It seems usually option 1 ir option 3. I can take a look deeper if we want to follow this or be very

[GitHub] spark issue #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to be pip i...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67701/ Test FAILed. ---

[GitHub] spark issue #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to be pip i...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15659 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to be pip i...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15659 **[Test build #67701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67701/consoleFull)** for PR 15659 at commit

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r85521549 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -135,7 +133,7 @@ private[spark] class SparkUI private ( private[spark] abstract

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r85521860 --- Diff: core/src/test/scala/org/apache/spark/ui/UISuite.scala --- @@ -179,16 +179,15 @@ class UISuite extends SparkFunSuite { test("verify

[GitHub] spark pull request #15652: [SPARK-16987] [None] Add spark-default.conf prope...

2016-10-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15652#discussion_r85521074 --- Diff: docs/configuration.md --- @@ -1598,7 +1598,7 @@ Apart from these, the following properties are also available, and may be useful false

[GitHub] spark pull request #15652: [SPARK-16987] [None] Add spark-default.conf prope...

2016-10-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15652#discussion_r85521020 --- Diff: docs/configuration.md --- @@ -1664,6 +1664,14 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85520992 --- Diff: dev/create-release/release-build.sh --- @@ -187,10 +208,10 @@ if [[ "$1" == "package" ]]; then # We increment the Zinc port each time to

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-28 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/15669 @vanzin @JoshRosen @yanboliang @brkyvz Please help review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67706/consoleFull)** for PR 15671 at commit

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67704/ Test FAILed. ---

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67704/consoleFull)** for PR 15671 at commit

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-10-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14788 What do other databases do? Does date_add in other databases support timestamps? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67703/ Test FAILed. ---

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67703/consoleFull)** for PR 15671 at commit

[GitHub] spark issue #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15603 **[Test build #67705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67705/consoleFull)** for PR 15603 at commit

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-28 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r85516045 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -135,7 +133,7 @@ private[spark] class SparkUI private ( private[spark]

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67704/consoleFull)** for PR 15671 at commit

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-10-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14788 Could I then go for option 1.? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67703/consoleFull)** for PR 15671 at commit

[GitHub] spark issue #15672: [SPARK-18148][SQL] Misleading Error Message for Aggregat...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15672 **[Test build #67702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67702/consoleFull)** for PR 15672 at commit

[GitHub] spark pull request #15672: [SPARK-18148][SQL] Misleading Error Message for A...

2016-10-28 Thread jiangxb1987
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/15672 [SPARK-18148][SQL] Misleading Error Message for Aggregation Without Window/GroupBy ## What changes were proposed in this pull request? Aggregation Without Window/GroupBy expressions

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67700/ Test FAILed. ---

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67700/consoleFull)** for PR 15671 at commit

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85509442 --- Diff: pom.xml --- @@ -26,6 +26,7 @@ org.apache.spark spark-parent_2.11 + --- End diff -- So during the binary

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-28 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r85509116 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -1016,6 +1016,8 @@ class

[GitHub] spark issue #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to be pip i...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15659 **[Test build #67701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67701/consoleFull)** for PR 15659 at commit

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85508101 --- Diff: dev/make-distribution.sh --- @@ -201,6 +201,12 @@ fi # Copy data files cp -r "$SPARK_HOME/data" "$DISTDIR" +# Make pip package

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85507888 --- Diff: dev/create-release/release-build.sh --- @@ -187,10 +208,10 @@ if [[ "$1" == "package" ]]; then # We increment the Zinc port each time to

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85507910 --- Diff: bin/spark-shell --- @@ -29,7 +29,7 @@ esac set -o posix if [ -z "${SPARK_HOME}" ]; then - export SPARK_HOME="$(cd "`dirname

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85507670 --- Diff: dev/run-tests.py --- @@ -583,6 +589,7 @@ def main(): modules_with_python_tests = [m for m in test_modules if m.python_test_goals]

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85507586 --- Diff: docs/building-spark.md --- @@ -259,6 +259,12 @@ or Java 8 tests are automatically enabled when a Java 8 JDK is detected. If you have JDK

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to MLlib train...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67700/consoleFull)** for PR 15671 at commit

[GitHub] spark pull request #15671: [SPARK-14567][ML]Add instrumentation logs to MLli...

2016-10-28 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/15671 [SPARK-14567][ML]Add instrumentation logs to MLlib training algorithms ## What changes were proposed in this pull request? 1, add instrumentation for

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15669 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67699/ Test PASSed. ---

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15669 **[Test build #67699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67699/consoleFull)** for PR 15669 at commit

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15668 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67698/ Test PASSed. ---

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15668 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15668 **[Test build #67698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67698/consoleFull)** for PR 15668 at commit

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15523 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15523 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67697/ Test PASSed. ---

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15523 **[Test build #67697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67697/consoleFull)** for PR 15523 at commit

[GitHub] spark issue #15660: [SPARK-18133][Examples] [ML] [Python ML Pipeline Example...

2016-10-28 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15660 @jagadeesanas2 I'd like to merge it into branch-2.0 also, but there is conflict. Could you please prepare a separate PR for branch-2.0? Thanks. --- If your project is set up for it, you can

[GitHub] spark pull request #15660: [SPARK-18133][Examples] [ML] [Python ML Pipeline ...

2016-10-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15660 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15660: [SPARK-18133][Examples] [ML] [Python ML Pipeline Example...

2016-10-28 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15660 LGTM2, merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15513 Thanks @gatorsmile. Just FYI, I would like to note the rule I used for argument types (just to avoid extra efforts when you review). As we all know, I did not mention implicit casting

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85491680 --- Diff: dev/run-tests.py --- @@ -583,6 +589,7 @@ def main(): modules_with_python_tests = [m for m in test_modules if m.python_test_goals]

[GitHub] spark issue #15667: [SPARK-18107][SQL] Insert overwrite statement runs much ...

2016-10-28 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15667 I checked the failed test. Special characters in the partition path cause the failure, e..g., alter table ppr_test add partition (ds = '12:4'); alter table ppr_test add partition

[GitHub] spark pull request #15665: [SPARK-18146] [SQL] Avoid using Union to chain to...

2016-10-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15665#discussion_r85490391 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -254,7 +254,8 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2016-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15670 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15667: [SPARK-18107][SQL] Insert overwrite statement runs much ...

2016-10-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15667 cc @ericl --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15670: [SPARK-18161] [Python] Allow use of pickle to ser...

2016-10-28 Thread singularperturbation
GitHub user singularperturbation opened a pull request: https://github.com/apache/spark/pull/15670 [SPARK-18161] [Python] Allow use of pickle to serialize > 4 GB objects where available ## What changes were proposed in this pull request? Since pickle protocol 4 (available

[GitHub] spark issue #15634: [SPARK-18103] [SQL] Rename *FileCatalog to *FileProvider

2016-10-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15634 FileIndex sounds good to me. I wouldn't call it "Metastore" though, since that is a Hive specific term. I'd call it Catalog. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #15024: [SPARK-17470][SQL] unify path for data source tab...

2016-10-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15024#discussion_r85488796 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -415,16 +422,34 @@ private[spark] class

[GitHub] spark pull request #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect N...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15523#discussion_r85488370 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -1617,6 +1617,72 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15669 **[Test build #67699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67699/consoleFull)** for PR 15669 at commit

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85486513 --- Diff: dev/pip-sanity-check.py --- @@ -0,0 +1,36 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #15659: [WIP][SPARK-1267][SPARK-18129] Allow PySpark to b...

2016-10-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15659#discussion_r85486184 --- Diff: dev/make-distribution.sh --- @@ -201,6 +201,12 @@ fi # Copy data files cp -r "$SPARK_HOME/data" "$DISTDIR" +# Make pip package

[GitHub] spark issue #14640: [SPARK-17055] [MLLIB] add groupKFold to CrossValidator

2016-10-28 Thread rdelassus
Github user rdelassus commented on the issue: https://github.com/apache/spark/pull/14640 There is an infinite number of ways to make folds. Until now we had the mlutils kfold. You want to add the groupedKfold. But I don't think we should add one by one every folding method that can

[GitHub] spark pull request #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile do...

2016-10-28 Thread zjffdu
GitHub user zjffdu opened a pull request: https://github.com/apache/spark/pull/15669 [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't work in yarn-cluster mode ## What changes were proposed in this pull request? Will fallback to container working directory if the

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15668 **[Test build #67698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67698/consoleFull)** for PR 15668 at commit

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-10-28 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15668 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

<    1   2   3   4   5   >