[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80710/ Test PASSed. ---

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #80710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80710/testReport)** for PR 18953 at commit [`22dbe35`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18956 Interesting, existing `PullupCorrelatedPredicates` produces unresolved plan. I'll figure out the reason. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80718/ Test FAILed. ---

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80718/testReport)** for PR 18956 at commit [`c99011d`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80717/testReport)** for PR 18956 at commit [`9170ceb`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #18955: [SPARK-21743][SQL] top-most limit should not cause memor...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18955 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80713/ Test FAILed. ---

[GitHub] spark pull request #18953: [SPARK-20682][SQL] Implement new ORC data source ...

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18953#discussion_r133368809 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala --- @@ -343,7 +343,7 @@ class OrcQuerySuite extends QueryTest with

[GitHub] spark issue #18955: [SPARK-21743][SQL] top-most limit should not cause memor...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18955 **[Test build #80713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80713/testReport)** for PR 18955 at commit [`67ac3aa`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80717/ Test FAILed. ---

[GitHub] spark issue #18955: [SPARK-21743][SQL] top-most limit should not cause memor...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18955 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18953: [SPARK-20682][SQL] Implement new ORC data source ...

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18953#discussion_r133368613 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -47,11 +47,11 @@ import org.apache.spark.util.SerializableC

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18953: [SPARK-20682][SQL] Implement new ORC data source ...

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18953#discussion_r133368561 --- Diff: sql/hive/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister --- @@ -1,2 +1,2 @@ -org.apache.spark.sql

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80715/ Test FAILed. ---

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18956 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80715/testReport)** for PR 18956 at commit [`21d86ba`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #80721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80721/testReport)** for PR 18953 at commit [`07778ed`](https://github.com/apache/spark/commit/07

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18953 Rebased to the master since #18640 is merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarchy to ma...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18958 **[Test build #80720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80720/testReport)** for PR 18958 at commit [`cd0de39`](https://github.com/apache/spark/commit/cd

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18640 Thank you, @gatorsmile !!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark issue #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarchy to ma...

2017-08-15 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18958 cc @cloud-fan @BryanCutler --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark pull request #18958: [SPARK-21745][SQL] Refactor ColumnVector hierarch...

2017-08-15 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/18958 [SPARK-21745][SQL] Refactor ColumnVector hierarchy to make ColumnVector read-only and to introduce MutableColumnVector. ## What changes were proposed in this pull request? This is a refacto

[GitHub] spark issue #18315: [SPARK-21108] [ML] [WIP] convert LinearSVC to aggregator...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18315 **[Test build #80719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80719/testReport)** for PR 18315 at commit [`94e0250`](https://github.com/apache/spark/commit/94

[GitHub] spark pull request #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18640 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18315: [SPARK-21108] [ML] [WIP] convert LinearSVC to aggregator...

2017-08-15 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18315 @hhbyyh Would you mind to remove ```WIP``` in the PR title if it's applicable. I'll take a look soon. Thanks. --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18640 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #18315: [SPARK-21108] [ML] [WIP] convert LinearSVC to aggregator...

2017-08-15 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18315 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-08-15 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/17862 cc @WeichenXu123 What do you think about this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18926 The current codes around what this PR changes look not quite clean to me too and we should clean around this. But I think this PR itself is quite well-formed with the fix that is valid,

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18640 Thank you so much, @rxin , @cloud-fan , @sameeragarwal , @mridulm , @viirya ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80718/testReport)** for PR 18956 at commit [`c99011d`](https://github.com/apache/spark/commit/c9

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18926 To be honest, the current codes do not look good to me. Since this does not make the code worse, I will not revert it back. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-08-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r133360995 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -37,6 +37,12 @@ import org.apache.spark.sql.types._

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-15 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r133360674 --- Diff: mllib/src/test/scala/org/apache/spark/ml/evaluation/ClusteringEvaluatorSuite.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apach

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18640 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #18957: [SPARK-21744][CORE] Add retry logic for new broadcast in...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18957 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-15 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r133360284 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,240 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #18957: [SPARK-21744][CORE] Add retry logic for new broad...

2017-08-15 Thread caneGuy
GitHub user caneGuy opened a pull request: https://github.com/apache/spark/pull/18957 [SPARK-21744][CORE] Add retry logic for new broadcast in BroadcastManager ## What changes were proposed in this pull request? When driver submit new stage and there is a bad disk before spa

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80717/testReport)** for PR 18956 at commit [`9170ceb`](https://github.com/apache/spark/commit/91

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-08-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r133360047 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -37,6 +37,12 @@ import org.apache.spark.sql.types._

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80715/testReport)** for PR 18956 at commit [`21d86ba`](https://github.com/apache/spark/commit/21

[GitHub] spark issue #18855: [SPARK-3151] [Block Manager] DiskStore.getBytes fails fo...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18855 **[Test build #80716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80716/testReport)** for PR 18855 at commit [`732073c`](https://github.com/apache/spark/commit/73

[GitHub] spark pull request #18955: [SPARK-21743][SQL] top-most limit should not caus...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18955#discussion_r133359698 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2658,4 +2658,9 @@ class SQLQuerySuite extends QueryTest with SharedSQ

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-08-15 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/18956 [SPARK-21726][SQL] Check for structural integrity of the plan in Optimzer in test mode. ## What changes were proposed in this pull request? We have many optimization rules now in `Optimzer`

[GitHub] spark issue #18492: [SPARK-19326] Speculated task attempts do not get launch...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18492 **[Test build #80714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80714/testReport)** for PR 18492 at commit [`8b8b128`](https://github.com/apache/spark/commit/8b

[GitHub] spark issue #18955: [SPARK-21743][SQL] top-most limit should not cause memor...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18955 **[Test build #80713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80713/testReport)** for PR 18955 at commit [`67ac3aa`](https://github.com/apache/spark/commit/67

[GitHub] spark issue #18955: [SPARK-21743][SQL] top-most limit should not cause memor...

2017-08-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18955 cc @gengliangwang @sameeragarwal @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request #18955: [SPARK-21743][SQL] top-most limit should not caus...

2017-08-15 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/18955 [SPARK-21743][SQL] top-most limit should not cause memory leak ## What changes were proposed in this pull request? For top-most limit, we will use a special operator to execute it: `Coll

[GitHub] spark issue #18492: [SPARK-19326] Speculated task attempts do not get launch...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18492 **[Test build #80712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80712/testReport)** for PR 18492 at commit [`f7cdad9`](https://github.com/apache/spark/commit/f7

[GitHub] spark issue #18954: [SPARK-17654] [SQL] Enable creating hive bucketed tables

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18954 **[Test build #80711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80711/testReport)** for PR 18954 at commit [`4b009a9`](https://github.com/apache/spark/commit/4b

[GitHub] spark issue #18954: [SPARK-17654] [SQL] Enable creating hive bucketed tables

2017-08-15 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/18954 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark pull request #18954: [SPARK-17654] [SQL] Enable creating hive bucketed...

2017-08-15 Thread tejasapatil
GitHub user tejasapatil opened a pull request: https://github.com/apache/spark/pull/18954 [SPARK-17654] [SQL] Enable creating hive bucketed tables ## What changes were proposed in this pull request? ### Semantics: - If the Hive table is bucketed, then INSERT node expect

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80707/ Test FAILed. ---

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #80707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80707/testReport)** for PR 18953 at commit [`051ed1f`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-15 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r133355548 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -291,6 +297,16 @@ private[spark] trait SparkListenerInterface {

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #80710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80710/testReport)** for PR 18953 at commit [`22dbe35`](https://github.com/apache/spark/commit/22

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18896 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80708/ Test PASSed. ---

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18896 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18896 **[Test build #80708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80708/testReport)** for PR 18896 at commit [`2eda876`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18951: [SPARK-21738] Thriftserver doesn't cancel jobs when sess...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18951 LGTM cc @cloud-fan @jiangxb1987 @wangyum @debugger87 @jerryshao --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18640 Hi, @cloud-fan , @rxin , @sameeragarwal and @mridulm . Could you merge this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18810 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark issue #18949: [SPARK-12961][CORE][FOLLOW-UP] Remove wrapper code for S...

2017-08-15 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18949 @viirya aha, ok. thanks. (btw, since the comment is still important, we better keep it in code comment, maybe). --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16763 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16763 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80709/ Test FAILed. ---

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16763 **[Test build #80709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80709/testReport)** for PR 16763 at commit [`1742c15`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #18949: [SPARK-12961][CORE][FOLLOW-UP] Remove wrapper code for S...

2017-08-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18949 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] spark issue #18949: [SPARK-12961][CORE][FOLLOW-UP] Remove wrapper code for S...

2017-08-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18949 @maropu There is another reason we leave the workaround in place: https://github.com/apache/spark/pull/11524#issuecomment-192409933 --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-15 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/18902 @hhbyyh I rewrite the impl, and now all `NaN` and `missingValue` will be transform to `null` at first, then current methods are used. For columns only containing `null`, `null` is returned

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16763 **[Test build #80709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80709/testReport)** for PR 16763 at commit [`1742c15`](https://github.com/apache/spark/commit/17

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16763 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18798 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-15 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18798 Merged into master, thanks for all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18896 **[Test build #80708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80708/testReport)** for PR 18896 at commit [`2eda876`](https://github.com/apache/spark/commit/2e

[GitHub] spark pull request #18930: [SPARK-21677][SQL] json_tuple throws NullPointExc...

2017-08-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18930#discussion_r133347400 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2034,4 +2034,25 @@ class JsonSuite extends QueryTe

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18926 Merged to master. Please open JIRAs / PRs related with the discussion above if anyone is willing to proceed. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #18926: [SPARK-21712] [PySpark] Clarify type error for Co...

2017-08-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18926 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18926 I am merging this as it looks there is an explicit objection for the current change itself and it looks the issue is fixed by this. To summarize the discussion here: - Cleaning

[GitHub] spark pull request #18950: [SPARK-20589][Core][Scheduler] Allow limiting tas...

2017-08-15 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/18950#discussion_r133344532 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -602,6 +604,21 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80703/ Test PASSed. ---

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80703/testReport)** for PR 18810 at commit [`44ce894`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #80707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80707/testReport)** for PR 18953 at commit [`051ed1f`](https://github.com/apache/spark/commit/05

[GitHub] spark pull request #18953: [SPARK-20682][SQL] Implement new ORC data source ...

2017-08-15 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/18953 [SPARK-20682][SQL] Implement new ORC data source based on Apache ORC ## What changes were proposed in this pull request? Since #17924, #17943, and #17980 are a little large PRs, this

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80706/ Test FAILed. ---

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12646 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12646 **[Test build #80706 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80706/testReport)** for PR 12646 at commit [`5e155bd`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80701/ Test PASSed. ---

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18887 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18887 **[Test build #80701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80701/testReport)** for PR 18887 at commit [`519dab0`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/18896 LGTM except for making the test's title more descriptive. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark issue #18488: [SPARK-21255][SQL][WIP] Fixed NPE when creating encoder ...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18488 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80700/ Test PASSed. ---

[GitHub] spark issue #18488: [SPARK-21255][SQL][WIP] Fixed NPE when creating encoder ...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18488 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18488: [SPARK-21255][SQL][WIP] Fixed NPE when creating encoder ...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18488 **[Test build #80700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80700/testReport)** for PR 18488 at commit [`fbdc599`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15770 info] Main Scala API documentation successful. [error] (spark/javaunidoc:doc) javadoc returned nonzero exit code [error] Total time: 95 s, completed Aug 15, 2017 4:59:59 PM [error] ru

[GitHub] spark pull request #18923: [SPARK-21710][StSt] Fix OOM on ConsoleSink with l...

2017-08-15 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/18923#discussion_r15831 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/console.scala --- @@ -49,7 +49,7 @@ class ConsoleSink(options: Map[String, Strin

  1   2   3   4   5   >