[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18460 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18873 Ah right this is open vs 2.1. That much makes sense, disregard me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18460 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80352/ Test PASSed. ---

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131731670 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -366,11 +370,15 @@ final class DataFrameNaFunctions private[sql

[GitHub] spark issue #18421: [SPARK-21213][SQL] Support collecting partition-level st...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18421 **[Test build #80353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80353/testReport)** for PR 18421 at commit [`2233b2c`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18421: [SPARK-21213][SQL] Support collecting partition-level st...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18421 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80353/ Test PASSed. ---

[GitHub] spark pull request #18421: [SPARK-21213][SQL] Support collecting partition-l...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18421#discussion_r131732850 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -37,31 +37,19 @@ case class AnalyzeTableComman

[GitHub] spark issue #18421: [SPARK-21213][SQL] Support collecting partition-level st...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18421 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18421: [SPARK-21213][SQL] Support collecting partition-l...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18421#discussion_r131733047 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -19,7 +19,7 @@ package org.apache.spark.sql.ex

[GitHub] spark issue #18801: SPARK-10878 Fix race condition when multiple clients res...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18801 **[Test build #80351 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80351/testReport)** for PR 18801 at commit [`1ace5cc`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #18874: [SPARK-21656][CORE] spark dynamic allocation shou...

2017-08-07 Thread yoonlee95
GitHub user yoonlee95 opened a pull request: https://github.com/apache/spark/pull/18874 [SPARK-21656][CORE] spark dynamic allocation should not idle timeout executors when tasks still to run ## What changes were proposed in this pull request? Right now spark lets go of exec

[GitHub] spark issue #18801: SPARK-10878 Fix race condition when multiple clients res...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18801 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80351/ Test PASSed. ---

[GitHub] spark issue #18801: SPARK-10878 Fix race condition when multiple clients res...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18801 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18874 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-07 Thread goldmedal
GitHub user goldmedal opened a pull request: https://github.com/apache/spark/pull/18875 [SPARK-21513][SQL] Allow UDF to_json support converting MapType to json # What changes were proposed in this pull request? UDF to_json only supports converting `StructType` or `ArrayType` of

[GitHub] spark pull request #18798: [WIP] [SPARK-19634][ML] Multivariate summarizer -...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r131735748 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -547,35 +533,11 @@ object SummaryBuilderImpl extends Logging {

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #18814: [SPARK-21608][SPARK-9221][SQL] Window rangeBetween() API...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18814 **[Test build #80354 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80354/testReport)** for PR 18814 at commit [`3b3d822`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #18814: [SPARK-21608][SPARK-9221][SQL] Window rangeBetween() API...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18814 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18814: [SPARK-21608][SPARK-9221][SQL] Window rangeBetween() API...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18814 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80354/ Test PASSed. ---

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18874 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18742 **[Test build #80361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80361/testReport)** for PR 18742 at commit [`d62154c`](https://github.com/apache/spark/commit/d6

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18874 **[Test build #80360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80360/testReport)** for PR 18874 at commit [`3c2cf90`](https://github.com/apache/spark/commit/3c

[GitHub] spark issue #18840: [SPARK-21565][SS] Propagate metadata in attribute replac...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18840 **[Test build #80356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80356/testReport)** for PR 18840 at commit [`578e26d`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18840: [SPARK-21565][SS] Propagate metadata in attribute replac...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18840 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80356/ Test PASSed. ---

[GitHub] spark issue #18840: [SPARK-21565][SS] Propagate metadata in attribute replac...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18840 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18840: [SPARK-21565][SS] Propagate metadata in attribute replac...

2017-08-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18840 Thanks! Merging to master and branch-2.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18474: [SPARK-21235][TESTS] UTest should clear temp results whe...

2017-08-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18474 In my Windows10 environment, to leave temp result does not occur for `"on-disk storage"`. It happens for `"SPARK-20640: Shuffle registration timeout and maxAttempts conf are working"`. I am lookin

[GitHub] spark pull request #18840: [SPARK-21565][SS] Propagate metadata in attribute...

2017-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18840 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18850: Remove dropping bus logic.

2017-08-07 Thread dirkraft
Github user dirkraft closed the pull request at: https://github.com/apache/spark/pull/18850 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark issue #18850: Remove dropping bus logic.

2017-08-07 Thread dirkraft
Github user dirkraft commented on the issue: https://github.com/apache/spark/pull/18850 Closing out this noise for now. Discussions continue in https://issues.apache.org/jira/browse/SPARK-18838 --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request #18876: WriteStatsTracker API

2017-08-07 Thread adrian-ionescu
GitHub user adrian-ionescu opened a pull request: https://github.com/apache/spark/pull/18876 WriteStatsTracker API WIP You can merge this pull request into a Git repository by running: $ git pull https://github.com/adrian-ionescu/apache-spark write-stats-tracker-api Alternati

[GitHub] spark pull request #18876: WriteStatsTracker API

2017-08-07 Thread adrian-ionescu
Github user adrian-ionescu closed the pull request at: https://github.com/apache/spark/pull/18876 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] spark issue #18474: [SPARK-21235][TESTS] UTest should clear temp results whe...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18474 @wangjiaochun, let's close this if you are unable to explain why this occurs in a certain environment. Looks hard to reproduce. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #18474: [SPARK-21235][TESTS] UTest should clear temp results whe...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18474 @wangjiaochun, let's close this if you are unable to explain why this happens in a certain environment. Looks hard to reproduce. --- If your project is set up for it, you can reply to this emai

[GitHub] spark issue #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread dmvieira
Github user dmvieira commented on the issue: https://github.com/apache/spark/pull/18873 removed other PR related code --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet to proxy...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18499 **[Test build #80355 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80355/testReport)** for PR 18499 at commit [`70c93b0`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet to proxy...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80355/ Test PASSed. ---

[GitHub] spark issue #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet to proxy...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18499 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18873#discussion_r131746369 --- Diff: dev/run-tests.py --- @@ -121,7 +121,7 @@ def determine_modules_to_test(changed_modules): if modules.root in modules_to_test:

[GitHub] spark issue #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18873 Is this a backport of another fix? If so you the PR title / summary should reflect that. If not, you need to open a bug and provide a better explanation in the PR title / summary. S

[GitHub] spark issue #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18873 @vanzin, I remember [this issue](http://apache-spark-developers-list.1001551.n3.nabble.com/Tests-failing-with-run-tests-py-SyntaxError-td22030.html) was fixed roughly 5 days ago, which might be

[GitHub] spark issue #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18873 Tests are already running on the PR, so we can see whether things still fail. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-08-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r131749037 --- Diff: python/pyspark/ml/util.py --- @@ -283,3 +333,201 @@ def numFeatures(self): Returns the number of features the model was trained on.

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-08-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r131749278 --- Diff: python/pyspark/ml/util.py --- @@ -283,3 +333,201 @@ def numFeatures(self): Returns the number of features the model was trained on.

[GitHub] spark issue #18864: [SPARK-21648] [SQL] Fix confusing assert failure in JDBC...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18864 @dongjoon-hyun `paritionColumn` is wrong. It lacks a `t`. That is why the existing assert is pretty confusing. --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark pull request #18849: [SPARK-21617][SQL] Store correct table metadata w...

2017-08-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18849#discussion_r131750071 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -611,21 +640,39 @@ private[spark] class HiveExternalCatalog(conf

[GitHub] spark issue #18864: [SPARK-21648] [SQL] Fix confusing assert failure in JDBC...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18864 Thanks! Merging to master/2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark issue #18864: [SPARK-21648] [SQL] Fix confusing assert failure in JDBC...

2017-08-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18864 Oh, I see. It's about typo. Thank you for fixing the error message. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request #18864: [SPARK-21648] [SQL] Fix confusing assert failure ...

2017-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18864 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18692 **[Test build #80362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80362/testReport)** for PR 18692 at commit [`281f955`](https://github.com/apache/spark/commit/28

[GitHub] spark pull request #18869: [SPARK-21654][SQL] Complement SQL predicates expr...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18869#discussion_r131754182 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -484,7 +501,20 @@ object Equality { }

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18869 Could you check all of binary predicates you are trying to modify? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18798 **[Test build #80363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80363/testReport)** for PR 18798 at commit [`6053d0e`](https://github.com/apache/spark/commit/60

[GitHub] spark issue #18607: [SPARK-21362][SQL][Adding Apache Drill JDBC Dialect]

2017-08-07 Thread radford1
Github user radford1 commented on the issue: https://github.com/apache/spark/pull/18607 @viirya I have not had a chance to build out the docker tests like @gatorsmile suggested. Will that be a requirement in order to merge it? --- If your project is set up for it, you can reply to th

[GitHub] spark issue #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Redact se...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18802 **[Test build #80357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80357/testReport)** for PR 18802 at commit [`7b419b4`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18742 **[Test build #80361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80361/testReport)** for PR 18742 at commit [`d62154c`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80361/ Test PASSed. ---

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18742 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Redact se...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18802 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Redact se...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18802 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80357/ Test PASSed. ---

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18742 **[Test build #80364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80364/testReport)** for PR 18742 at commit [`860132e`](https://github.com/apache/spark/commit/86

[GitHub] spark issue #18851: [SPARK-21644][SQL] LocalLimit.maxRows is defined incorre...

2017-08-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18851 Looks like the strip global limit is used by at least some test cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131756595 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null|

[GitHub] spark issue #18607: [SPARK-21362][SQL][Adding Apache Drill JDBC Dialect]

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18607 @radford1 Yes. That is needed for merging this PR. Could you check it? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #18855: [SPARK-3151] [Block Manager] DiskStore.getBytes f...

2017-08-07 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/18855#discussion_r131758308 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -165,6 +147,62 @@ private[spark] class DiskStore( } +privat

[GitHub] spark pull request #18790: [SPARK-21587][SS] Added pushdown through watermar...

2017-08-07 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/18790#discussion_r131761258 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -872,6 +886,25 @@ object PushDownPredicate exten

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18790 **[Test build #80365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80365/testReport)** for PR 18790 at commit [`f0dd408`](https://github.com/apache/spark/commit/f0

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18798 **[Test build #80359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80359/testReport)** for PR 18798 at commit [`4f32e27`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18798 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18798 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80359/ Test PASSed. ---

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18742 **[Test build #80366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80366/testReport)** for PR 18742 at commit [`2f216b7`](https://github.com/apache/spark/commit/2f

[GitHub] spark issue #18849: [SPARK-21617][SQL] Store correct table metadata when alt...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18849 **[Test build #80358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80358/testReport)** for PR 18849 at commit [`40ebc96`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #18849: [SPARK-21617][SQL] Store correct table metadata when alt...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18849 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80358/ Test PASSed. ---

[GitHub] spark issue #18849: [SPARK-21617][SQL] Store correct table metadata when alt...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18849 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #14326: [SPARK-3181] [ML] Implement RobustRegression with...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/14326#discussion_r131762320 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/RobustRegression.scala --- @@ -0,0 +1,497 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request #14326: [SPARK-3181] [ML] Implement RobustRegression with...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/14326#discussion_r131764683 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/RobustRegression.scala --- @@ -0,0 +1,466 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request #14326: [SPARK-3181] [ML] Implement RobustRegression with...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/14326#discussion_r131763824 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/RobustRegression.scala --- @@ -0,0 +1,497 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131766593 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -746,8 +746,20 @@ abstract class Re

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131766797 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2662,6 +2662,30 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark issue #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Redact se...

2017-08-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18802 Merging to 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Redact se...

2017-08-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18802 @dmvieira please close the PR since github doesn't do it automatically. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #15770: [SPARK-15784][ML]:Add Power Iteration Clustering ...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/15770#discussion_r131767248 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/PowerIterationClustering.scala --- @@ -0,0 +1,213 @@ +/* + * Licensed to the Apa

[GitHub] spark pull request #15770: [SPARK-15784][ML]:Add Power Iteration Clustering ...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/15770#discussion_r131766119 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/PowerIterationClustering.scala --- @@ -0,0 +1,213 @@ +/* + * Licensed to the Apa

[GitHub] spark pull request #15770: [SPARK-15784][ML]:Add Power Iteration Clustering ...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/15770#discussion_r131766525 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/PowerIterationClustering.scala --- @@ -0,0 +1,213 @@ +/* + * Licensed to the Apa

[GitHub] spark issue #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18873 @dmvieira the other PR passed tests, doesn't seem like this is necessary. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131768761 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -596,14 +596,15 @@ object CollapseProject extends R

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18874 **[Test build #80360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80360/testReport)** for PR 18874 at commit [`3c2cf90`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18742 **[Test build #80364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80364/testReport)** for PR 18742 at commit [`860132e`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18874 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80360/ Test FAILed. ---

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18874 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80364/ Test PASSed. ---

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18742 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18818: [SPARK-21110][SQL] Structs, arrays, and other ord...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18818#discussion_r131771278 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -453,6 +453,14 @@ case class Or(left: Expression

[GitHub] spark issue #18764: [SPARK-21306][ML] For branch 2.0, OneVsRest should suppo...

2017-08-07 Thread facaiy
Github user facaiy commented on the issue: https://github.com/apache/spark/pull/18764 Thanks, @yanboliang @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r131771002 --- Diff: python/pyspark/ml/tests.py --- @@ -1158,6 +1165,33 @@ def test_decisiontree_regressor(self): except OSError: pas

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r131772274 --- Diff: python/pyspark/ml/util.py --- @@ -61,33 +66,86 @@ def _randomUID(cls): @inherit_doc -class MLWriter(object): +class B

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18742 **[Test build #80366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80366/testReport)** for PR 18742 at commit [`2f216b7`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18742 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18742: [Spark-21542][ML][Python]Python persistence helper funct...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80366/ Test PASSed. ---

<    1   2   3   4   5   6   >