[GitHub] spark issue #14511: [SPARK-16927] [SPARK-16923] Override task properties at ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14511 **[Test build #63464 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63464/consoleFull)** for PR 14511 at commit [`81f2b1c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14527: [SPARK-16938][SQL] `dropDuplicate` should not raise exce...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63468/ Test PASSed. ---

[GitHub] spark issue #14527: [SPARK-16938][SQL] `dropDuplicate` should not raise exce...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14527 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14527: [SPARK-16938][SQL] `dropDuplicate` should not raise exce...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14527 **[Test build #63468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63468/consoleFull)** for PR 14527 at commit [`fdeb02e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14571 **[Test build #63477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63477/consoleFull)** for PR 14571 at commit [`2c875dc`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #14571: [SPARK-16983][SQL] Add `prettyName` for row_numbe...

2016-08-09 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/14571 [SPARK-16983][SQL] Add `prettyName` for row_number, dense_rank, percent_rank, cume_dist ## What changes were proposed in this pull request? Currently, two-word window functions like

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63470/ Test FAILed. ---

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63470 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63470/consoleFull)** for PR 14568 at commit [`1107182`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74159170 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedu

[GitHub] spark issue #14570: [SPARK-10601][SQL] Support `MINUS` set operator

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14570 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14570: [SPARK-10601][SQL] Support `MINUS` set operator

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14570 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63463/ Test PASSed. ---

[GitHub] spark pull request #14524: [SPARK-16832] [ML] [WIP] CrossValidator and Train...

2016-08-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/14524#discussion_r74159065 --- Diff: python/pyspark/ml/param/shared.py --- @@ -311,7 +311,7 @@ class HasSeed(Params): def __init__(self): super(HasSeed,

[GitHub] spark issue #14570: [SPARK-10601][SQL] Support `MINUS` set operator

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14570 **[Test build #63463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63463/consoleFull)** for PR 14570 at commit [`0cf8f43`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14560: [SPARK-16971][SQL] Strip trailing zeros for decimal's st...

2016-08-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14560 (BTW, it seems this is what hive does) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14472: [SPARK-16866][SQL] Infrastructure for file-based SQL end...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14472 **[Test build #63476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63476/consoleFull)** for PR 14472 at commit [`5eb01fe`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14569: Fixed typo

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63461/ Test PASSed. ---

[GitHub] spark issue #14569: Fixed typo

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14569 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14569: Fixed typo

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14569 **[Test build #63461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63461/consoleFull)** for PR 14569 at commit [`1b17aff`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14552: [SPARK-16952] don't lookup spark home directory when exe...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14552 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63460/ Test PASSed. ---

[GitHub] spark issue #14552: [SPARK-16952] don't lookup spark home directory when exe...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14552 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14552: [SPARK-16952] don't lookup spark home directory when exe...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14552 **[Test build #63460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63460/consoleFull)** for PR 14552 at commit [`880a1f4`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

2016-08-09 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/14524 +1 for changing this. If left as is, JIRAs like this one will be a recurring theme. A lot of users would probably not even notice, then wind up getting degraded results and not know why. ---

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63475/consoleFull)** for PR 14568 at commit [`4d0cd3a`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74158089 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedule

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74158079 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedule

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74157953 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedule

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63469/ Test FAILed. ---

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63469 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63469/consoleFull)** for PR 14568 at commit [`f78d6aa`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74156824 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedu

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost [WIP]

2016-08-09 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/14547 CC: @hhbyyh Would you mind taking a look at this since you're familiar with GBTs? Thanks in advance! This should be one of the most important improvements in terms of accuracy, especially once w

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost [WIP]

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14547 **[Test build #3210 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3210/consoleFull)** for PR 14547 at commit [`fe256f7`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74156582 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedu

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74156410 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedu

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63466/ Test FAILed. ---

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63466/consoleFull)** for PR 14568 at commit [`81e342d`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74155413 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedu

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost [WIP]

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14547 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63447/ Test FAILed. ---

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost [WIP]

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14547 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost [WIP]

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14547 **[Test build #63447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63447/consoleFull)** for PR 14547 at commit [`fe256f7`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor po...

2016-08-09 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/11157#discussion_r74155283 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -358,6 +376,119 @@ private[mesos] trait MesosSchedu

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14537 **[Test build #63474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63474/consoleFull)** for PR 14537 at commit [`75bca1b`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63473/consoleFull)** for PR 14568 at commit [`b713f23`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14397 Thank you so much! Then, see you later. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-09 Thread rajeshbalamohan
Github user rajeshbalamohan commented on the issue: https://github.com/apache/spark/pull/14537 Thanks @mallman . Fixed review comments in latest commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-08-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14397 @dongjoon-hyun I'll take a look in the morning (CET) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14531 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63457/ Test PASSed. ---

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14531 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14531 **[Test build #63457 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63457/consoleFull)** for PR 14531 at commit [`6180e80`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63472/consoleFull)** for PR 14568 at commit [`8198d9c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63472/ Test FAILed. ---

[GitHub] spark pull request #14397: [SPARK-16771][SQL] WITH clause should not fall in...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14397#discussion_r74153582 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -125,21 +125,23 @@ class Analyzer( object CT

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14397 Could you review this PR again, @hvanhovell ? Last time, I did a huge mistake, but now I think I fixed correct according to your advice. --- If your project is set up for it, you can repl

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63472 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63472/consoleFull)** for PR 14568 at commit [`8198d9c`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #14258: [Spark-16579][SparkR] add install.spark function

2016-08-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14258#discussion_r74152794 --- Diff: R/pkg/R/install.R --- @@ -0,0 +1,230 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor lice

[GitHub] spark pull request #12990: [SPARK-15083] [Web UI] Limit total tasks displaye...

2016-08-09 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/12990#discussion_r74151754 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -137,6 +138,17 @@ class JobProgressListener(conf: SparkConf) extend

[GitHub] spark issue #14527: [SPARK-16938][SQL] `dropDuplicate` should not raise exce...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14527 Hi, @cloud-fan . Could you review this PR when you have some time? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark issue #14472: [SPARK-16866][SQL] Infrastructure for file-based SQL end...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14472 **[Test build #63471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63471/consoleFull)** for PR 14472 at commit [`0359756`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63470 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63470/consoleFull)** for PR 14568 at commit [`1107182`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63459/ Test FAILed. ---

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63459 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63459/consoleFull)** for PR 14568 at commit [`46ea8a4`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14472: [SPARK-16866][SQL] Infrastructure for file-based SQL end...

2016-08-09 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/14472 I have updated this based on review feedback. The harness is now using hiveResultString() rather than show(), similar to the existing Hive compatibility test. --- If your project is set up fo

[GitHub] spark issue #12990: [SPARK-15083] [Web UI] Limit total tasks displayed on We...

2016-08-09 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/12990 I'll create a new pr based on this one with a higher default max tasks (maybe 10,000?) and an on/off config when I have time this week --- If your project is set up for it, you can reply to this

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63469/consoleFull)** for PR 14568 at commit [`f78d6aa`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14568 So I'm still confused what offset does, even after reading the code. Can you please write a more clear documentation? An example would help. Also let's update the Python API as well, and add an

[GitHub] spark pull request #14558: [SPARK-16508][SparkR] Fix warnings on undocumente...

2016-08-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14558#discussion_r74146966 --- Diff: .gitignore --- @@ -77,3 +77,8 @@ spark-warehouse/ # For R session data .RData .RHistory +.Rhistory --- End diff -- Du

[GitHub] spark issue #14527: [SPARK-16938][SQL] `dropDuplicate` should not raise exce...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14527 **[Test build #63468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63468/consoleFull)** for PR 14527 at commit [`fdeb02e`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14568 A high-level question: is it a problem when the offset is larger than `1 << 33`? I can't really think of one. --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark pull request #14568: [SPARK-10868] monotonicallyIncreasingId() support...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14568#discussion_r74145985 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/MonotonicallyIncreasingID.scala --- @@ -40,13 +40,14 @@ import org.apache.spark

[GitHub] spark pull request #14568: [SPARK-10868] monotonicallyIncreasingId() support...

2016-08-09 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14568#discussion_r74145557 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -993,12 +993,31 @@ object functions { * This expression would return

[GitHub] spark issue #14570: [SPARK-10601][SQL] Support `MINUS` set operator

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14570 Thank you so much, @hvanhovell ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark issue #14570: [SPARK-10601][SQL] Support `MINUS` set operator

2016-08-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14570 LGTM - I'll merge as soon as test pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-08-09 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14431 Sure - Appending more information to the R object is fine. Also it looks like we actually have a handle to the RelationalGroupedDataset when we call groupBy on the scala side https://github.com/a

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13680 **[Test build #63467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63467/consoleFull)** for PR 13680 at commit [`d9a269d`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63466/consoleFull)** for PR 14568 at commit [`81e342d`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #14570: [SPARK-10601][SQL] Support `MINUS` set operator

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14570 Could you review this PR when you have some time, @hvanhovell ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14475: [SPARK-16862] Configurable buffer size in `UnsafeSorterS...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14475 **[Test build #63465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63465/consoleFull)** for PR 14475 at commit [`6ce70a9`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14511: [SPARK-16927] [SPARK-16923] Override task properties at ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14511 **[Test build #63464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63464/consoleFull)** for PR 14511 at commit [`81f2b1c`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #14570: [SPARK-10601][SQL] Support `MINUS` set operator

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14570 **[Test build #63463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63463/consoleFull)** for PR 14570 at commit [`0cf8f43`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #14475: [SPARK-16862] Configurable buffer size in `Unsafe...

2016-08-09 Thread tejasapatil
GitHub user tejasapatil reopened a pull request: https://github.com/apache/spark/pull/14475 [SPARK-16862] Configurable buffer size in `UnsafeSorterSpillReader` ## What changes were proposed in this pull request? Jira: https://issues.apache.org/jira/browse/SPARK-16862

[GitHub] spark issue #12990: [SPARK-15083] [Web UI] Limit total tasks displayed on We...

2016-08-09 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/12990 Yeah. That might be a much larger change though too. Ideally the history server is smart and can load just meta data quickly to list application, then if application actually clicked it would loa

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63462/consoleFull)** for PR 14568 at commit [`ea77d78`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63462/ Test FAILed. ---

[GitHub] spark pull request #14570: [SPARK-10601][SQL] Support `MINUS` set operator

2016-08-09 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/14570 [SPARK-10601][SQL] Support `MINUS` set operator ## What changes were proposed in this pull request? This PR adds `MINUS` set operator which is equivalent `EXCEPT DISTINCT`. This will

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63462/consoleFull)** for PR 14568 at commit [`ea77d78`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor port rest...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11157 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63450/ Test PASSed. ---

[GitHub] spark pull request #14568: [SPARK-10868] monotonicallyIncreasingId() support...

2016-08-09 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14568#discussion_r74140971 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/MonotonicallyIncreasingID.scala --- @@ -40,13 +40,14 @@ import org.apache

[GitHub] spark pull request #14475: [SPARK-16862] Configurable buffer size in `Unsafe...

2016-08-09 Thread tejasapatil
Github user tejasapatil closed the pull request at: https://github.com/apache/spark/pull/14475 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature i

[GitHub] spark issue #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor port rest...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11157 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor port rest...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11157 **[Test build #63450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63450/consoleFull)** for PR 11157 at commit [`50a34b8`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14569: Fixed typo

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14569 **[Test build #63461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63461/consoleFull)** for PR 14569 at commit [`1b17aff`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #14569: Fixed typo

2016-08-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14569 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14569: Fixed typo

2016-08-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14569 OK. Feel free to look for other typos to include in a PR like this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark issue #14552: [SPARK-16952] don't lookup spark home directory when exe...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14552 **[Test build #63460 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63460/consoleFull)** for PR 14552 at commit [`880a1f4`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #14544: [SPARK-16956] Make ApplicationState.MAX_NUM_RETRY config...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14544 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

<    1   2   3   4   5   6   7   8   >