[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15607 **[Test build #67441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67441/consoleFull)** for PR 15607 at commit

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15607 **[Test build #67441 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67441/consoleFull)** for PR 15607 at commit

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15607 Hmm, it works fine with `roxygen2 "roxygen2" "/usr/local/lib/R/site-library" "5.0.1"` maybe it is the older version on Jenkins? ``` First time using roxygen2 4.0.

[GitHub] spark pull request #15605: [WIP] [SPARK-18067] [SQL] SortMergeJoin adds shuf...

2016-10-24 Thread tejasapatil
Github user tejasapatil closed the pull request at: https://github.com/apache/spark/pull/15605 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67429/ Test PASSed. ---

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67430/ Test PASSed. ---

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15428 **[Test build #67429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67429/consoleFull)** for PR 15428 at commit

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15428 **[Test build #67430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67430/consoleFull)** for PR 15428 at commit

[GitHub] spark issue #15608: [SPARK-17838][SparkR] Check named arguments for options ...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15608 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67440/ Test PASSed. ---

[GitHub] spark issue #15608: [SPARK-17838][SparkR] Check named arguments for options ...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15608 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15608: [SPARK-17838][SparkR] Check named arguments for options ...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15608 **[Test build #67440 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67440/consoleFull)** for PR 15608 at commit

[GitHub] spark issue #15597: [SPARK-18063][SQL] Failed to infer constraints over mult...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15597 **[Test build #67433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67433/consoleFull)** for PR 15597 at commit

[GitHub] spark issue #15597: [SPARK-18063][SQL] Failed to infer constraints over mult...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15597 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67433/ Test PASSed. ---

[GitHub] spark issue #15597: [SPARK-18063][SQL] Failed to infer constraints over mult...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15597 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15609 Hm, does this cause a actual problem? I thought this is covered in type-widening. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #13891: [SPARK-6685][MLLIB]Use DSYRK to compute AtA in ALS

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13891 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13891: [SPARK-6685][MLLIB]Use DSYRK to compute AtA in ALS

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13891 **[Test build #67435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67435/consoleFull)** for PR 13891 at commit

[GitHub] spark issue #13891: [SPARK-6685][MLLIB]Use DSYRK to compute AtA in ALS

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13891 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67435/ Test PASSed. ---

[GitHub] spark issue #15606: [SPARK-18070][SQL] binary operator should not consider n...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15606 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15606: [SPARK-18070][SQL] binary operator should not consider n...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15606 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67437/ Test PASSed. ---

[GitHub] spark issue #15606: [SPARK-18070][SQL] binary operator should not consider n...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15606 **[Test build #67437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67437/consoleFull)** for PR 15606 at commit

[GitHub] spark issue #15608: [SPARK-17838][SparkR] Check named arguments for options ...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15608 **[Test build #67440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67440/consoleFull)** for PR 15608 at commit

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15607 **[Test build #67442 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67442/consoleFull)** for PR 15607 at commit

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67442/ Test FAILed. ---

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15607 **[Test build #67442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67442/consoleFull)** for PR 15607 at commit

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15541 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67428/ Test PASSed. ---

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15541 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15053: [SPARK-18069][Doc] improve python API docstrings

2016-10-24 Thread mortada
Github user mortada commented on the issue: https://github.com/apache/spark/pull/15053 @HyukjinKwon I've created a JIRA ticket and also went through all the files you mentioned, please take a look --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15607 **[Test build #67443 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67443/consoleFull)** for PR 15607 at commit

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67443/ Test PASSed. ---

[GitHub] spark issue #15566: [SPARK-18026][SQL] should not always lowercase partition...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15566 **[Test build #67439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67439/consoleFull)** for PR 15566 at commit

[GitHub] spark issue #15608: [SPARK-17838][SparkR] Check named arguments for options ...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15608 cc @felixcheung I recall we talked about this before. I first wanted to handle all the argument type checking but I just decided to do what I am pretty sure of (I remember we were concern of

[GitHub] spark pull request #15608: [SPARK-17838][SparkR] Check named arguments for o...

2016-10-24 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/15608 [SPARK-17838][SparkR] Check named arguments for options and use formatted R friendly message from JVM exception message ## What changes were proposed in this pull request? This PR

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67441/ Test FAILed. ---

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15541 **[Test build #67428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67428/consoleFull)** for PR 15541 at commit

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15607 **[Test build #67443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67443/consoleFull)** for PR 15607 at commit

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15609 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15609: [SPARK-18048][SQL] To make behaviour of If consis...

2016-10-24 Thread priyankagargnitk
GitHub user priyankagargnitk opened a pull request: https://github.com/apache/spark/pull/15609 [SPARK-18048][SQL] To make behaviour of If consistent, in case of true expression and false expression are of compatible data types. ## What changes were proposed in this pull request?

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15053: [SPARK-18069][Doc] improve python API docstrings

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15053 @srowen Do you mind if I ask to trigger the build via Jenkins if it looks reasonable (or maybe "ok to test")? Let me try to take a look closer but I would like @holdenk also to take a look

[GitHub] spark issue #15606: [SPARK-18070][SQL] binary operator should not consider n...

2016-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15606 This bug only affects the complex type, (`ArrayType`, `StructType`, and `MapType`)? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread priyankagargnitk
Github user priyankagargnitk commented on the issue: https://github.com/apache/spark/pull/15609 actually, its not covered in type widening... So, if any of my expression is calling if expression ( in nested) .. its failing because of this issue. --- If your project is set up for it,

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15553 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15553 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67436/ Test PASSed. ---

[GitHub] spark issue #15566: [SPARK-18026][SQL] should not always lowercase partition...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15566 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67439/ Test PASSed. ---

[GitHub] spark issue #15566: [SPARK-18026][SQL] should not always lowercase partition...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15566 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r84640346 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -93,15 +91,15 @@ private[spark] class SparkUI private ( /** Stop the server

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r84640931 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -135,7 +133,7 @@ private[spark] class SparkUI private ( private[spark] abstract

[GitHub] spark issue #10162: [SPARK-11250] [SQL] Generate different alias for columns...

2016-10-24 Thread NarineK
Github user NarineK commented on the issue: https://github.com/apache/spark/pull/10162 I am trying different ways to solve the problem without renaming the columns and it seems that a better place to change the column names would be here:

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67445/consoleFull)** for PR 15377 at commit

[GitHub] spark pull request #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15377#discussion_r84643899 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2508,6 +2508,27 @@ private[spark] object Utils extends Logging { } }

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread priyankagargnitk
Github user priyankagargnitk commented on the issue: https://github.com/apache/spark/pull/15609 Actually our use case is little different, we are not invoking it with the Select queries.. We are using JS to let user type expressions and then we have created out own layer on top of

[GitHub] spark pull request #15598: [SPARK-18027][YARN] .sparkStaging not clean on RM...

2016-10-24 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/15598#discussion_r84643826 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1059,9 +1059,11 @@ private[spark] class Client( } catch {

[GitHub] spark issue #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-24 Thread jaceklaskowski
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/15603 The build has just proved that my thinking was correct. I think I'll propose few other (more agressive) changes to clean the code up a little bit more. I'd appreciate any comments (or

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15553 **[Test build #67436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67436/consoleFull)** for PR 15553 at commit

[GitHub] spark issue #15568: [SPARK-18028][SQL] simplify TableFileCatalog

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15568 **[Test build #67444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67444/consoleFull)** for PR 15568 at commit

[GitHub] spark issue #15566: [SPARK-18026][SQL] should not always lowercase partition...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15566 **[Test build #67439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67439/consoleFull)** for PR 15566 at commit

[GitHub] spark issue #15588: [SPARK-18039][Scheduler] fix bug maxRegisteredWaitingTim...

2016-10-24 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/15588 I think this fix cannot really handle this imbalance receiver allocation problem, also blindly waste the CPU time. What @lw-lin mentioned is a feasible solution to wait for executors to

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15513 @HyukjinKwon @gatorsmile on the question just above, I'm neutral. I don't think either style is particularly better. All else equal, leave it as is, as we've got a ton of changes in flight here.

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15609 @priyankagargnitk Could I please ask some codes which I can reproduce this issue? It seems fine the sql below: ```sql spark-sql> SELECT if(true, cast('2014-01-01' as date),

[GitHub] spark pull request #15598: [SPARK-18027][YARN] .sparkStaging not clean on RM...

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15598#discussion_r84644113 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1059,9 +1059,11 @@ private[spark] class Client( } catch {

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15609 Then, wouldn't the responsibility of type coercion be your own layer? I guess this PR virtually mean every expression handled in `TypeCoercion` should be fixed too. For example, I guess

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67468/consoleFull)** for PR 15515 at commit

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67468/ Test PASSed. ---

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in the UI

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15441 **[Test build #67471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67471/consoleFull)** for PR 15441 at commit

[GitHub] spark pull request #15463: [SPARK-17894] [CORE] Ensure uniqueness of TaskSet...

2016-10-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15463 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #12358: [SPARK-14597] [Streaming] Streaming Listener timing metr...

2016-10-24 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/12358 @agsachin Any thoughts on @ScrapCodes comment? There are unrelated code changes and best to remove. And if you are unable to make these change, mind closing the PR? --- If your project is

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-24 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15535 @tdas could you close this PR? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15607 **[Test build #67476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67476/consoleFull)** for PR 15607 at commit

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #12195: [Spark-14300][Docs][MLLIB]Scala MLlib examples co...

2016-10-24 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12195#discussion_r84809994 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLogisticRegression.scala --- @@ -1,75 +0,0 @@ -/* - * Licensed to the

[GitHub] spark pull request #12195: [Spark-14300][Docs][MLLIB]Scala MLlib examples co...

2016-10-24 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12195#discussion_r84810043 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/TallSkinnySVD.scala --- @@ -1,62 +0,0 @@ -/* - * Licensed to the Apache Software

[GitHub] spark pull request #12195: [Spark-14300][Docs][MLLIB]Scala MLlib examples co...

2016-10-24 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12195#discussion_r84809881 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegression.scala --- @@ -1,73 +0,0 @@ -/* - * Licensed to the

[GitHub] spark pull request #12195: [Spark-14300][Docs][MLLIB]Scala MLlib examples co...

2016-10-24 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12195#discussion_r84810032 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/TallSkinnyPCA.scala --- @@ -1,62 +0,0 @@ -/* - * Licensed to the Apache Software

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15596 **[Test build #67480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67480/consoleFull)** for PR 15596 at commit

[GitHub] spark issue #12257: [SPARK-14483][WEBUI] Display user name for each job and ...

2016-10-24 Thread sarutak
Github user sarutak commented on the issue: https://github.com/apache/spark/pull/12257 @ajbozarth @tdas Thanks for your interest. I'll rebase this to master and reflect the comments. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15566: [SPARK-18026][SQL] should not always lowercase pa...

2016-10-24 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15566#discussion_r84816423 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -370,17 +370,6 @@ class InsertIntoHiveTableSuite extends

[GitHub] spark issue #13259: [SPARK-15480][UI][Streaming]show missed InputInfo in str...

2016-10-24 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/13259 @zsxwing Could you take a look at this PR. Would be good to merge this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84789919 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -528,13 +581,41 @@ public SparkAppHandle

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84789427 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -73,6 +74,14 @@ /** Logger name to use when launching a child

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84791881 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1186,9 +1209,13 @@ private[spark] class Client( }

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84788565 --- Diff: launcher/src/main/java/org/apache/spark/launcher/LauncherProtocol.java --- @@ -32,6 +32,9 @@ /** Environment variable where the secret for

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84782803 --- Diff: core/src/main/scala/org/apache/spark/SparkApp.scala --- @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84789538 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -107,6 +123,27 @@ public static void setConfig(String name, String

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84785459 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -710,7 +723,11 @@ object SparkSubmit { printWarning("Subclasses of

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84789777 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -107,6 +123,27 @@ public static void setConfig(String name, String

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84787477 --- Diff: launcher/src/main/java/org/apache/spark/launcher/AbstractSparkAppHandle.java --- @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84788125 --- Diff: launcher/src/main/java/org/apache/spark/launcher/ChildProcAppHandle.java --- @@ -103,67 +46,22 @@ public synchronized void kill() { try

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84791767 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -645,11 +664,13 @@ private[spark] class Client( remoteFs,

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84790527 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitRunner.java --- @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84786490 --- Diff: core/src/main/scala/org/apache/spark/launcher/LauncherBackend.scala --- @@ -17,38 +17,71 @@ package org.apache.spark.launcher

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84785044 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -710,7 +723,11 @@ object SparkSubmit { printWarning("Subclasses of

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84791332 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -56,23 +56,38 @@ import org.apache.spark.internal.config._ import

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84788429 --- Diff: launcher/src/main/java/org/apache/spark/launcher/ChildThreadAppHandle.java --- @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-10-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r84782648 --- Diff: core/src/main/scala/org/apache/spark/SparkApp.scala --- @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

  1   2   3   4   5   6   7   >