[GitHub] spark pull request #15843: [SPARK-18274] Memory leak in PySpark StringIndexe...

2016-11-10 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/15843 [SPARK-18274] Memory leak in PySpark StringIndexer ## What changes were proposed in this pull request? Make Java Gateway dereference object in destructor, using `SparkContext._gateway.detach

[GitHub] spark pull request #15800: [SPARK-18334] MinHash should use binary hash dist...

2016-11-10 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15800#discussion_r87429950 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -76,7 +72,19 @@ class MinHashModel private[ml] ( @Since("2.1.0")

[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...

2016-11-10 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 @vijoshi please look at test failures in more detail to see if these changes could have affected. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15809: [SPARK-18268][ML][MLLib] ALS fail with better message if...

2016-11-10 Thread aditya1702
Github user aditya1702 commented on the issue: https://github.com/apache/spark/pull/15809 @srowen Ok. Thanks 👍 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark issue #15830: [SPARK-18375][SPARK-18383][BUILD][CORE]Upgrade netty to ...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68470/ Test PASSed. ---

[GitHub] spark issue #15830: [SPARK-18375][SPARK-18383][BUILD][CORE]Upgrade netty to ...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15830 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15830: [SPARK-18375][SPARK-18383][BUILD][CORE]Upgrade netty to ...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15830 **[Test build #68470 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68470/consoleFull)** for PR 15830 at commit [`c4546dc`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/15840 @hvanhovell Thank for pointing out `NullIntolerant`. I did not know it. I also provide it as a starting point based on two instances that I saw today. I agree that it would be good to add `NullInto

[GitHub] spark issue #15809: [SPARK-18268][ML][MLLib] ALS fail with better message if...

2016-11-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15809 Python API calls the Scala API so I think this is sufficient. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68471/ Test FAILed. ---

[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68471/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15809: [SPARK-18268][ML][MLLib] ALS fail with better message if...

2016-11-10 Thread aditya1702
Github user aditya1702 commented on the issue: https://github.com/apache/spark/pull/15809 What about the python code? this was for scala only right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14124: [SPARK-16472][SQL] Inconsistent nullability in schema af...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14124 **[Test build #68477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68477/consoleFull)** for PR 14124 at commit [`d240c0d`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #15838: [SPARK-18396][HISTORYSERVER]"Duration" column makes sear...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15838 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68469/ Test PASSed. ---

[GitHub] spark issue #15838: [SPARK-18396][HISTORYSERVER]"Duration" column makes sear...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15838 **[Test build #68469 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68469/consoleFull)** for PR 15838 at commit [`d014892`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15838: [SPARK-18396][HISTORYSERVER]"Duration" column makes sear...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15838 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15840 @kiszk that could be a start. However I would like to take a step back, and see which nullability checking patterns are common, and provide generic utilities for them. I also think that the prope

[GitHub] spark issue #14124: [SPARK-16472][SQL] Inconsistent nullability in schema af...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14124 **[Test build #68474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68474/consoleFull)** for PR 14124 at commit [`7306937`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-10 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/15644 @kayousterhout I've updated again, and did another pass for a little more cleanup. I think it should be OK now. sorry the earlier update left out some things. --- If your project is set up for it

[GitHub] spark issue #14124: [SPARK-16472][SQL] Inconsistent nullability in schema af...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14124 Actually, nvm. I think handling this in `DataFrameReader.schema` will deal with most of general cases. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark issue #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15644 **[Test build #68476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68476/consoleFull)** for PR 15644 at commit [`ff08d8c`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #15842: [SPARK-18401][SPARKR][ML] SparkR random forest should su...

2016-11-10 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15842 cc @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15842: [SPARK-18401][SPARKR][ML] SparkR random forest should su...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15842 **[Test build #68475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68475/consoleFull)** for PR 15842 at commit [`61f25b5`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14124: [SPARK-16472][SQL] Inconsistent nullability in schema af...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14124 Oh wait, @cloud-fan, it seems, at least, Parquet files could possibly be written with not nullable fields. So, reading it without user-specified schema might also cause the inconsistency between

[GitHub] spark pull request #15842: [SPARK-18401][SPARKR][ML] SparkR random forest sh...

2016-11-10 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/15842 [SPARK-18401][SPARKR][ML] SparkR random forest should support output original label. ## What changes were proposed in this pull request? SparkR ```spark.randomForest``` classification predic

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15172 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15172 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68467/ Test PASSed. ---

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15172 **[Test build #68467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68467/consoleFull)** for PR 15172 at commit [`cb93e18`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14124: [SPARK-16472][SQL] Inconsistent nullability in schema af...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14124 Thanks @cloud-fan, sure, that sounds great. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark issue #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-10 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15644 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark issue #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15644 **[Test build #68473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68473/consoleFull)** for PR 15644 at commit [`2fbd841`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-11-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/15219 @sameeragarwal did you have any change to take a closer look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request #15806: [SPARK-18345][STRUCTURED STREAMING] Structured St...

2016-11-10 Thread oza
Github user oza closed the pull request at: https://github.com/apache/spark/pull/15806 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] spark issue #15806: [SPARK-18345][STRUCTURED STREAMING] Structured Streaming...

2016-11-10 Thread oza
Github user oza commented on the issue: https://github.com/apache/spark/pull/15806 This was my mistake of configuration. After clearing the configuration about Hadoop, it works well. Closing this as not a problem. Thanks all for helping me, reviewing patch! --- If your project is se

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13909 **[Test build #68468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68468/consoleFull)** for PR 13909 at commit [`6941b58`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14719: [SPARK-17154][SQL] Wrong result can be returned or Analy...

2016-11-10 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/14719 @cloud-fan, I was studying the ResolveSubquery code for my work on SPARK-17348. I was first puzzle about the code in `def rewriteSubQuery` // Make sure the inner and the outer query attr

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68468/ Test FAILed. ---

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/15840 Do we prepare a utility method like this in `CodeGenerator`? ```java def genNullableAssignment(genIsNull: boolean, isNull: boolean, lhs: String, rhs: String): String = { if (genIsNull) {

[GitHub] spark issue #15806: [SPARK-18345][STRUCTURED STREAMING] Structured Streaming...

2016-11-10 Thread oza
Github user oza commented on the issue: https://github.com/apache/spark/pull/15806 I'm sorry, but I had a hadoop configuration on my local: I noticed it on PR about SPARK-18345. I will check whether it happens after clearing configuration. --- If your project is set up for it, you

[GitHub] spark issue #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in SparkSQL/D...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15841 +1 for no problem :). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in SparkSQL/D...

2016-11-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15841 No problem. Thanks for the effort! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread oza
Github user oza closed the pull request at: https://github.com/apache/spark/pull/15841 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] spark issue #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in SparkSQL/D...

2016-11-10 Thread oza
Github user oza commented on the issue: https://github.com/apache/spark/pull/15841 I'm sorry, it's completely --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in SparkSQL/D...

2016-11-10 Thread oza
Github user oza commented on the issue: https://github.com/apache/spark/pull/15841 @srowen it makes sense to me. In that case, we can update our documentation to avoid the problem. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in SparkSQL/D...

2016-11-10 Thread oza
Github user oza commented on the issue: https://github.com/apache/spark/pull/15841 Thanks a lot for your comments! I see, in fact, I had the problem on Mac OS. Okay, I will try it on your VM, and on my local linux. Please wait a moment. --- If your project is set up for it, you can r

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15841#discussion_r87404895 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala --- @@ -60,7 +60,8 @@ object SparkSQLExample { private

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15840 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15840 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68466/ Test PASSed. ---

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15840 **[Test build #68466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68466/consoleFull)** for PR 15840 at commit [`b287ded`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in SparkSQL/D...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15841 As far as I know, the default file system should be `file:///`. I created a virtual machine with the OS `Ubuntu Server Trusty 14.04 amd64`, from the link https://oss-binaries.phusionpas

[GitHub] spark issue #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in SparkSQL/D...

2016-11-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15841 We want the examples to be consistent, so wouldn't this affect all examples? And yes they're supposed to be run with the given script. Putting the path in the source code might help a bit for p

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15780 **[Test build #68472 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68472/consoleFull)** for PR 15780 at commit [`38991d0`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15841#discussion_r87402224 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala --- @@ -60,7 +60,8 @@ object SparkSQLExample { private

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15841#discussion_r87401962 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala --- @@ -60,7 +60,8 @@ object SparkSQLExample { private

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-11-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/15780 @cloud-fan , this optimization leads to failure of the program `Seq((0, 1.0, 3.0), (2, 2.0, 5.0)).toDF("id", "v").printSchema` . While the previous code returns `false`, this optimization returns `

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15840 On a whole this PR looks good. On a more general level, I would like to suggest that we should have a more generic way of preventing unneeded null-checks. --- If your project is set up for it,

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread oza
Github user oza commented on a diff in the pull request: https://github.com/apache/spark/pull/15841#discussion_r87399578 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala --- @@ -60,7 +60,8 @@ object SparkSQLExample { private def run

[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68471/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...

2016-11-10 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...

2016-11-10 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 I've seen other pr tests passing now, let try again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...

2016-11-10 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15803 ah, ok I did misunderstand, I thought you were talking changing timezone based on browser local time. --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread oza
Github user oza commented on a diff in the pull request: https://github.com/apache/spark/pull/15841#discussion_r87397730 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala --- @@ -60,7 +60,8 @@ object SparkSQLExample { private def run

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread oza
Github user oza commented on a diff in the pull request: https://github.com/apache/spark/pull/15841#discussion_r87396970 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala --- @@ -60,7 +60,8 @@ object SparkSQLExample { private def run

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15841#discussion_r87396243 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala --- @@ -60,7 +60,8 @@ object SparkSQLExample { private

[GitHub] spark issue #15806: [SPARK-18345][STRUCTURED STREAMING] Structured Streaming...

2016-11-10 Thread oza
Github user oza commented on the issue: https://github.com/apache/spark/pull/15806 Updating patch based on kiszk's comment: https://github.com/apache/spark/pull/15806#discussion_r87393524 --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #15806: [SPARK-18345][STRUCTURED STREAMING] Structured St...

2016-11-10 Thread oza
Github user oza commented on a diff in the pull request: https://github.com/apache/spark/pull/15806#discussion_r87394293 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala --- @@ -209,7 +209,7 @@ class StreamingQueryManager private[sql] (s

[GitHub] spark issue #15833: [SPARK-18353][CORE] spark.rpc.askTimeout defalut value i...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15833 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15833: [SPARK-18353][CORE] spark.rpc.askTimeout defalut value i...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15833 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68465/ Test PASSed. ---

[GitHub] spark issue #15830: [SPARK-18375][SPARK-18383][BUILD][CORE]Upgrade netty to ...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15830 **[Test build #68470 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68470/consoleFull)** for PR 15830 at commit [`c4546dc`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15833: [SPARK-18353][CORE] spark.rpc.askTimeout defalut value i...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15833 **[Test build #68465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68465/consoleFull)** for PR 15833 at commit [`55cd875`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15838: [SPARK-18396]"Duration" column makes search result confu...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15838 **[Test build #68469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68469/consoleFull)** for PR 15838 at commit [`d014892`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #15806: [SPARK-18345][STRUCTURED STREAMING] Structured St...

2016-11-10 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15806#discussion_r87393524 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala --- @@ -209,7 +209,7 @@ class StreamingQueryManager private[sql]

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-11-10 Thread cjjnjust
Github user cjjnjust commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r87389888 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesConfigMessage.java --- @@ -0,0 +1,104 @@ +/* + * Licensed to the

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-11-10 Thread cjjnjust
Github user cjjnjust commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r87389461 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,305 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13909 **[Test build #68468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68468/consoleFull)** for PR 13909 at commit [`6941b58`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15172 **[Test build #68467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68467/consoleFull)** for PR 15172 at commit [`cb93e18`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15827: [SPARK-18187][STREAMING] CompactibleFileStreamLog should...

2016-11-10 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/15827 cc @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15835: [SPARK-17059][SQL] Allow FileFormat to specify partition...

2016-11-10 Thread pwoody
Github user pwoody commented on the issue: https://github.com/apache/spark/pull/15835 Ah awesome thanks - my IDE settings must not be properly catching the style - will fix and make sure this doesn't happen in the future. Also just took a look at the bucketing. I've implemente

[GitHub] spark pull request #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in Spa...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15841#discussion_r87384086 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala --- @@ -60,7 +60,8 @@ object SparkSQLExample { private

[GitHub] spark pull request #15835: [SPARK-17059][SQL] Allow FileFormat to specify pa...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15835#discussion_r87382447 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -272,6 +275,83 @@ class ParquetFileF

[GitHub] spark pull request #15835: [SPARK-17059][SQL] Allow FileFormat to specify pa...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15835#discussion_r87382135 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala --- @@ -75,6 +76,17 @@ trait FileFormat { }

[GitHub] spark pull request #15835: [SPARK-17059][SQL] Allow FileFormat to specify pa...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15835#discussion_r87382199 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -272,6 +275,83 @@ class ParquetFileF

[GitHub] spark issue #15797: [SPARK-17990][SPARK-18302][SQL] correct several partitio...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15797 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15797: [SPARK-17990][SPARK-18302][SQL] correct several partitio...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68461/ Test PASSed. ---

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15840 **[Test build #68466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68466/consoleFull)** for PR 15840 at commit [`b287ded`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15797: [SPARK-17990][SPARK-18302][SQL] correct several partitio...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15797 **[Test build #68461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68461/consoleFull)** for PR 15797 at commit [`0a15edc`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14649: [SPARK-17059][SQL] Allow FileFormat to specify partition...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14649 @andreweduffy, are you fine with #15835? If so, we might be able to close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark issue #15835: [SPARK-17059][SQL] Allow FileFormat to specify partition...

2016-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15835 Ah, I missed the last comment from the old PR. Okay, we can make this shaped nicer. BTW, Spark collects small partitions for each task so I guess this would not introduce a lot of tasks always b

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/15840 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15840 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15840 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68462/ Test FAILed. ---

[GitHub] spark issue #15841: [SPARK-18399][SQL][DOCUMENTATION] Examples in SparkSQL/D...

2016-11-10 Thread oza
Github user oza commented on the issue: https://github.com/apache/spark/pull/15841 Before the patch: ``` scala> val df = spark.read.json("examples/src/main/resources/people.json") 16/11/10 21:02:59 WARN datasources.DataSource: Error while looking for metadata directory.

[GitHub] spark issue #15840: [SPARK-18398][SQL] Fix nullabilities of MapObjects and o...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15840 **[Test build #68462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68462/consoleFull)** for PR 15840 at commit [`b287ded`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68463/ Test FAILed. ---

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13909 **[Test build #68463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68463/consoleFull)** for PR 13909 at commit [`3ae0da7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15620 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15620 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68464/ Test FAILed. ---

<    1   2   3   4   5   >