[GitHub] spark pull request: [SPARK-3452] Maven build should skip publishin...

2014-09-10 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2329#issuecomment-55074587 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

2014-09-10 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2318#issuecomment-55075420 Hey All, Please correct me if I am wrong, my conclusion is that this also will not help because pom published is not appropriate(Since it does not have dependency

[GitHub] spark pull request: [SPARK-3325] Add a parameter to the method pri...

2014-09-10 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/spark/pull/2216#discussion_r17344850 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaDStreamLike.scala --- @@ -50,8 +50,8 @@ trait JavaDStreamLike[T, This :

[GitHub] spark pull request: [SPARK-3469] Call all TaskCompletionListeners ...

2014-09-10 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/2343 [SPARK-3469] Call all TaskCompletionListeners even if some fail This is necessary because we rely on this callback interface to clean resources up. The old behavior would lead to resource leaks.

[GitHub] spark pull request: [SPARK-2890][SQL] Allow reading of data when c...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2209#issuecomment-55076950 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL] Add test case with workaround for readin...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2340#issuecomment-55076938 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3447][SQL] Remove explicit conversion w...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2323#issuecomment-55076971 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3030] [PySpark] Reuse Python worker

2014-09-10 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2259#issuecomment-55077329 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3433][BUILD] Fix for Mima false-positiv...

2014-09-10 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2285#issuecomment-55077314 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3030] [PySpark] Reuse Python worker

2014-09-10 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2259#issuecomment-55077384 @JoshRosen The problem that will cause hanging has been fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-3395] [SQL] DSL sometimes incorrectly r...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2266#issuecomment-55077558 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2096][SQL] Correctly parse dot notation...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2230#issuecomment-55077709 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-3468] WebUI Timeline-View feature

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-55078122 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20080/consoleFull) for PR 2342 at commit

[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...

2014-09-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1823#issuecomment-55078979 Do you mind closing this? This whole code is going to be updated by https://github.com/apache/spark/pull/2330 --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-3171] Don't print meaningless informati...

2014-09-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2078#issuecomment-55079128 This no longer merges with master. Do you mind rebasing it? I moved the files into the o.a.s.network.nio package. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1846#issuecomment-55080701 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20082/consoleFull) for PR 1846 at commit

[GitHub] spark pull request: [SPARK-3171] Don't print meaningless informati...

2014-09-10 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2078#issuecomment-55080894 @rxin I've rebased. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...

2014-09-10 Thread sarutak
Github user sarutak closed the pull request at: https://github.com/apache/spark/pull/1823 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2805] akka 2.3.4

2014-09-10 Thread rbraley
Github user rbraley commented on the pull request: https://github.com/apache/spark/pull/1685#issuecomment-55081114 This is a critical patch for https://github.com/krasserm/akka-analytics/ --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-3465] fix task metrics aggregation in l...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2338#issuecomment-55082144 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20081/consoleFull)** after a configured wait of `120m`. --- If your project

[GitHub] spark pull request: [SPARK-1720][SPARK-1719] Add the value of LD_L...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-55082152 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20095/consoleFull) for PR 1031 at commit

[GitHub] spark pull request: [SPARK-3171] Don't print meaningless informati...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2078#issuecomment-55082131 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20094/consoleFull) for PR 2078 at commit

[GitHub] spark pull request: [SPARK-2096][SQL] Correctly parse dot notation...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2230#issuecomment-55082124 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20093/consoleFull) for PR 2230 at commit

[GitHub] spark pull request: [SPARK-3469] Call all TaskCompletionListeners ...

2014-09-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2343#issuecomment-55083964 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3469] Call all TaskCompletionListeners ...

2014-09-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2343#issuecomment-55083955 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2713] Executors of same application in ...

2014-09-10 Thread li-zhihui
Github user li-zhihui commented on the pull request: https://github.com/apache/spark/pull/1616#issuecomment-55084516 Thanks @chenghao-intel It seems the sql unit test failure is fixed. @andrewor14 Can you retest this again? --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-3325] Add a parameter to the method pri...

2014-09-10 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2216#discussion_r17348838 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaDStreamLike.scala --- @@ -50,8 +50,8 @@ trait JavaDStreamLike[T, This :

[GitHub] spark pull request: [SPARK-1720][SPARK-1719] Add the value of LD_L...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-55087507 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20095/consoleFull) for PR 1031 at commit

[GitHub] spark pull request: [SPARK-3171] Don't print meaningless informati...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2078#issuecomment-55088015 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20094/consoleFull) for PR 2078 at commit

[GitHub] spark pull request: [SPARK-3168]The ServletContextHandler of webui...

2014-09-10 Thread XuTingjun
Github user XuTingjun commented on the pull request: https://github.com/apache/spark/pull/2073#issuecomment-55088380 Now Spark doesn't support SSL --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-09-10 Thread allwefantasy
Github user allwefantasy commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-55089256 @witgo 看了你的性能测试 你 里面没有提到迭代次数。是多少次迭代呢?一个小时就完成了。

[GitHub] spark pull request: [SPARK-3325] Add a parameter to the method pri...

2014-09-10 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/spark/pull/2216#discussion_r1735 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaDStreamLike.scala --- @@ -50,8 +50,8 @@ trait JavaDStreamLike[T, This :

[GitHub] spark pull request: [SPARK-2096][SQL] Correctly parse dot notation...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2230#issuecomment-55093270 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20093/consoleFull) for PR 2230 at commit

[GitHub] spark pull request: [SPARK-3411] Improve load-balancing of concurr...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1106#issuecomment-55093889 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20101/consoleFull) for PR 1106 at commit

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-09-10 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-55094784 @allwefantasy 我的测试语料库大小是`196558` 个文档, `7897767` 个词. 迭代次数是`100`次. 你的24万文档总共有多个词? 你

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-09-10 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-55095034 (Pardon, I think it's important to also summarize in English, as the lingua franca of the project, for the benefit of other readers.) --- If your project is set up for

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-09-10 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-55095890 @allwefantasy 我认为这里的代码` Document(parts(0).toInt,(0 until wordInfo.value.size).map(k= values.getOrElse(k,0)).toArray)` 是有点问题的..

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-09-10 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-55096559 @srowen I will try to translate the comments into English --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-3363][SQL] Type Coercion should support...

2014-09-10 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/2246#issuecomment-55098584 The fail is in BroadcastSuite, has nothing to do with this PR... --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-3362][SQL] bug in casewhen resolve

2014-09-10 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/2245#issuecomment-55098867 failure is in pyspark... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3411] Improve load-balancing of concurr...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1106#issuecomment-55099544 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20101/consoleFull) for PR 1106 at commit

[GitHub] spark pull request: Added support for accessing secured HDFS

2014-09-10 Thread huozhanfeng
Github user huozhanfeng commented on the pull request: https://github.com/apache/spark/pull/2320#issuecomment-55102455 @pwendell @tgravescs Thanks for your advice. Yes I want to support multiple users all using different credentials and in our application scenario users

[GitHub] spark pull request: [SPARK-2207][SPARK-3272][MLLib]Add minimum inf...

2014-09-10 Thread chouqin
Github user chouqin commented on a diff in the pull request: https://github.com/apache/spark/pull/2332#discussion_r17355688 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala --- @@ -674,6 +676,45 @@ class DecisionTreeSuite extends FunSuite with

[GitHub] spark pull request: [MLLIB] SPARK-2329 Add multi-label evaluation ...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1270#issuecomment-55104337 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20102/consoleFull) for PR 1270 at commit

[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support

2014-09-10 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/2344 [SPARK-3407][SQL]Add Date type support You can merge this pull request into a Git repository by running: $ git pull https://github.com/adrian-wang/spark date Alternatively you can review

[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-55110094 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20103/consoleFull) for PR 2344 at commit

[GitHub] spark pull request: [MLLIB] SPARK-2329 Add multi-label evaluation ...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1270#issuecomment-55110210 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20102/consoleFull) for PR 1270 at commit

[GitHub] spark pull request: SPARK-3462 push down filters and projections i...

2014-09-10 Thread koeninger
GitHub user koeninger opened a pull request: https://github.com/apache/spark/pull/2345 SPARK-3462 push down filters and projections into Unions You can merge this pull request into a Git repository by running: $ git pull https://github.com/mediacrossinginc/spark SPARK-3462

[GitHub] spark pull request: [SPARK-2140] Updating heap memory calculation ...

2014-09-10 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2253#issuecomment-55115519 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-3462 push down filters and projections i...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2345#issuecomment-55117296 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-3470 [CORE] [STREAMING] Add Closeable / ...

2014-09-10 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/2346 SPARK-3470 [CORE] [STREAMING] Add Closeable / close() to Java context objects ... that expose a stop() lifecycle method. This doesn't add `AutoCloseable`, which is Java 7+ only. But it should be

[GitHub] spark pull request: Add a function that can build an EdgePartition...

2014-09-10 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/792#issuecomment-55122492 @ankurdave yes, i agree with you. like this: def toEdgePartition(sort: Boolean = true): EdgePartition[ED, VD] = { } --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-2207][SPARK-3272][MLLib]Add minimum inf...

2014-09-10 Thread chouqin
Github user chouqin commented on the pull request: https://github.com/apache/spark/pull/2332#issuecomment-55122746 @jkbradley thanks for your replies. as I replied in your comments, I have changed `minInstancePerNode` to 2 in test cases, and add one more test case to test that when a

[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-55123057 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20103/consoleFull) for PR 2344 at commit

[GitHub] spark pull request: SPARK-3470 [CORE] [STREAMING] Add Closeable / ...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2346#issuecomment-55126763 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20104/consoleFull) for PR 2346 at commit

[GitHub] spark pull request: [SPARK-2207][SPARK-3272][MLLib]Add minimum inf...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2332#issuecomment-55126783 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20105/consoleFull) for PR 2332 at commit

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-09-10 Thread allwefantasy
Github user allwefantasy commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-55129263 @witgo 那就是我犯了错误,对Document 中content 理解错了。我以为content

[GitHub] spark pull request: [SPARK-2207][SPARK-3272][MLLib]Add minimum inf...

2014-09-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/2332#discussion_r17367203 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala --- @@ -674,6 +676,45 @@ class DecisionTreeSuite extends FunSuite with

[GitHub] spark pull request: [SPARK-3160] [mllib] DecisionTree: eliminate p...

2014-09-10 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/2341#issuecomment-55131403 Failure is unrelated to this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-3470 [CORE] [STREAMING] Add Closeable / ...

2014-09-10 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/2346#issuecomment-55132817 The relevant PR: #991 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2207][SPARK-3272][MLLib]Add minimum inf...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2332#issuecomment-55135444 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20105/consoleFull) for PR 2332 at commit

[GitHub] spark pull request: [WIP][SPARK-2491] Don't handle uncaught except...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1482#issuecomment-55136558 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20106/consoleFull) for PR 1482 at commit

[GitHub] spark pull request: SPARK-3470 [CORE] [STREAMING] Add Closeable / ...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2346#issuecomment-55136925 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20104/consoleFull) for PR 2346 at commit

[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...

2014-09-10 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/2318#issuecomment-55138035 If there's no way for the maven-install-plugin to fix the existing pom (instead of generating a minimal pom with no dependencies), then I guess I don't see any other

[GitHub] spark pull request: [SPARK-1484][MLLIB] Warn when running an itera...

2014-09-10 Thread staple
GitHub user staple opened a pull request: https://github.com/apache/spark/pull/2347 [SPARK-1484][MLLIB] Warn when running an iterative algorithm on uncached data. Add warnings to KMeans, GeneralizedLinearAlgorithm, and computeSVD when called with input data that is not cached.

[GitHub] spark pull request: [SPARK-1484][MLLIB] Warn when running an itera...

2014-09-10 Thread staple
Github user staple commented on the pull request: https://github.com/apache/spark/pull/2347#issuecomment-55138304 See above where I describe how, for python RDDs, the input data is automatically cached and then deserialized via a map to an uncached RDD, requiring deserialization of

[GitHub] spark pull request: [SPARK-3040] pick up a more proper local ip ad...

2014-09-10 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1946#issuecomment-55144780 I'm not for or against the fix, but just wanted to point out that event the order the OS returns the interfaces is not really reliable. On my machine, where I constantly

[GitHub] spark pull request: [SPARK-1484][MLLIB] Warn when running an itera...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2347#issuecomment-55145287 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [WIP][SPARK-2491] Don't handle uncaught except...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1482#issuecomment-55145787 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20106/consoleFull) for PR 1482 at commit

[GitHub] spark pull request: [SPARK-3286] - Cannot view ApplicationMaster U...

2014-09-10 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2276#issuecomment-55145976 +1, looks good. Thanks @benoyantony --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-3286] - Cannot view ApplicationMaster U...

2014-09-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2276 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-10 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-55147473 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-3470 [CORE] [STREAMING] Add Closeable / ...

2014-09-10 Thread mattf
Github user mattf commented on the pull request: https://github.com/apache/spark/pull/2346#issuecomment-55149328 since spark targets java 7 8, why not just use the correct AutoCloseable? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-3470 [CORE] [STREAMING] Add Closeable / ...

2014-09-10 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2346#issuecomment-55150236 It doesn't target Java 7: https://github.com/apache/spark/blob/master/pom.xml#L113 --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-3286] - Cannot view ApplicationMaster U...

2014-09-10 Thread benoyantony
Github user benoyantony commented on the pull request: https://github.com/apache/spark/pull/2276#issuecomment-55151029 Thank you. Could one of you please add me as Contributor so that I can assign this jira to me ? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-3286] - Cannot view ApplicationMaster U...

2014-09-10 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2276#issuecomment-55152158 @pwendell can you add @benoyantony as contributor in jira. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-3430] [PySpark] [Doc] generate PySpark ...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2292#issuecomment-55152172 Weird, I've never seen Hive fail to initialize like this before. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-3430] [PySpark] [Doc] generate PySpark ...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2292#issuecomment-55152188 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-3470 [CORE] [STREAMING] Add Closeable / ...

2014-09-10 Thread mattf
Github user mattf commented on the pull request: https://github.com/apache/spark/pull/2346#issuecomment-55152257 It doesn't target Java 7: https://github.com/apache/spark/blob/master/pom.xml#L113 you're right. i hope it becomes java 7+ and we can move to AutoCloseable ---

[GitHub] spark pull request: [SPARK-3421][SQL] Allows arbitrary character i...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2291#issuecomment-55152400 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-3470 [CORE] [STREAMING] Add Closeable / ...

2014-09-10 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2346#issuecomment-55152475 `AutoCloseable` is a superinterface of `Closeable`, so, this is already true. Implementing `Closeable` is a stronger condition, strangely. --- If your project is set up

[GitHub] spark pull request: [SPARK-3362][SQL] bug in casewhen resolve

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2245#issuecomment-55152467 @JoshRosen, here's an example of the pyspark failure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3362][SQL] bug in casewhen resolve

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2245#issuecomment-55152478 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3363][SQL] Type Coercion should support...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2246#issuecomment-55152587 @JoshRosen, another flakey one in Core --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-3362][SQL] bug in casewhen resolve

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2245#issuecomment-55153460 Actually, I'm just going to merge... Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3362][SQL] bug in casewhen resolve

2014-09-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2245 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3363][SQL] Type Coercion should support...

2014-09-10 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2246#issuecomment-55154043 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Added support for accessing secured HDFS

2014-09-10 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/2320#issuecomment-55154141 So it has nothing to do with whom start the spark daemons but with whom start the driver. Well, standalone supports cluster mode. In cluster mode, the Worker

[GitHub] spark pull request: [SPARK-3363][SQL] Type Coercion should support...

2014-09-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2246 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-10 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2339#discussion_r17377106 --- Diff: ec2/spark_ec2.py --- @@ -618,6 +620,57 @@ def wait_for_cluster(conn, wait_secs, master_nodes, slave_nodes): time.sleep(wait_secs)

[GitHub] spark pull request: [SPARK-3430] [PySpark] [Doc] generate PySpark ...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2292#issuecomment-55154527 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20107/consoleFull) for PR 2292 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-55154549 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20109/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [SPARK-3421][SQL] Allows arbitrary character i...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2291#issuecomment-55154552 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20108/consoleFull) for PR 2291 at commit

[GitHub] spark pull request: [SPARK-3421][SQL] Allows arbitrary character i...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2291#issuecomment-55154697 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20108/consoleFull) for PR 2291 at commit

[GitHub] spark pull request: [SPARK-3430] [PySpark] [Doc] generate PySpark ...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2292#issuecomment-55154659 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20107/consoleFull) for PR 2292 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-55154710 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20109/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-10 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2339#discussion_r17377246 --- Diff: ec2/spark_ec2.py --- @@ -618,6 +620,57 @@ def wait_for_cluster(conn, wait_secs, master_nodes, slave_nodes): time.sleep(wait_secs)

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-09-10 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/1846#discussion_r17377308 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -47,4 +47,16 @@ class SQLQuerySuite extends QueryTest {

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-10 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2339#discussion_r17377782 --- Diff: ec2/spark_ec2.py --- @@ -60,9 +60,11 @@ def parse_args(): parser.add_option( -s, --slaves, type=int, default=1,

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-10 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2339#discussion_r17377820 --- Diff: ec2/spark_ec2.py --- @@ -872,7 +925,14 @@ def real_main(): (master_nodes, slave_nodes) = get_existing_cluster(conn, opts,

  1   2   3   4   >